eh, what the hell

or: Sev's Adventures With ChatGPT

This is not an article defending AI scraping programs in any way. I find their existence worrisome and exploitative. However, while reading what's going down at AO3 with regards to AI-generated fanworks - more specifically, writing - I realized I actually don't know a lot about what these kinds of things actually produce.

I am, of course, teh lady who thought the band was actually called One Directional, so. No wonder.

But I decided I would take a little look and see what the concerns are with regards to written fanworks. (Again, this is in NO way defending the way these programs were trained on the backs of content written by others. That's separate.) I wanted to see what a fanwork might look like coming out of one of these guys.

I wanted to see how - threatening? - AI-generated content is at the moment only in terms of comparison to other fanfictions. Again, this says nothing about legal use or existence as a transformative work. I wanted to see if I could make it make me a fanfiction.

So if you, too, were curious about it, but didn't want to use the AI guy / were too lazy to do so, here's my short 15 minute experiment.

I figured I'd start somewhere I've been a while: Winterhawk fandom. So I said:

Help me write a story where Bucky Barnes and Clint Barton get together while working at a restaurant and eventually kiss.

You might recognize this premise from the garlic fic.

( Here's what it gave me: )

So on one hand, I was expecting the writing to be terrible, and it isn't. But this reads more like the summary of a story than an actual story, right? Plus, the phrasings are ultimately cliche: the soft glow of the moonlight is easily predictable text because it's been used thousands of times. This isn't bad, but it isn't something I see as being at all popular over other fic offerings.

But, I thought, maybe I'm using it wrong. So I made the sacrifice and put in the first 3 paragraphs of Old Vines. Yes, ChatGPT now has the first 3 paragraphs of Old Vines in its database. I'm sorry / You're welcome.

( and )

It's basically... a summary of the first three paragraphs. Rewritten.

So I put in the 4th paragraph.

( next )

Again. It's just rewriting my words.

So then I asked it:

what happens when Aziraphale lands in wine country

mainly because I was curious whether it would find Old Vines itself somewhere, or like, idk. I was just curious.

( So it gave me this: )

What a lovely story that would have been! Y'all never would have had to deal with Gabriel.

and THEN i thought, well, OV is a relatively small fanfic i'm using because it's mine. I wonder what happens if I ask:

what happens when he meets Crowley

( And )

Note the mention of "the apocalypse" and "angel and demon" -- the addition of Crowley to the mix has led it back to Good Omens source material. None of this is in OV, which is an AU, and way way way way way WAY smaller than Good Omens as a whole, obviously -- but it didn't pick it up from those paragraphs. It knows Crowley and Aziraphale stopped the apocalypse.

And it's also still very much a summary of something. It isn't... it isn't writing what I would call fanfiction.

So what am I saying here? Nothing yet. I know a lot about writing; I know very little about AI.

I find its general coherency surprising and worrisome -- I'm so cynical that all I can see are the ways people could use it to steal and rewrite work, or fuck stuff up, but I'm sure there are cool things about it too. I don't think it's yet a challenger to work on AO3 written by humans.

I don't know what else to say about it but I wrote this whole thing, so like, idk man.