I’ve messed around with AI a little, because I like to understand a thing when I’m formulating opinions on it and I’ve found the best way to do that is to climb into the thing and start pushing buttons. I’ve been unimpressed with ChatGPT; its fiction chops are oddly saccharine, and of course you can’t trust the nonfiction it spits out because everything it says needs to be fact checked. It claims it can’t “lie”, only misinterpret data from the sets it was trained on; devs call the misinterpreted data “hallucination”.
Part of what I was testing was whether it could competently do my job, so that if my boss starts asking about it, I can give him a decent report. I asked it for biographies of a few wealthy people I’ve researched in the past, and it spat out some respectably generic information that was mostly correct. However, most of the people I research now are not like “so rich I’m famous for it” levels of wealthy, and harder generally for me to find information on, as I assumed it would be for ChatGPT.
So I thought I’d see what it could do with someone more middle class, and asked it for a biography of Sam Starbuck.
What it returned was like what you would get if you told me “Write a flattering biography of yourself and don’t worry that I’ll be fact-checking anything you say.” It was mostly true, but it hyped up my achievements as an author in ways that I would consider not entirely honest, and said I was also a professional editor and that I had led prestigious writing workshops in the past. That’s plain untrue, but I can see where it would be making that assumption, because my author bio sounds like a lot of other, more famous peoples’ author bios, and I would guess it just pulled in some of their verbiage for color.
But the wildest part of the bio was that it named three of the novels I’ve written. Or rather, it named two novels I’ve written and one novel that I definitely have not. It said I was the author of a novel called “Like Clockwork”. Just in case I had written a fanfic titled “Like Clockwork” and forgotten about it, I checked AO3 and also asked ChatGPT for a plot summary of Sam Starbuck’s “Like Clockwork”. And sure enough it hallucinated a multi-paragraph summary of an entire novel I’ve never written, on AO3 or anywhere else. (It was not a good summary. Very Generic YA SF Thriller.)
ChatGPT is very good at one thing: apologizing. When I pressed it about where it found the data it couldn’t say, when I asked why it had made up the plot summary it couldn’t tell me, when I asked if it could show me source links or data it drew on to create “Like Clockwork” it of course would not. But it always said it was sorry…
Anyway, my best theory from googling is that every year there are roughly nine million news stories about how Starbucks Coffee’s holiday cups are back “like clockwork”.
I suppose I should be glad the novel’s title isn’t Unicorn Frappucino.