“I once had ChatGPT insist that a particular composer wrote music for a game, even going so far as to list particular songs from the soundtrack that they were supposedly responsible for, and it helpfully provided hallucinatory citations when I asked for them (a broken link on the game publisher’s website and a link to Wikipedia, which did not in fact support its assertion either now or at any point in the article’s history). Nor could I find anywhere else on the internet where someone even mistakenly believed that that composer had worked on the game. ChatGPT lies not because it’s regurgitating falsehoods that it found on the internet - it lies because it invents new falsehoods on its own. It’s not just trained on stuff on the internet that’s wrong; it’s trained to be confidently wrong in general. It doesn’t know what facts are, it just knows how to produce things that are shaped like facts and shove them in fact-shaped holes. I personally wasted 30 minutes of my life fact-checking/“not believing everything it says”, when it confidently told me something surprising. My horizons were not broadened by exposing me to “different worldviews”. This was unequivocally a negative experience for me.”
— comment on a MetaFilter post about AI: “My goal is to be helpful, harmless, and honest.”
Source: metafilter.com
