I wrote a post last month about prompting Claude for sources on a topic, getting fabricated sources, asking Claude why it fabricated sources, and then asking it for a better prompt that would get me unfabricated sources. My question to my loyal reader(s) was, Should I believe the reasons Claude gave me for why it fabricated sources, and should I trust its "improved" prompt? (Well, an easy way to test the latter question would be to try it out, but for some reason I haven't done that yet!)
My loyal reader who shares the excellent name of Jonathan said "no" to both of those questions, and this post on the Anthropic website (they're the makers of Claude, in case you don't know) seems to agree with him (h/t Leon Furze). If I understand this article correctly, whether or not Claude's explanation for why it fabricates is plausible (I probably shouldn't even call it an explanation, at least not in the traditional sense of the word), and whether or not the AI-generated prompt is useful (I suppose I should test it sometime), there's no intentionality behind the texts generated. That is, Claude isn't actually "trying" to give me a good prompt that will result in better sources, even if it in effect does. And if Claude gives an accurate representation of its "thought" process, it's an accidental result of whatever predictions it makes about what pieces of text would go together to generate something related to the words in the prompt.
Hmmm...
No comments:
Post a Comment