2 Comments
User's avatar
Kathleen Moriarty's avatar

Thanks for sharing your findings! I have had similar results asking for APA formatted references of specific documents, such as NIST CSF with incorrect authors coming back in the results. I appreciate you running the tests periodically to look for improvements in capabilities.

Expand full comment
Karen Scarfone's avatar

Thanks, Kathleen! So far I've tested five chatbots and found significant differences in their performance. It will be interesting to see how their performance improves (or declines!) over time. It'll also be interesting to try different prompts. At the moment I'm acting as a newbie to GenAI usage, which is pretty accurate for me.

Expand full comment