Fact Checking the AI

Using known good values to verify AI-generated content

In previous posts [1][2][3], Karen has highlighted the dangers of using generative AI as a knowledge source. As many users have seen, the AI “hallucinates” responses (NIST refers to this phenomenon as “confabulation”). That is to say, the information that is returned to the user is not factually correct. For this article we will use the common term "hallucination” without quotes; just know that we understand there is a lot of nuance here.

Hallucinations are a problem in any use case that is not purely writing fiction. There are two distinct errors which arise with hallucinations:

1. They miss something (don’t tell the whole story).

2. They make up something plausible (a CSF 2.0 Category called “PR.TR”).

In the security world, handling both types of errors is critical to maintaining confidentiality, integrity, and availability. Therefore, using any tool which would endanger those critical pillars would be a risky endeavor. However, users are still turning to generative AI models, solutions, and agents to get advice, handle tasks, and manage workloads. This is an all too common tension in the fight between using technology to generate value versus the risk of using that technology.

In order to mitigate the “hallucination” risks, we have developed tools to help. First and foremost, the TCAnnex API is a fantastic way to get to the bedrock of the issue and have known good information, verified by humans. Without a known good anchor to fall back on, no process (human or machine) will be able to guard against conceptual drift, feature creep, or bloat.

However, an API by itself is not sufficient to mitigate risk; further tooling must be developed. A brief summary of the tools is below:

TCAnnex API Driver

This is a small python library which wraps the functionality of the API into a set of simple programming calls. The library can take in an API key as a parameter or will default to whatever is in the environment variable TCANNEX_API_KEY. The driver simplifies development for both the human and the machine.

pip install tcannex

Alignment Check

Utilizing the API driver, we can then begin to automate calls to the API for specific NIST documents. The positive assertion here is to find specific NIST document element_identifiers in a given document (policy, procedure, process, plan, etc.). We provide the script a candidate document and then a NIST document_identifier to check for alignment. From there we can take the element_identifiers from the elements list and do a lookup of those identifiers within the document. This check is pretty simple:

• Fetch known good element_identifiers from API

• Parse target document into a list of text strings

• Compare each line of text strings against each element_identifiers

• Write a report

Hallucination Check

Again, utilizing the API driver, we can automate the process of checking for hallucinations. While the alignment check is a positive assertion (what did we find that is a known good), the hallucination check is a negative assertion (what did we find that is NOT a known good but that looks like a known good). This check is slightly more complicated. A brief process is below:

• Fetch known good element_identifiers from API

• Determine what those element_identifiers look like (regex)

• Parse target document into a list of text strings

• Find candidate matches from the text strings based on the regex

• Determine which of the candidate matches are real against the known good identifiers

• Determine which of the candidate matches are NOT real based on the known good identifiers

• Write a report

Data

To test these scripts, TCAnnex used an AI-generated “quick start guide” for the NIST CSF 2.0. A snippet is below.

Step 3: Protect Data, Models, and Access (PR)
Objective: Build protection into AI pipelines, training, and inference.
•	Enforce RBAC and least privilege for model access.
•	Protect training data with encryption, DLP, and input validation.
•	Harden ML environments with container isolation and secret management.
•	Secure retraining and versioning with MLOps controls.
•	Train teams on adversarial ML, responsible AI, and ethical design.
✅ Use: PR.AC, PR.DS, PR.PT, PR.MA, PR.TR

Results

TCAnnex used these checks against the “quick start guide” and used the NIST CSF 2.0 as the alignment document. We reviewed the reports with a human eye. The alignment check correctly identified all relevant CSF 2.0 identifiers. The hallucination check was able to find the content that looked like CSF 2.0 identifiers but was hallucinated (PR.TR).

Astute readers will notice that the last line of the data snippet contains CSF Category identifiers. These identifiers fall into a few buckets:

• CSF 2.0 identifiers

• CSF 1.1 identifiers

• Hallucinated identifiers

Given that the check was done against CSF 2.0 identifiers, the script correctly identified the CSF 1.1 identifiers as hallucinations. Many human readers, familiar with the CSF (author included), would glance over this document snippet and say “looks good, I’ve seen PR.AC a million times”. However, CSF 2.0 uses “PR.AA” for that Category. This demonstrates a key point in any review cycle: we all have blind spots.

Conclusion

TCAnnex is committed to providing high-integrity data and information regarding cybersecurity risk management. As AI becomes more prevalent in our work and personal spaces, it is critical to verify what is in our information systems. By providing the TCAnnex API and API driver, we are giving the community a way to ground their systems in truth and fact. This article demonstrated a way to use those tools in an AI context for ensuring the veracity of generated content.

The writing of this post did not use AI labor. The creation of the scripts and test data used AI labor - Claude Opus 4.6. All code and content was reviewed by human eyes.