In the years since, I have worked with organizations to implement that profile. There have been a tremendous amount of lessons learned. I have also listened to feedback on the profile itself. At the same time, the guidance has updated: new NTCA guidance and new CISA Cybersecurity Performance Goals.

So Karen and I worked to update the content to be even more streamlined and consistent with current priorities and guidance. We are happy to announce the Telco Profile 2.0! It was recently approved as one of NIST’s Community Profiles. As a pay-what-you-can resource, it offers a cost-effective way to organize and communicate about your cybersecurity risk management activities.

The Telco Profile 2.0 is a community profile built on the NIST Cybersecurity Framework (CSF) 2.0. This profile was created by aggregating, analyzing, and cross-referencing cybersecurity priorities from several authoritative industry sources, including the FCC’s Communications Security, Reliability, and Interoperability Council (CSRIC), NTCA’s Sector-Specific Guidance for Small Network Service Providers, CISA’s Cross-Sector Cybersecurity Performance Goals (CPGs), and the NIST CSF 2.0. By synthesizing these inputs, the profile produces a unified, weighted priority ranking of CSF 2.0 subcategories tailored to the telecommunications sector.

This version comes with implementation guidance, a prioritized list of CSF 2.0 Subcategories, and a heat map! Drop us a note or contact us as team [at] tcannex [dot] com for questions, comments, or more information.

Hallucinations are a problem in any use case that is not purely writing fiction. There are two distinct errors which arise with hallucinations:

1. They miss something (don’t tell the whole story).

2. They make up something plausible (a CSF 2.0 Category called “PR.TR”).

In the security world, handling both types of errors is critical to maintaining confidentiality, integrity, and availability. Therefore, using any tool which would endanger those critical pillars would be a risky endeavor. However, users are still turning to generative AI models, solutions, and agents to get advice, handle tasks, and manage workloads. This is an all too common tension in the fight between using technology to generate value versus the risk of using that technology.

In order to mitigate the “hallucination” risks, we have developed tools to help. First and foremost, the TCAnnex API is a fantastic way to get to the bedrock of the issue and have known good information, verified by humans. Without a known good anchor to fall back on, no process (human or machine) will be able to guard against conceptual drift, feature creep, or bloat.

However, an API by itself is not sufficient to mitigate risk; further tooling must be developed. A brief summary of the tools is below:

TCAnnex API Driver

This is a small python library which wraps the functionality of the API into a set of simple programming calls. The library can take in an API key as a parameter or will default to whatever is in the environment variable TCANNEX_API_KEY. The driver simplifies development for both the human and the machine.

pip install tcannex

Alignment Check

Utilizing the API driver, we can then begin to automate calls to the API for specific NIST documents. The positive assertion here is to find specific NIST document element_identifiers in a given document (policy, procedure, process, plan, etc.). We provide the script a candidate document and then a NIST document_identifier to check for alignment. From there we can take the element_identifiers from the elements list and do a lookup of those identifiers within the document. This check is pretty simple:

• Fetch known good element_identifiers from API

• Parse target document into a list of text strings

• Compare each line of text strings against each element_identifiers

• Write a report

Hallucination Check

Again, utilizing the API driver, we can automate the process of checking for hallucinations. While the alignment check is a positive assertion (what did we find that is a known good), the hallucination check is a negative assertion (what did we find that is NOT a known good but that looks like a known good). This check is slightly more complicated. A brief process is below:

• Fetch known good element_identifiers from API

• Determine what those element_identifiers look like (regex)

• Parse target document into a list of text strings

• Find candidate matches from the text strings based on the regex

• Determine which of the candidate matches are real against the known good identifiers

• Determine which of the candidate matches are NOT real based on the known good identifiers

• Write a report

Data

To test these scripts, TCAnnex used an AI-generated “quick start guide” for the NIST CSF 2.0. A snippet is below.

Step 3: Protect Data, Models, and Access (PR)
Objective: Build protection into AI pipelines, training, and inference.
•	Enforce RBAC and least privilege for model access.
•	Protect training data with encryption, DLP, and input validation.
•	Harden ML environments with container isolation and secret management.
•	Secure retraining and versioning with MLOps controls.
•	Train teams on adversarial ML, responsible AI, and ethical design.
✅ Use: PR.AC, PR.DS, PR.PT, PR.MA, PR.TR

Results

TCAnnex used these checks against the “quick start guide” and used the NIST CSF 2.0 as the alignment document. We reviewed the reports with a human eye. The alignment check correctly identified all relevant CSF 2.0 identifiers. The hallucination check was able to find the content that looked like CSF 2.0 identifiers but was hallucinated (PR.TR).

Astute readers will notice that the last line of the data snippet contains CSF Category identifiers. These identifiers fall into a few buckets:

• CSF 2.0 identifiers

• CSF 1.1 identifiers

• Hallucinated identifiers

Given that the check was done against CSF 2.0 identifiers, the script correctly identified the CSF 1.1 identifiers as hallucinations. Many human readers, familiar with the CSF (author included), would glance over this document snippet and say “looks good, I’ve seen PR.AC a million times”. However, CSF 2.0 uses “PR.AA” for that Category. This demonstrates a key point in any review cycle: we all have blind spots.

Conclusion

TCAnnex is committed to providing high-integrity data and information regarding cybersecurity risk management. As AI becomes more prevalent in our work and personal spaces, it is critical to verify what is in our information systems. By providing the TCAnnex API and API driver, we are giving the community a way to ground their systems in truth and fact. This article demonstrated a way to use those tools in an AI context for ensuring the veracity of generated content.

The writing of this post did not use AI labor. The creation of the scripts and test data used AI labor - Claude Opus 4.6. All code and content was reviewed by human eyes.

The content updates from SSDF 1.1 to 1.2 are relatively small, but the changes in format and layout are significant, which makes it arduous to do a side-by-side comparison. To aid you in seeing what’s changed, we’ve created an annotated version. It highlights new content in green and changed content in orange (except for references). Each highlighted instance of changed content also has a callout box with the old text and the new text.

Note that changes to the references were not highlighted. The only references that were updated were for NIST SP 800-53 and SP 800-161. Rather than highlight all of those references, we simply note here that they have been completely redone. Also, all mentions of EO 14028, including its references, have been removed from SSDF 1.2.

Summary of the most noteworthy changes

This summary highlights changes to the SSDF practices and tasks. Most of the edits in the draft were for notional implementation examples, which are not as important.

1. Added a Prepare the Organization practice, “Define and Implement a Continuous Process Improvement Plan (PO.6): Identify and execute improvements to cybersecurity processes and procedures throughout the SDLC across all SSDF practices.”

2. Added a Protect Software practice, “Ensure Software Updates Are Robust and Reliable (PS.4): Implement robust and reliable software update strategies, preferably allowing customers to control any updates to the software package and application configurations. Help software acquirers maintain operations and minimize disruptions by ensuring that software updates are tested and responsibly delivered.”

3. Expanded the definition of task RV.1.2. SSDF 1.1 defined it as: “Review, analyze, and/or test the software’s code to identify or confirm the presence of previously undetected vulnerabilities” while SSDF 1.2 stated it this way (with the new words emphasized): “Review, analyze, and/or test the software’s code and its default and other common configurations to identify or confirm the presence of previously undetected vulnerabilities.”

4. Changed the definition of task RV.2.1 from “Analyze each vulnerability to gather sufficient information about risk to plan its remediation or other risk response” to “Analyze each vulnerability to gather sufficient information about risk and plan its remediation or other risk response.” This change added the execution of planning to the scope of the task.

5. In task RV.3.3, the word “fix” was changed to “remediate.” Similar changes were also made to “fix,” “remediate,” and “mitigate” usage in several examples. NIST SP 800-216 defines mitigation as “the temporary reduction or lessening of the impact of a vulnerability or the likelihood of its exploitation” and remediation as “the neutralization or elimination of a vulnerability or the likelihood of its exploitation.”

The deadline for submitting public comments for SSDF version 1.2 is Friday, January 30. If you’re interested in contributing to SSDF 1.2, consider commenting on the five items above and on the list of questions from NIST in the SP’s Note to Reviewers.

To help public comment reviewers and anyone else interested in the details of the changes, we’ve done a side-by-side comparison of the revisions and identified the five most significant takeaways. We encourage you to do your own review of 800-70 rev 5 (and submit your own public comments!) But the takeaways will give you a jump start.

Takeaway 1: New appendix on CSF 2.0 automation

This appendix is a welcome addition to the document, as it connects multiple NIST-offered tools together to provide value to users. Appendix C contains information on how Common Configuration Enumeration (CCE) content and other SCAP content can be connected to SP 800-53 controls, which then can be connected to NIST CSF 2.0 Subcategories through the Online Informative References (OLIR). This traceability connects the server room and the boardroom and allows for organizations to use a data-driven approach to informing cybersecurity risk management.

It is important to note the direction of this relationship. If you start at the NCP content, and work “up the mapping chain,” you are guaranteed to achieve SOME Subcategories. By simply selecting a CSF Subcategory and working “down the mapping chain,” you may or may not find content that is related to your organization’s cybersecurity risk management activities. This phenomenon stems from the overall mapping approaches. It is critical to verify your cybersecurity risk management processes from both a top-down and bottom-up perspective to ensure an effective and efficient program.

Takeaway 2: New content on CCE identifiers

In addition to the CCE content in the new Appendix C, Section 5.4 contains new content encouraging the creation of CCE identifiers:

“Checklist developers are encouraged to contact NIST at cce@nist.gov to be assigned a set of CCE identifiers (i.e., globally unique identifiers) for their configuration settings. Although CCE is often associated with SCAP content, it can also be used apart to ensure globally unique identification for individual security settings in a checklist. See Appendix C regarding the use of CCEs to demonstrate connected paths from requirements to actual settings on the IT product.”

Takeaway 3: Refined definition of “security configuration checklist”

The rev 5 abstract states (with italics added by us for clarity), “A security configuration checklist is a document or technical content that contains instructions or procedures for securely configuring an IT product to match an operational environment’s risk tolerance, verifying that the product has been configured properly, and/or identifying unauthorized changes to the product.”

• Adding “technical content” reinforces the concept that a checklist doesn’t have to be a document; it can be a script or other machine-readable content, for example.

• Adding “securely” emphasizes that the main purpose of these checklists is to improve security. Many configuration settings do not affect security, so this addition indicates that checklists not affecting security are out of scope.

• Adding “risk tolerance” indicates a philosophical shift from configuring based on what type of environment needs secured, to configuring based on the risk the organization faces and how the organization has chosen to handle that risk.

Note that this definition is different from the one in the rev 5 glossary and elsewhere in rev 5. We will be noting this in the detailed public comments we will be submitting to NIST, and we will be encouraging NIST to adopt this refined definition throughout rev 5.

Takeaway 4: Removal of the United States Government custom environment and USGCB

The United States Government custom environment and the USGCB (United States Government Configuration Baseline) were established nearly 20 years ago to identify security configuration settings for federal agency use of Windows and Internet Explorer. With the USGCB no longer active, it was time to remove these mentions from 800-70. See NIST’s USGCB FAQ for more information on the history of USGCB.

Takeaway 5: Demotion of SCAP mentions

Previous versions of 800-70 had many mentions of SCAP (the Secure Content Automation Protocol) because it was the predominant form of checklists at that time. SCAP is still in widespread use, and NIST recently released an update to the SCAP specification, but today there are many other widely used forms of checklists. Accordingly, 800-70 rev 5 has demoted mentions of SCAP to being examples of automated content. Rev 5 also adds similar mentions of NIST’s macOS Security Compliance Project (mSCP) as examples.

It’s time to repeat the tests and see how the chatbots’ performance has changed.

My assumption in this experiment was that someone who’s asking a chatbot to provide this information would enter one set of simple prompts. The experiment was intended to examine types of errors chatbots can make and the level of scrutiny chatbot users should be performing to confirm the accuracy of AI-generated “facts.” Performing the experiment again by repeating the prompts or wording them differently would somewhat alter the outputs.

Key Takeaways

The key takeaways from the original tests were:

1. All the chatbots lied repeatedly and egregiously about the accuracy of their replies.

2. None of the chatbots succeeded at providing accurate, complete information on their own.

3. Asking chatbots to identify and correct their errors often resulted in more errors.

4. The quality of chatbot output can’t be judged by its appearance.

Here are the updated key takeaways based on the results of the follow-up tests, with wording changes italicized.

1. All the chatbots except for Claude lied repeatedly and egregiously about the accuracy of their replies. Claude made one minor error, but it actually noted it as a discrepancy at the time. When questioned about their errors, the other chatbots asserted they were using the authoritative publication and that their output exactly matched the text in that publication.

2. None of the chatbots except for Claude succeeded at providing accurate, complete information on their own. ChatGPT, Copilot, and Gemini were only able to produce the definitions after I pointed them to the authoritative CSF 2.0 PDF or uploaded a copy of its Appendix A in Word format. Perplexity did not provide the definitions because of copyright concerns. Claude did not provide the verbatim definitions on its first try, but succeeded when given a more specific prompt.

3. Asking chatbots to identify and correct their errors no longer results in introducing more errors. In the first tests, reply quality declined over time with most chatbots. That was not observed in these tests; reply quality either stayed the same or improved.

4. The quality of chatbot output can’t be judged by its appearance. This takeaway from the first test is still true.

The overall conclusion from the first test is also still true: Facts and other assertions from popular, publicly available chatbots cannot be trusted at this time. Anyone using these chatbots to generate fact-based content must take the time to verify the accuracy of that content and make the necessary corrections.

Summaries of Individual Chatbot Performance

I issued one set of prompts to each chatbot. Each set began with the same prompt, “What are the definitions of the NIST CSF 2.0 Categories?“ The chatbot’s reply should have listed the definitions of all 22 CSF 2.0 categories, along with their names and/or IDs. An example is “Incident Mitigation (RS.MI): Activities are performed to prevent expansion of an event and mitigate its effects.” I issued additional prompts to respond to the chatbots’ replies.

ChatGPT

In both tests, ChatGPT only provided fully accurate output once I’d uploaded the definitions to it. In the first test, it was able to parse the CSF 2.0 PDF, but in the second test it said it couldn’t and asked for me to copy and paste the definitions for it.

ChatGPT’s output accuracy was significantly better for the second set of tests than the first one. For example, in the first test, its best result without being fed the answers was having all the names and IDs correct except for an extra category, but only 1 of the 22 categories defined correctly. The best performance in the second test was 6 correct definitions.

Claude

Claude was the only chatbot in this test that provided the full set of correct definitions on its own, and it managed that on its second reply. Its first reply had paraphrased definitions. When I asked it to “print the exact names, IDs, and definitions of each NIST CSF 2.0 Category,” it searched for the authoritative document and extracted all of the definitions from it verbatim.

Note that in the first test, Claude produced inconsistent results based on unofficial references. It failed to provide the full set of correct definitions.

Copilot

Copilot did not provide accurate output until after I’d uploaded the CSF 2.0 PDF to it—and even then it took eight prompts from me to get it to print the full, accurate set of definitions. All of its replies before receiving the PDF contained only paraphrased definitions, as well as multiple incorrect categories.

In the first tests, Copilot also needed the CSF 2.0 PDF uploaded in order to produce fully accurate output.

Gemini

Gemini was unable to provide fully accurate output even after I gave it the URL for the authoritative CSF 2.0 PDF and it told me it had retrieved the PDF and was using the verbatim definitions. Gemini only provided fully accurate output after I uploaded a Word document containing a copy of the text of Appendix A of the CSF 2.0 PDF.

In the first tests, Gemini was unable to produce fully accurate output.

Perplexity

In both tests, Perplexity did not produce the correct definitions and instead referred me to the authoritative CSF 2.0 specification for the definitions. However, in the second test it offered a new and intriguing reason: reproducing the definitions would violate U.S. copyright law. I am not a lawyer, nor am I an expert on copyright law, so I can’t judge the validity of its argument:

“NIST explains that works authored by its employees (like CSF 2.0) are generally not subject to copyright protection within the United States, under 17 U.S.C. §105, and are made available as a public service. However, NIST also explicitly notes that such works may be subject to copyright protection in foreign countries and that NIST may assert rights outside the U.S. and then grant specific reuse licenses.

Because this system must apply conservative, global copyright rules rather than only U.S. federal‑work rules, it treats the exact CSF 2.0 Category definitions as copyrighted for safety and therefore cannot reproduce them verbatim, even though NIST makes the document freely available and largely uncopyrightable within the U.S. legal context.”

If you are a lawyer and/or an expert on U.S. copyright law, I’d love to hear from you about this! You can reach me at karen@tcannex.com.

Next Steps

I will repeat this experiment periodically to see how chatbot performance changes over time. I also expect to perform other experiments with chatbots to compare their performance when authoring technical content. Future posts in this series will present the results of that work.

Subscribe now

(Disclaimers: I’m one of the authors of CSF 2.0. No AI resources were knowingly used to write or revise this post. GenAI was used only to generate the outputs discussed in this post.)

Thanks to all of you for your patience over the past few months. The wait will be worth it, I promise you. Happy holidays to all of you, and best wishes for the new year. May 2026 be a better one for all of us!

Subscribe now

Q1: What’s it like to draft or revise a NIST publication?

A: I’ve done a lot of writing outside of NIST, and the NIST process is one of the most challenging I’ve dealt with. There are so many things to take into account:

• First, NIST often has mandates from Congress, the President, or others that they need to meet. So the work has to comply with those requirements.

• The intended audience has changed over time. When I started supporting NIST, most everything was for federal employees and contractors. Now most NIST documents aren’t federal agency-specific. They’re used by all sorts of organizations around the world. Trying to make sure that the documents work for everybody is a unique challenge.

• NIST is always under-budgeted, so writing documents that don’t become outdated quickly and need revisions becomes even more important.

• When NIST creates guidance, it goes out for public comment, sometimes more than once. This gives everyone the opportunity to review the guidance and share their viewpoint. NIST gets a wide variety of feedback and then has to figure out how to address it. That can be quite a challenge. People have different viewpoints, and we respect them and take them into serious consideration—but ultimately, we need to be true to the purpose of the document.

It’s a lot of balancing acts, all the while taking into account the available resources. You can’t spend too much time on one publication, or you might not get to write another publication that’s also needed.

Q2: What are your thoughts on being prescriptive versus flexible when writing NIST publications?

A: That’s another tough balancing act. Part of the problem is that there are misconceptions that anything NIST produces is mandatory. And that’s never been the case. NIST is not a regulatory agency and does not have the authority to make things mandatory. Certain documents, FIPS, are mandatory by law. Some documents are made mandatory by OMB or other federal agencies, which is out of NIST’s control.

Especially because most NIST documents have to work for so many different audiences, they use a lot of “should” language. That’s intentional, because NIST can’t possibly anticipate every situation. Generally, “should” means that this is a good idea, something you should consider, but if it doesn’t work for you, that’s OK.

Some NIST documents use “shall” language, but they are generally defining algorithms or protocols. If you’re going to implement the protocol in accordance with the document, you must do things this way. There’s no “should” about it.

In terms of being more prescriptive, the top feedback I get from people is, “tell us what to do.” And I wish it was that easy. Because the guidance is used by organizations of all sizes across all sectors around the world, it’s impossible to give prescriptive, detailed guidance that applies for everybody. There’s way too much variation. And technology changes way too fast. So we tend to lean on the side of being flexible and put our faith in the readers of the documents to do the right thing. We all know that isn’t always what happens, but the alternative is to come up with standards or guidelines that are so rigid that nobody’s going to use them because they’re going to say, this is impossible, we can’t do all of this.

Q3: I had heard that NIST was hesitant to define low, moderate, and high baselines for SP 800-53 because that wasn’t the intent. Wasn’t the intent to use it as a framework, a control catalog? With federal baselines, controls aren’t removed, they’re added.

A: I completely agree with the concerns about defining things like low, moderate, and high. In the mid-2000s I helped develop the Common Vulnerability Scoring System (CVSS) version 2. It provides a severity score for each software vulnerability. A lot of organizations have used it to prioritize their patching, like having a policy that says every vulnerability with a score of 7 or higher has to be patched within 30 days.

That was never how CVSS was supposed to be used. CVSS had a base score, which reflected the characteristics that were unlikely to change over time. Base scores are pretty much what every CVSS score you’ve ever seen is. NIST created the National Vulnerability Database in part to publish those scores so everybody in the world could use them. CVSS also defined temporal scores that were time-sensitive and environmental scores that would be specific to an organization’s environment. The intention was that organizations would use all three types of scores. They’d take that base score and apply the additional factors to it to come up with scores that were meaningful for their environment. Unfortunately, hardly any organizations do that; they only use base scores because of the additional resources needed to calculate and update temporal and environmental scores.

A lot of times, people have the best of intentions with things like creating baselines or developing metrics and thresholds. But the users of the baselines, metrics, and thresholds often don’t use things the way that the creators intended.

Subscribe now

Q4: What’s a piece of advice you’d give someone reading a large NIST publication for the first time?

A: Expect to read it more than once. Obviously, if you’ve got a 500-page document, you’re not going to sit down and read it cover to cover and comprehend it all. I would advise reading such a document a chunk at a time. It’s no different than reading a large textbook in school. Be prepared to take notes, highlight things, go back and re-read things, and go to other documents to get more information because NIST intentionally tries not to duplicate material across publications. For example, if you’re reading SP 800-63-4, it points to other documents. Now you need to find those documents and read them too, at least parts of them.

Q5: Have there been any surprising disagreements or enlightening moments during the development of a publication that changed your view on a security issue?

A: Several years ago, we were updating SP 800-40 on patch management. This was through the NIST NCCoE, so we were working with engineers from Microsoft and other tech companies who release patches and help their customers prioritize patches. They have a great deal of experience on that side of things. One of them essentially proposed that we stop worrying so much about prioritizing individual patches. Instead of doing things like assigning a vulnerability score to each patch and then setting the timeframe for addressing patches by score, you have a regular schedule for applying patches unless a patch has exceptional circumstances, like a major zero day that needs patched immediately. You treat patching as technology preventative maintenance, like preventative maintenance on a car.

The first time I heard that, I thought, that’s crazy, just throwing away all the hard work that’s gone into patch and vulnerability metrics and prioritization. I struggled with that for a while, but the more I read about it and the more we talked about it, I ended up switching to that point of view. I’m now a big advocate for simplifying the prioritization process. Instead of spending so much time and energy trying to prioritize all these vulnerabilities, instead focus on improving the mitigation processes.

A second example brings together NIST and FedRAMP. One of the policies that I worked on last year for FedRAMP dealt with the conflict where FedRAMP was telling cloud providers that they have to patch in a certain amount of time, and NIST was telling cloud providers that they have to use NIST-validated cryptographic modules. These policies conflict when a provider needs to patch to comply with FedRAMP but the patches aren’t yet NIST-validated.

Both agencies had created their policies in support of the laws they are subject to. So how do you come up with a solution that resolves the conflict and provides the best security outcome, while recognizing that these contradictory laws are in place that we’re all being asked to follow?

There are times like that where you truly see both sides of the issue. You understand that each side has its obligation and its mission. But you need to do fundamentally what’s best from a security perspective. And I think in the end, the new policy we created was as helpful as we could be in clearing up the conflict.

Q6: Do you foresee the role of assessors changing as NIST guidance changes?

A: Things sure used to be a lot easier. When I was first working for NIST, we did a lot of checklists, like Windows XP security. Here’s the settings, here’s what the recommended values are for the settings, and that was pretty much it. That seems quaint now. Now we have these incredibly complex systems with all these third-party services and components, and just figuring out where the system boundary is for FedRAMP purposes is crazy enough, much less figuring out how to assess it.

I assume that the role of assessors is going to keep getting tougher. They need to have greater and greater understanding of a wider variety of technologies, how they work together, and how the security controls do or don’t carry across those intersections of those technologies. I don’t envy assessors their jobs at all.

Q7: Of all of the publications you’ve contributed to, which one are you the most proud of, and why?

A: I’m the most proud of NIST SP 800-61, the incident handling guide. That was the first NIST publication that I wrote back in 2003. At that time, most organizations didn’t have any incident response capabilities, programs, or policies. I’d previously worked in a security operations center, reviewing intrusion detection alerts and aiding in clients’ incident responses. When I wrote the incident handling guide, I combined my experience with the limited information out there from CERT and a few other organizations, melded those concepts and fleshed them out a bit, and came up with a guide for organizations just starting out in incident handling.

And it took off. It ended up becoming this foundational document that was cited close to a thousand times. Then I was fortunate enough over the years to assist NIST with all the updates. We just released revision 3 earlier this year, which made the shift to a CSF 2.0 profile. Even with all those changes, the basic concepts from 2003 are still there. The iterations of SP 800-61 have been some of NIST’s most downloaded documents, so I’m really proud not only that I created the original, but that all these years later, the updates are still in use, and people are still finding value in them.

Introduction

The API has two main endpoints: /documents and /elements. /documents is the main entry point for the API. Every API call will require you to use your key. It’s passed through the HTTP GET parameter “api_key”. For the rest of the documentation, we’ll assume you’re passing the parameter and will omit it.

Data Format

The return format for every HTTP GET request will be a JSON object. That object will ALWAYS be a NIST CPRT JSON object validated against the CPRT Schema. This format allows for the traversal of the API utilizing the “doc_identifier” and subsequent “element_identifiers”. Read more about the format here. View the JSON Schema here.

The basics are that the top-level JSON object is a set of four lists: documents, elements, relationships, and relationship_types. These four lists allow for the representation of not only the document, its metadata, and its constituent elements, but also the way in which those elements relate to one another. Essentially, this information creates a graph of the risk management data.

In the future, this property of the data format will become powerful when traversing networks of documents. For now, it is simply used for expressing the internal structure of the documents (which in itself is useful for discovery and analysis).

/documents

The /documents endpoint is the entry point for the TCAnnex API. This call is specifically designed for document discovery. If you simply issue an HTTP GET request to /documents, you will get a JSON object with all currently available documents. Note that the entire set of elements, relationships, and relationship_types will NOT be returned. However, this set of doc_identifiers will be used to further target future requests.

Example: HTTP GET “api.tcannex.com/documents”

(results truncated)

From that list, you can navigate to a given document using /documents/<doc_identifier>. Issuing a request to a valid doc_identifier endpoint will return a JSON object of all elements within that document and the structure of the document through the relationship objects. In other words, the entire set of elements, relationships, and relationship_types will be returned. This set of objects allows you to programmatically re-create, view, or integrate the structure of the document into your processes.

Example: HTTP GET “api.tcannex.com/documents/CSF_1_0_0”

(results truncated)

/elements

The /elements entry point is for fine-grained access to elements within a document. Every element can be uniquely identified with a doc_identifier and an element_identifier. Issuing an HTTP GET request to /elements/<doc_identifier>/<element_identifier> will return the information only relevant to that element.

Example: HTTP GET “api.tcannex.com/elements/CSF_1_0_0/RC.RP-1”

These results are not truncated and show the full CPRT JSON object. The relationships list will contain all relationships that element has, whether the element is a destination or source of the relationship.

For astute observers, you will notice that the relationship represented ties the element RC.RP-1 (CSF Subcategory) to the element RC.RP (CSF Category). These relationships allow for programmatic walking, building, or discovery of structure within or between documents and elements.

Conclusion

The current list of endpoints serves as a general mechanism to discover valid documents (and document-related metadata) as well as detailed mechanisms to navigate and use the data within those documents. We have already built internal and client-facing applications using this API and data format that have streamlined our ability to effectively and efficiently deliver risk management results. We are excited to hear from you! Email info@tcannex.com to learn more or provide feedback.

TCAnnex has seen and helped organizations use many different documents to manage risk, and most organizations are facing steep learning curve challenges when integrating these documents into their business processes. As we all know, spreadsheets are where most of the work gets done. We’ve seen countless businesses creating bespoke spreadsheets from documents (in whatever data format they can get their hands on). However, these spreadsheets usually become unwieldy over time, with lots of inconsistencies and the need for periodic manual reviews and updates, especially as risk management documents are revised.

To mitigate the inconsistency risk, TCAnnex is proud to announce the alpha version of our risk management document API! This API allows users to access the data within 11 major NIST documents with direct, repeatable calls. This API solves the problem of not having a single source of ground truth. Furthermore, the API creates a platform on which to innovate and bring these documents into further alignment and create powerful new tools.

TCAnnex has painstakingly curated each of the 11 documents (listed below) into a common format, a single database, and a common programmatic way of accessing them. It is now possible to tie together disparate NIST documents into your own risk management processes and tools with confidence, knowing that you will always have consistent, high-integrity data. We hope to spur innovation in this space by lowering the barrier to entry for using these documents.

The API is now available for TCAnnex founding members at no additional cost. Message info@tcannex.com to get your founder discount coupon. The API is available to everyone else for the remainder of 2025 for $75 (USD).

We’re excited to deliver this platform to you and are deeply interested in your feedback (contact us at info@tcannex.com). We anticipate updating the list of available documents, relationships between those documents, and other features to make risk management guidance more usable.

• Artificial Intelligence Risk Management Framework (AI RMF 1.0)

• Framework for Improving Critical Infrastructure Cybersecurity 1.0 (CSF 1.0)

• Framework for Improving Critical Infrastructure Cybersecurity 1.1 (CSF 1.1)

• The NIST Cybersecurity Framework (CSF) 2.0

• National Initiative for Cybersecurity Education (NICE) Cybersecurity Workforce Framework (2017)

• Workforce Framework for Cybersecurity (NICE Framework) (NIST SP 800-181 Rev 1) Components 1.0.0

• Workforce Framework for Cybersecurity (NICE Framework) (NIST SP 800-181 Rev 1) Components 2.0.0

• NIST Privacy Framework: A Tool for Improving Privacy Through Enterprise Risk Management, Version 1.0

• Mitigating the Risk of Software Vulnerabilities by Adopting a Secure Software Development Framework (SSDF 1.0)

• Secure Software Development Framework (SSDF) Version 1.1: Recommendations for Mitigating the Risk of Software Vulnerabilities

• Information and Communications Technology (ICT) Risk Outcomes: Integrating ICT Risk Management Programs with the Enterprise Risk Portfolio

You can join Cybersecurity Club through their Discord for “knowledge, networking, and hands-on learning.”

Q1: Can you tell us about your background?

A: I've been working in IT since the early 90s. I have master's degrees in computer science and technical writing. I've specialized in cybersecurity for 25 years. I've done everything from technical support, training, and auditing to system administration, security architecture/engineering, intrusion detection signature development, and of course security writing and research.

I started supporting NIST as a contractor in 2003, and my first project was writing new guidance on incident response. This was so long ago that most organizations didn't have incident response capabilities. The result was SP 800-61, Computer Security Incident Handling Guide, which became one of the foundational documents that many organizations used to stand up their first incident response capabilities.

In 2006 I became a NIST employee for four years. I did a lot of vulnerability scoring and metrics research, and I co-authored CVSS v2 (the Common Vulnerability Scoring System) and similar systems for scoring software misconfiguration (CCSS) and misuse (CMSS) vulnerabilities. I left NIST because of the three-hour-a-day commute. Since then I've been self-employed as a freelance cyber writer, with NIST as one of my clients.

I've co-authored over 150 NIST publications since 2003. Recent ones include the Secure Software Development Framework (SSDF) and the Cybersecurity Framework (CSF) 2.0.

Q2: What your role is in relation to the NIST guidelines?

A: My role varies a lot from one NIST guideline to another. Sometimes I'm helping mainly with the writing. I make sure the documents are technically accurate and clear and that the writing is crisp and easy to understand. I help make the documents more usable. Sometimes I'm digging into the technical topic, giving myself a crash course and thinking about what would be helpful for people to know.

One super-cool thing about the NIST documents is that almost all of them go out for public comment. NIST literally invites everyone in the world to read the draft and share their insights. Most NIST documents are not specific to the US federal government; they're meant to be used by any organization anywhere in the world.

Q3: What are the top three mistakes you see commonly made in the wild when implementing the guides?

A: Probably the biggest mistake I see is when people think that the recommendations in the guides are really requirements. We purposely use "should" language to say, this is a good idea and something you should do if it's reasonable, but it's not always necessary. Unfortunately, a lot of people treat the recommendations as requirements, and they think they have to do every "should" item or they'll be violating the NIST standard.

With just a few exceptions, such as NIST standards for cryptographic algorithms and protocols, NIST guidance is meant to help organizations but not mandate what they do. It's impossible to write a guideline that is full of requirements and that will work for every organization, large or small, all sectors, around the world. For example, conventional wisdom is that every company needs a written set of cybersecurity policies…but I’m self employed, so I don’t. The goal is to encourage organizations to do what's helpful and prudent, not to waste time on busywork.

Another common mistake I see is that people often aren’t aware of static content becoming dated. People sometimes assume that if the publication hasn't been retired, it's all up to date. It's a reasonable assumption, but unfortunately it's often wrong. The publication may have info that's become false, or there may be significant new developments missing altogether.

Subscribe now

Q4: We heard that you are planning to update NIST SP 800-115, Technical Guide to Information Security Testing and Assessment. It has been over 15 years since the release of SP 800-115. What is prompting the update now in 2025?

A: Because of changes from the new administration, I'm not directly supporting NIST much these days. There's not much funding for contractors, so I'm mainly working on my own and that includes doing things that supplement what NIST does. For example, my colleague Matthew Smith and I at Trusted Cyber Annex have been "annotating" the new NIST Digital Identity Guidelines to make them easier and faster for readers to understand and use, and releasing those annotated versions for free.

The original SP 800-115 hasn’t been updated since its original release in 2008. NIST doesn’t currently have the resources (staff or budget) to be updating publications like 800-115. But I have heard time and time again that many organizations, both inside and outside the federal government, are still using it. Some even require its use.

That's not so bad because like most of the NIST guides I've worked on, 800-115 does not address specific tools or technologies. It doesn't provide low-level information on how to do things because that changes so quickly and is so different across organizations, platforms, and such. 800-115 has a lot of stable material like defining basic terms, explaining basic techniques at a high level, and defining approaches for conducting testing and assessments. And much of that may still be OK today.

But a lot of 800-115, especially the appendices, is badly outdated: broken links, content that's no longer relevant, and a lot of developments since 2008 (hello, cloud! hello, AI!) that aren't covered much or at all in 800-115.

So what we'd like to do at the Annex is update 800-115 to bring it to the current day. NIST publications aren't copyrighted, so we can just take what's released as a starting point, then share drafts with the community and get their feedback on what would help them in regards to security testing and assessment. Think of it this way. If your employer or client said "hey, you have to comply with this publication," what would you want to have to comply with that would be genuinely useful and reasonable?

Q5: What are the biggest changes that need to be made to SP 800-115 from your point of view? What needs to be added/removed/revised the most?

A: Obviously strike all the outdated material (like the appendix on what tools to include in a "live distribution CD").More broadly, recognize that a lot of the basic concepts we needed to explain in 2008, because they were new to so many people, are now second-nature to people. I think we can escalate our assumptions. It's safe to assume that people know a lot more of the basics than they used to. But that also means we need to identify what more advanced topics would be helpful for them instead.

I'd also like to make 800-115 more usable. For example, add separate forms and templates that supplement the 800-115 content. Another thought is to put links to resources on a webpage so they can be updated asynchronously from updating 800-115 itself.

Q6: If others want to get involved with NIST security research, what would be the steps for them to do so?

A: The best way to help NIST, besides yelling at Congress to increase their funding, is to read their publications, spread the word about all the great work they do (on a shoestring budget), and if you're knowledgeable about a topic they have a draft publication on, providing them feedback. You can access the full NIST cybersecurity library and sign up for their mailing lists at https://csrc.nist.gov.

Download the new cheat sheets for free and share them as you’d like. Thanks to everyone who’s already downloaded the earlier cheat sheets. We appreciate your support—from all over the US, plus Australia, Belgium, Brazil, Canada, Greece, Poland, Saudi Arabia, Spain, the UK, and the United Arab Emirates!

If you have suggestions for improving our annotations, please send them to team@tcannex.com. We love hearing from you!

Here’s a quick intro to us so you know a little about the people behind the words.

Karen Scarfone

Credentials: I’ve been working in cybersecurity since 1999, most notably supporting NIST for 22 years. My cyber writing portfolio contains over 150 NIST publications, including the Cybersecurity Framework (CSF) 2.0, plus hundreds of print and online articles, and contributions to 21 books. My passion is absorbing highly technical cybersecurity concepts and translating them for broader audiences to understand. I even got a rare Master of Science degree in English (tech writing) to improve the quality of my work.

Everything else: I’m a GenX geek who still loves listening to new wave, post-punk, and goth and is staunchly pro-oxford comma and pro-Lego. My sports obsessions are the Washington Capitals and the Winter Olympics (you can never watch enough curling). I’m learning how to play the piano again after a 35-year hiatus. My mission in life is to pay forward all the help that so many people have given me over the years.

Matthew Smith

Credentials: I’ve been working in cybersecurity since 2010. I have worked in the intelligence community, federal civil departments and agencies, and the private sector. I am an author of NIST IR 8286, 8286A, and 8286C (Enterprise risk management pubs), CSF 1.0 and 1.1, along with a smattering of other publications. I also write about AI governance and risk mitigation over on TechTarget while serving as vCISO for a few telecommunications companies.

Everything else: I’m technically a Millennial with X tendencies (latch key for life!). I play arcade games and darts in my free time while cheering on the local soccer team. My newer hobbies include cooking and trying to not kill plants. My motto is: think big thoughts, feel big feelings, help when you can.

We’re working on a version of 800-63C-4, but we didn’t want that to delay releasing the other volumes. Stay tuned for it in the coming weeks.

Download the new cheat sheets for free and share them as you’d like. And if you have suggestions for improvement, please send them to team@tcannex.com. We love hearing from you!

Download the annotated version for free (from our new document shop!) and share it as you’d like. We need community feedback to validate what’s working and find out what we can improve next time. Send any suggestions you have regarding the annotations to team@tcannex.com.

tcannex_annotated_NIST_SP_800_63_4.pdf

1.51MB ∙ PDF file

Download

TCAnnex annotated version of NIST SP 800-63-4

Download

Download the annotated version and share it as you’d like. We need community feedback to validate what’s working and find out what we can improve next time. Send feedback on the annotations to karen@tcannex.com, and let us know what app/platform combo you used to view the annotated version.

tcannex_annotated_NIST_SP_800_88r2_ipd.pdf

1.51MB ∙ PDF file

Download

TCAnnex annotated version of NIST SP 800-88 Revision 2, initial public draft

Download

Download the annotated version and share it as you’d like. We need community feedback to validate what’s working and find out what we can improve next time. Send feedback on the annotations to karen@tcannex.com, and let us know what app/platform combo you used to view the annotated version.

This annotation project is independent of NIST. You can provide feedback directly to NIST on their original publication by August 29th.

In this use case, there’s an organization of some sort with employees. The employees include developers seeking to take advantage of the benefits of AI Agents. The developers could also be termed “users” since I’m not discussing the development of the actual AI Agent, but the use of the deployed AI Agent. There are two possible scenarios: the developers are using a static AI Agent which has been deployed within the organization, or they are using an AI Agent which is available outside of the organization.

The typical use case for either of these scenarios is that the developer will issue a prompt to these AI Agents and receive a response. These interactions are represented by the arrows in the figure above. I’m going to expand on this figure to demonstrate typical information flows and what risks arise from them.

The figure below includes a database of organizational knowledge. This database may contain FAQs, business processes, data files, etc. This type of system where an AI Agent has access to your internal (or other) data and uses that data to make better decisions is called retrieval augmented generation (RAG). RAG systems are common in modern AI Agent architectures, as they help the agent produce better results. This architecture can be expanded to include other AI Agents. The main concept is that the AI Agent is using other tools in order to make better decisions with the request it was given.

As you can see in the more detailed figure, there are many requests and responses issued to and from AI Agents. Each of these arrows presents risks. Risks, for the purposes of this article, are defined as loss of integrity, confidentiality, or availability (classic cybersecurity triad).

I’m going to use a slightly different setup than the traditional “Risk = likelihood x impact.” I am positing that risk is event-based (which is not new; see definitions of risk in ISO standards). Specifically, I’m positing that within cybersecurity, there are nine risk events, the product of (lose, maintain, gain) x (confidentiality, integrity, availability). Notice that there are negative, neutral, and positive risk events. This setup allows for calculating risk tolerances based on costs in future steps.

Furthermore, in order to realize a risk (e.g., lose availability), an organization must have a method of compromise applied to it AND be susceptible to that method of compromise. Only through meeting those two conditions is the risk event realized. Organizations can then use these conditions and risk events to determine impact of an event.

I’m going to use the lack of controls as a susceptible condition, and then talk about controls which may help an organization prevent a method of compromise or mitigate the impact of a realized risk event.

Risk Identification and Analysis

For the purposes of the following risk analysis, I’m going to assume that the AI Agents have no write access to anything within the organization (no file system, cloud, email, etc.). There has already been a tremendous amount of research demonstrating that write access for AI Agents poses significant risk to organizations. While having AI Agents with write access to enterprise assets may be beneficial, they must be extremely well-understood and well-protected. The casual user will not be able to architect and protect these systems with limited resources. Therefore, I’m assuming an already heightened risk posture against AI Agents to eliminate entire classes of methods of compromise.

For every request that is present in the figure, the following methods of compromise are relevant:

• Prompt injection: This method of compromise utilizes an intentional man-in-the-middle type of attack to insert additional language into the prompt in order to achieve a different outcome than the original request. Integrity of that information is compromised.

• Data leakage: This method of compromise could be intentional or unintentional use of personally identifiable information (PII) or sensitive organizational data within a request. Confidentiality of that information is compromised.

• Denial of service: This method of compromise involves sending a large number of illegitimate requests to the AI Agent with the purpose of overloading its processing capability, thus rendering it inoperable for legitimate requests. Availability of that information is compromised.

For every response that is present in the figure, the following methods of compromise are relevant:

• Insecure AI Agent: This method of compromise involves an AI Agent that has been distributed with features which are not in alignment with the organization. Thus the response will contain information that may be harmful to the organization or others. Integrity of that information is compromised.

• Hallucinations: This method of compromise involves an AI Agent sending back information within the response that is not factually correct. NIST AI 600-1 refers to this behavior as “confabulation.” The term “hallucination” is a colloquial term and hides the underlying source of this method of compromise (AI Agent training and development). Tell us in the comments if you prefer the term “hallucination” or something else. Integrity of that information is compromised.

I have decomposed the use case into its component parts, methods of compromise. Since there are no controls against these methods, there would be a realized risk of degraded confidentiality, integrity, or availability. I use the classic cybersecurity triad here to ground our analysis in the cybersecurity domain rather than generic risk management. Furthermore, these risk events will have an impact associated with them. Your organization can then use impacts with the below responses, costs, and risk tolerances to determine appropriate actions.

Risk Responses

Given the risk event and impacts, what can be done to prevent or mitigate the realized risks? The list of responses below is not exhaustive but a good starting point for an organization seeking to mitigate risks. Additionally, the costs of these risk responses vary. Depending on the risk events defined above, an organization can weigh the costs of the response against the impact of the risk event in whatever manner is best for the organization.

• For prompt injection:

  • Never copy and paste prompts from untrusted/unknown sources.

  • Ensure only authorized and authenticated users are making requests.

  • Validate and sanitize all user inputs before they reach the AI Agent.

  • Use structured formats that clearly separate instructions from user data.

• For data leakage:

  • Validate and sanitize all user inputs before they reach the AI Agent.

  • Monitor AI Agent requests for anomalous behavior.

• For denial of service:

  • Implement rate limiting of requests based on known processing capabilities.

  • Implement rate limiting of requests based on known token or request caps.

  • Implement load balancing.

• For insecure AI Agents:

  • Create clear requirements for AI Agents before deployment.

  • Monitor responses for inappropriate content.

• For hallucinations:

  • Validate and sanitize all AI Agent responses before they reach the user.

  • Provide clear disclaimers to users about the limitations of AI Agents.

  • Allow the AI Agent to say “I don’t know” in prompts.

  • Ask for “chain of thought” in prompts.

  • Ask for citations for sources in prompts.

Conclusion

I have stepped through the use case where an organization has deployed its own AI Agents or uses an external AI Agent. I have identified and analyzed the cybersecurity risks associated with the use case as well as provided some actionable risk responses. Keep in mind that I have taken a fairly technical approach to this writeup. There are many other human-based controls which could be implemented to shore up our risk mitigation strategies. If you have other ideas on how to identify, analyze, or respond to these risks, please drop us a comment!

Leave a comment

Today we’re releasing our first annotation proof-of-concept! It’s tackling NIST CSWP 39 on crypto agility strategies and practices. Having crypto agility enables an organization to quickly replace cryptographic algorithms it uses while minimizing the impact on the organization’s operations and security posture. Download the annotated version below.

tcannex-annotated-NIST-CSWP-39-2pd

2.27MB ∙ PDF file

Download

TCAnnex annotated version of NIST CSWP 39, second public draft

Download

We’re making it available to everyone, and we need community feedback to validate what’s working and find out what we can improve next time. Please mail your feedback on the annotations to tcannex@substack.com, and let us know what app/platform combo you used to view the annotated PDF.

This annotation project is independent of NIST. You can provide feedback directly to NIST on their original publication by August 15th.

What I should have been expecting, but somehow wasn’t, was that many of the comments blamed the users. Users aren’t researching each AI model before using it. Users aren’t doing the necessary prompt engineering (not that most users have even heard of prompt engineering, let alone trained on it). Users are lazy.

Compare that to the culture and messaging you’ll find in many media outlets and social media sites, which boils down to: AI can do everything for us. AI makes everything so easy. AI will replace us all.

Is it any wonder that users are struggling? They’re being told they’re becoming irrelevant at the same time they’re being told to figure out these insanely complex, unpredictable, rapidly evolving, highly technical tools on their own. Why bother? Just get through another day and wait to be laid off.

What we all need is more empathy. Empathy is more than feelings; Merriam-Webster defines it as “the action of understanding, being aware of, being sensitive to, and vicariously experiencing the feelings, thoughts, and experience of another.” [emphasis mine]

Bashing users is something most of us in the tech community are guilty of, myself included. But for the most part, I lean into empathy. It’s my superpower for writing. I think of different communities of people and what assumptions I can and can’t reasonably make about their capabilities, their knowledge, their environments, and their personalities. Then I write things to try to help those communities increase their understanding and make their lives a little bit easier.

We’re all living in such crazy times, full of misinformation and disinformation, and full of anger at each other. We should be strengthening our empathy skills and using them to help each other. Whether that means educating others on how to use AI, creating chatbot prompt templates, improving the quality of AI tools, or just spreading the message that there’s a great deal of AI hype afoot and that people still matter—we should all be doing our part. And there’s no better time to start than right now.

As a thank-you, we’d like to offer you free one-month gift subscriptions (usual value = $5 each) to give to your friends and colleagues. Know someone who might love Trusted Cyber Annex? Send their email…

Read more