The NIST AI Risk Management Framework, published in January 2023, provided a voluntary, risk-based approach for organizations to govern AI systems across their lifecycle. When it was published, generative AI was a nascent technology. By mid-2023, large language models and generative architectures had moved from research curiosities to enterprise deployments at scale. NIST recognized that the AI RMF 1.0 did not fully address the unique risks that generative AI introduces. NIST AI 600-1, published in July 2024, is the result: a companion profile tailored specifically to generative AI.
What Makes Generative AI Governance Different
Traditional AI systems, such as classification models or predictive analytics pipelines, generate outputs from a defined output space. A fraud detection model outputs a risk score. An image classifier outputs a category. Generative AI systems produce novel content, including text, code, images, audio, and video, from learned distributions over training data. This fundamentally different output modality introduces risk categories that do not exist in the same form for conventional AI systems.
The inability to fully enumerate possible outputs, the sensitivity of outputs to prompt construction, the reliance on training data whose provenance and biases may be opaque, and the potential for plausible-sounding but factually incorrect outputs all create governance challenges that require purpose-built frameworks. AI 600-1 fills that gap.
The 12 Generative AI Risk Categories
NIST AI 600-1 identifies 12 unique risk categories for generative AI systems that practitioners must assess and address as part of their governance program.
- Confabulation (Hallucination): The AI produces outputs that are plausible but factually incorrect or entirely fabricated. This is particularly dangerous in high-stakes domains such as healthcare, legal advice, and financial guidance.
- Data Privacy Violations: Training data may contain personally identifiable information that the model can reproduce, either directly or through inference attacks.
- Information Integrity Hazards: AI-generated content used in information environments may undermine the integrity of factual discourse, including disinformation at scale.
- Data Provenance: The sources and quality of training data are often opaque, making it difficult to assess whether a model's knowledge base is reliable and representative.
- Bias and Harmful Content: Generative models can reproduce and amplify biases present in training data, producing discriminatory or harmful outputs.
- Intellectual Property and Copyright: Model outputs may reproduce copyrighted training data, creating legal exposure for deploying organizations.
- Cybersecurity: Generative AI can be used to produce malicious code, phishing content, and social engineering attacks at scale.
- Prompt Injection: Malicious inputs embedded in prompts can manipulate model behavior in ways the deploying organization did not intend.
- Dual Use: The same capabilities that make generative AI useful for legitimate purposes can be adapted for harmful applications.
- Environmental Impacts: Training and operating large generative models has significant computational and energy cost.
- Human-AI Configuration: Users may not understand the boundaries of AI capability, leading to over-reliance or inappropriate trust.
- Obscured AI Identity: AI systems may interact with users in ways that obscure their non-human nature, raising transparency and consent concerns.
Mapping AI 600-1 to the AI RMF Functions
AI 600-1 is structured as a profile within the AI RMF, meaning it maps its risk guidance to the four core AI RMF functions: Govern, Map, Measure, and Manage. For each of the 12 risk categories, the profile provides suggested actions organized under these functions. This means organizations that have already begun AI RMF implementation can extend their existing program to cover generative AI without starting from scratch.
The Govern function actions for generative AI focus on establishing organizational policies around acceptable generative AI use, transparency requirements, and human oversight thresholds. The Map function actions focus on identifying where generative AI is deployed in organizational processes and characterizing the specific risks present in each deployment context. Measure actions focus on testing and evaluation approaches, including red-teaming and adversarial testing specific to language model vulnerabilities. Manage actions address incident response, monitoring, and model update protocols.
Practical Governance Steps for Organizations
- Inventory your generative AI deployments. Many organizations have more generative AI in use than their governance teams are aware of, particularly through third-party software that has incorporated LLM capabilities.
- Classify each deployment against the 12 risk categories. Not every risk category is relevant to every deployment, but the classification exercise surfaces which specific risks require mitigation investment.
- Establish a confabulation management policy. For any generative AI deployment that produces factual content, define how outputs will be verified before acting on them.
- Conduct prompt injection testing. This is often overlooked in enterprise deployments that use API-connected LLMs. Define your attack surface and test it.
- Require AI system cards or model cards from vendors. When procuring generative AI tools, require documentation that addresses the 12 AI 600-1 risk categories. This is now expected in federal procurement under M-25-22.
NIST AI 600-1 gives practitioners a structured, authoritative framework for tackling generative AI governance. Organizations that build their programs around it will be better prepared for both federal procurement requirements and emerging regulation domestically and internationally.