Function
Evaluate system behavior
Assess behavior against stated criteria and document the evidence.
Studio / Area
Governance, Evaluation & Assurance
Set constraints, examine behavior, and make accountability visible.
How is the system constrained, observed, evaluated, and held accountable?
In scope
- Evaluation, risk, privacy, security, oversight, documentation, incident handling, and assurance
Boundary
- Treating a framework as software or as proof of compliance
Start with the outcome
Functions
Function
AI risk management
Use a structured framework to identify, measure, and manage AI risks.
Function
Telemetry instrumentation
Emit traces, metrics, and logs from a running system for later analysis.
Function
OpenTelemetry
Toolkit for instrumenting systems and exporting traces, metrics, and logs.
Function
NIST AI RMF 1.0
A voluntary framework for managing risks in AI systems.
Function
Observe system execution
Collect signals that help explain what the running system did and where it failed.
Function
Assess AI-system risk
Identify and manage risks in the context of an AI system’s intended use.
Function
Detect sensitive data
Identify spans or fields that may contain personal or other sensitive information.
Function
Sensitive-entity detection
Use recognizers to find possible sensitive entities before review or masking.
Function
Presidio
Open-source components for sensitive-data detection and anonymization.
Function
Test-case evaluation
Run representative inputs and score application outputs against defined criteria.
Function
Promptfoo
Open-source CLI and library for LLM application evaluation and red teaming.
Function
LLM application tracing
Record request-level model, retrieval, and tool steps as linked observations.
Function
Langfuse
Application tracing platform for LLM calls, retrieval, and tool steps.
Function
OpenTelemetry GenAI Semantic Conventions
Developing conventions for recording generative-AI spans, metrics, and events.
Function
NIST AI RMF Generative AI Profile (AI 600-1)
A cross-sectoral profile for applying the AI Risk Management Framework to generative AI risks.
Function
Decide whether an action is allowed
Evaluate request context against explicit policy to return a permission decision.
Function
Policy-as-code decision
Evaluate structured requests against declarative policy outside application business logic.
Function
Open Policy Agent (self-hosted)
Open-source engine that evaluates structured input against policy and returns decisions.
Representative paths
Methods and options
approach
AI risk management
Use a structured framework to identify, measure, and manage AI risks.
framework
NIST AI RMF 1.0
A voluntary framework for managing risks in AI systems.
approach
Telemetry instrumentation
Emit traces, metrics, and logs from a running system for later analysis.
implementation
OpenTelemetry
Toolkit for instrumenting systems and exporting traces, metrics, and logs.
approach
Sensitive-entity detection
Use recognizers to find possible sensitive entities before review or masking.
implementation
Presidio
Open-source components for sensitive-data detection and anonymization.
approach
Test-case evaluation
Run representative inputs and score application outputs against defined criteria.
implementation
Promptfoo
Open-source CLI and library for LLM application evaluation and red teaming.
approach
LLM application tracing
Record request-level model, retrieval, and tool steps as linked observations.
implementation
Langfuse
Application tracing platform for LLM calls, retrieval, and tool steps.
standard
OpenTelemetry GenAI Semantic Conventions
Developing conventions for recording generative-AI spans, metrics, and events.
framework
NIST AI RMF Generative AI Profile (AI 600-1)
A cross-sectoral profile for applying the AI Risk Management Framework to generative AI risks.
approach
Policy-as-code decision
Evaluate structured requests against declarative policy outside application business logic.
implementation
Open Policy Agent (self-hosted)
Open-source engine that evaluates structured input against policy and returns decisions.
Across the Studio
Governance connection
Governance applies across every area, rather than as a final stage.
Related records
- In this areaEvaluate system behavior
- In this areaAI risk management
- In this areaTelemetry instrumentation
- In this areaOpenTelemetry
- In this areaNIST AI RMF 1.0
- In this areaObserve system execution
- In this areaAssess AI-system risk
- In this areaDetect sensitive data
- In this areaSensitive-entity detection
- In this areaPresidio
- In this areaTest-case evaluation
- In this areaPromptfoo
- In this areaLLM application tracing
- In this areaLangfuse
- In this areaOpenTelemetry GenAI Semantic Conventions
- In this areaNIST AI RMF Generative AI Profile (AI 600-1)
- In this areaDecide whether an action is allowed
- In this areaPolicy-as-code decision
- In this areaOpen Policy Agent (self-hosted)
Sources
Last reviewed 2026-09-29