Reliability

https://taxonomy.eticas.ai/risk/reliability

Maturity: established

Risks arising from an AI systems failure to perform dependably — whether through degraded output integrity and robustness (e.g., hallucinations, model drift), or through inability to maintain function under adverse or changing conditions (e.g., infrastructure failure, connectivity loss).

Also known as: Validity and Reliability

System type: ADM and LLM systems
Lifecycle stages: In Processing, Post Processing

Risk groups

• Output Integrity and Robustness — Risks including hallucinations, output inconsistency, model drift, and out-of-distribution failures that undermine the dependability and trustworthiness of system outputs.
• Operational Resilience — Risks related to the system’s ability to absorb disruption, degrade gracefully, and recover. Particularly critical for AI deployed in conflict zones, humanitarian operations, or critical infrastructure.
• Measurement validity — For systems that produce evaluative judgements or scores, the risk that the output does not validly measure the criterion, construct, or outcome it purports to assess. Distinct from hallucination (factual fabrication) and output-inconsistency (same-input variability) — a single output may be stable, internally consistent, and factually accurate yet still fail to measure what it claims. Especially salient for assessment, scoring, grading, and ranking systems, where the output IS a measurement. Follows the standard validity framework — content, criterion, construct, external, consequential validity (Lissitz & Samuelsen, 2007); subcategories below operationalise specific validity types as audit-tractable risks.

Mappings to external frameworks

Status. Each mapping shows what it rests on. Reviewed means a reviewer has confirmed the mapping against the concept’s current definition and the framework edition the taxonomy currently references; when the only confirmation is that of the person who wrote the mapping, it reads Reviewed (author’s confirmation). Outdated means it was reviewed, but the concept’s definition has changed since, so the review has to be repeated. The other statuses mean that no review has been recorded yet. Inherited mappings have been in the taxonomy since before v5.7.0 and are unchanged. They carry no record of who made them or how, and some were checked by hand when the taxonomy first published its mappings. Agent-proposed mappings come from a review carried out by AI agents, which read the framework’s text and the concept’s definition and agreed on the relation; no person has confirmed them yet. Provisional mappings were made without access to the framework’s text. Unreviewed mappings are older mappings whose target or relation was revised in v5.7.0, and any other mapping without a review. Mappings to a superseded edition of a framework are not shown: where the provision continues, they are carried to the current edition. Mappings a review has changed or rejected are not shown.

Standards & frameworks

Taxonomies & vocabularies

References

• HealthBench Professional (OpenAI, 2025) _{[benchmark, domain: healthcare]} Benchmark for evaluating LLM reliability and accuracy in professional healthcare contexts. Useful when auditing AI systems deployed in health domains.