TITLE: New Protocol Tests AI Models' Factual Reliability Under Stress

New Protocol Exposes Vulnerabilities in AI Factual Accuracy

A novel evaluation method called the Drill-Down and Fabricate Test (DDFT) has been developed to assess how large language models (LLMs) handle factual accuracy when subjected to degraded information and adversarial challenges. The protocol reveals that many advanced AI systems falter in maintaining reliable knowledge under realistic pressures, regardless of their size or design.

Standard benchmarks like MMLU and TruthfulQA test models in controlled settings with full context and straightforward queries. However, they overlook epistemic robustness—the ability to sustain factual integrity amid incomplete data or probing attacks. DDFT addresses this by simulating a dialogue where models must drill into specifics as context compresses, ending with fabricated information to test error detection.

Evaluations involved nine frontier models across eight knowledge domains at five compression levels, yielding 1,800 turn-level assessments. Results indicate that epistemic robustness does not correlate with parameter scale (correlation coefficient r=0.083, p=0.832) or architecture type (r=0.153, p=0.695). Bootstrap resampling with 10,000 iterations confirmed these null findings despite the small sample size of nine models.

The study proposes a two-system cognitive framework for LLMs: a Semantic System for generating fluent text and an Epistemic Verifier for fact-checking. Under DDFT stress, the Verifier often fails, leading to 'semantic-epistemic dissociation'—coherent but incorrect outputs. Error detection capability correlated strongly with robustness (Spearman's ρ=-0.817, p=0.007), pinpointing it as the primary bottleneck.

Flagship models such as GPT-5 and Claude-Haiku-4.5 displayed unexpected brittleness, while smaller variants like o4-mini showed greater resilience. This challenges assumptions that larger models inherently yield more reliable performance.

Theoretical Framework and Two-System Model

The research draws on human cognitive analogies, likening the Semantic System to fast, intuitive System 1 thinking and the Epistemic Verifier to deliberate System 2 processes. The Semantic System excels at pattern-based fluency from pre-training, but the Verifier, responsible for validation, proves fragile under load.

Predictions from this model include dissociation patterns where fluency persists but accuracy erodes. Empirical support came from observing inverted performance: models strong in semantics weakened epistemically as compression increased.

DDFT incorporates a 'three-judge jury system' for evaluations, using metrics like Hallucination Onset Compression (HOC), Comprehension Resilience Index (CRI), and modified False Acceptance Rate (FAR') and Semantic Accuracy Score (SAS'). These quantify degradation and resilience.

A new Comprehension Integrity (CI) Index aggregates these, classifying models into epistemic phenotypes based on verifier strength. CI provides a multi-dimensional view beyond binary pass-fail benchmarks.

Key Findings from Evaluations

In tests spanning domains like medicine, finance, and law, epistemic degradation accelerated with compression. Robustness proved orthogonal to conventional metrics; models excelling on static tasks like MMLU still hallucinated under DDFT.

Comparisons to existing benchmarks, including hallucination detection and adversarial robustness tests, showed DDFT's unique focus on verifier stress. Ablation studies validated the protocol's turns and granularity, confirming correlations with CI at the turn level.

Semantic-epistemic dissociation emerged as a core failure mode, with mechanistic interpretations suggesting training gaps in verification. Smaller models outperformed expectations, implying that robustness stems from specialized training rather than scale.

Implications for AI Deployment

The findings underscore risks in high-stakes applications where incomplete or misleading inputs are common. A model reciting facts flawlessly in isolation may fabricate details when pressed, amplifying dangers in critical sectors.

DDFT positions itself as a diagnostic tool, not a competitive benchmark, to guide improvements in verifier mechanisms. Future directions include scaling to more models, integrating uncertainty signals, and exploring training interventions.

Limitations noted include the protocol's focus on English-language domains and reliance on jury evaluations, though rubrics for FAR and SAS ensured consistency. Code, datasets, and prompts are available for replication.

This work highlights the need for epistemic safeguards in LLMs, urging developers to prioritize verifier robustness over raw generative power.