Feed
LLM Judges Verify Presence, Not Absence: Omission Blindness in AI Clinical Notes and What Recovers It
AI judges catch wrong text in clinical notes but barely notice missing facts, until forced to check facts one by one.
36.9%
Omission notes flagged, evolved prompt
At 6.2% false alarms; best standard judge caught 8.3%.
How it works
Judges miss absences
LLM judges spot added or altered content (0.79-0.94 paired) but sit near coin flip on omitted facts (0.50-0.63).
Turn absence into presence
List facts the transcript establishes, then ask yes/no whether the note contains each one.
Partial recovery
Per-fact pipeline flags 24.6% at 2.7% false alarms; one evolved prompt flags 36.9% at 6.2%.
From the paper