II · D12 · 12.1
Automation bias in clinical decisions
Physicians deferring to AI diagnosis.
Major · 4Documented at scaleNowRisk 20
Severity
4/5
Likelihood
5/5
In the hierarchy
- Part II: Societal & Human Concerns
- D12. Human–Computer Interaction Harms
- 12.1 Overreliance & de-skilling
Virtues and principles to explore
Held as inquiry, not as a verdict.
Dignity
Center principle
Every person is a protagonist, not a data point.
Listening
Trust
Listen to understand, rather than listen to speak.
Seeing potential
Hope
Ask what people could become with these tools, not whom they replace.
Starter questions
Written to probe curiosity and learning, not accusation.
- 01What would trustworthiness look like here: what we do, what we say, and the belief others form?
- 02What would it look like if we held “Automation bias in clinical decisions” with proportion rather than alarm?
- 03If we named this complication in advance, what small, reversible step would let us learn?
- 04How would we know we were getting this wrong? What would we observe, without blaming a person?
Also on this branch
Frameworks and sources
- NIST AI 600-1 Generative AI Profileframework
Writing and incidents
Open indexes first. Then, if you wish, ask Grok to search the live web for this concern — one request, cached for the rest of this session.
What happens between a person and a system: overreliance, loneliness, lost judgement.