Health, wellness & care
Questions about AI in health and care.
Explore concerns in the clinic, at home, and in the community. Each section offers questions, related concerns, and a discussion guide.
- Named in care
- 41
- Scored 20–25
- 15
- Clinical AI
- 24
- All concerns
- 1000
Composite score · Health, Safety & Care
Most of what we named is already serious. Two sit at the top of the scale.
- 25 · irreversible & common · 2
- 20 · 13
- 16 · 24
- 15 and under · 2
Three rooms, one ecosystem
The clinic
A wrong score can harm a patient.
Triage, imaging, notes, prescribing, a robot at the table. The problem is not that AI is here. It is that it can be here without being named, checked, or owned.
- Who was this tested on, and do they look like the people we serve?
- What must remain a human judgement even if the model is faster?
- If this is wrong, is it a delay or an irreversible harm — and who is named?
- 25
Triage algorithm bias
Triage algorithm bias occurs when an automated urgency score gives some groups lower priority than their medical needs justify.
- 20
Diagnostic errors
Diagnostic error happens when AI suggests the wrong explanation for a person's symptoms or fails to identify an illness that is present.
- 20
AI documentation "ambient scribes"
Ambient-scribe risk arises when a tool listening to a clinical conversation creates an inaccurate record or captures and uses speech in ways the patient did not understand.
- 20
Clinical-note summarization errors
Clinical-note summarization error happens when an AI-written summary leaves out, changes or invents information that matters to a patient's care.
- 20
AI-driven prescribing
AI-driven prescribing risk arises when a medication suggestion overlooks a patient's circumstances or contains an error in the drug, dose, timing or instructions.
- 20
AI in surgery (robotics)
AI-assisted surgery risk concerns mistakes in automated perception, guidance or movement that could affect a procedure inside a person's body.
The household
Companions, elders, children, a wellness app in the pocket.
Care does not only happen in a hospital. A chatbot at 2 a.m., a parent negotiating a school tool, an elder with a device that talks back: that is health too.
- When is this a tool, and when is it being asked to be a friend?
- Does the person understand what is listening, and can they refuse?
- What would I need to know before I let this sit with a child, or with someone I love who is fading?
- 20
AI companions and adolescent development
AI companions may affect adolescent development when they become a major setting for learning about trust, boundaries and relationships.
- 20
Chatbots producing self-harm content
Unsafe self-harm responses occur when a chatbot encourages, enables or fails to respond appropriately to a person expressing a wish to hurt themselves.
- 20
Chatbots on suicidality
Inconsistent crisis responses mean a chatbot's handling of suicidal distress changes unpredictably across wording, context or repeated messages.
- 20
Pediatric AI safety
Pediatric AI safety concerns arise when a health tool has not been shown to work appropriately for children at their particular stage of development.
- 20
Geriatric AI
Geriatric AI risk arises when tools overlook the varied needs of older people, including multiple health conditions, many medicines and different care goals.
- 16
Companion apps for the elderly
AI companionship for older adults raises a choice between adding support and replacing human contact or care that the person still needs.
The community
Public health, school screening, the air and water models.
A town does not buy most of the systems it lives under. It still has to ask who is scored, who is missed, and who may refuse.
- Who is flagged, who is missed, and who owns the record that follows?
- What would public accountability look like for a tool we did not purchase?
- How do we speak about benefit without hiding who is left out of the training set?
- 20
Suicide-prevention AI in schools
School suicide-prevention AI risk arises when monitoring software misreads a student's activity or triggers a response that fails to protect their wellbeing and privacy.
- 20
Environmental-health AI
Environmental-health AI risk occurs when unreliable measurements or incomplete coverage lead a model to misjudge pollution exposure and who needs protection.
- 25
Substance-use disorder screening bias
Substance-use risk-score bias occurs when automated estimates of misuse or overdose unfairly shape how people are assessed or given access to treatment.
- 20
Special-education AI
Special-education AI risk arises when a tool intended to support a learner with a disability does not fit their needs or is used to replace necessary individualized support.
Borrowed from MIT · Causal taxonomy
Three questions that turn a worry into a sentence.
MIT asks of every risk: who set it in motion, whether it was on purpose, and when it arose. Tap a column. The example is from care, home, or community — not from a lab.
A person · By accident · Once people were using it
In the clinic
A scribe no one has time to check
An ambient note-taker is switched on to save minutes. The clinician is still responsible for the record. In a full clinic, review thins out. A fabricated symptom enters a chart.
Three questions that turn a worry into a sentence: who set it going, whether it was on purpose, and when. Walked with cases from care. Use them in a huddle. The sources page holds the denser maps when you need them.