Can we trust the tool?
75 · Any group. Youth can take this as 'does it actually work?'
Walk Part I: data, performance, security, explanation. A group that skips this will moralize a failure that is sometimes just a bad system.
Opening question
What would we need to know about a tool before we let it rank a person, a student, or a patient?
To hold together
- If we cannot see how a score was made, should we use it?
- What does 'it leaked' mean in a household, not only in a company?
- When is a confident answer more dangerous than a humble one?
A practice
Open Part I on the map. Choose one family. Read its highest-scored concern. Ask who, in our setting, would notice that failure first.
Concerns to name in the room
Principles to sit beside the facts
Trustworthiness
Words and actions align. Say only what you will do, and do it.
Prudence
Small, reversible steps before irreversible ones.
Suspending judgement
Neither trust nor scorn the tool. Test it.
Read or watch beforehand
One piece is enough. Save the rest for later.
Cathy O'Neil
Mathematician and writer
Weapons of Math Destruction is still the plainest account of scoring systems that punish the people least able to contest them.
Bruce Schneier
Security technologist · Harvard Kennedy School
Writes about security as a social problem. A reliable first read when a group is mixing fear of hackers with fear of the model itself.
Inioluwa Deborah Raji
Researcher · UC Berkeley
Wrote the case for closing the AI accountability gap: documentation, audit, and the right to know what a system was tested on.