Concern 1000 · Working draft · editorial review in progress
Uncertainty and disagreement about probability
Disagreement about catastrophic AI risk reflects uncertainty about future capabilities, human choices and the ways events could unfold.
Understanding the concern
People may agree that a harm would be severe while disagreeing sharply about its likelihood. Their estimates can depend on different time frames, definitions and assumptions. Surveys of AI researchers describe those beliefs; they do not turn an opinion into a measured probability or reveal a single scientific consensus.
A forecast of flooding is hard to compare if one person means next week and another means the next century. AI risk conversations need the same clarity. Rather than collecting alarming or reassuring numbers, ask what event is being estimated, what evidence supports the estimate and what would change it. Some practical precautions may be worthwhile across several possible futures.
An example
Two speakers give very different risk estimates, then discover that they were imagining different time horizons and different kinds of harm.
Illustrative unless identified as a documented case.
Words in plain English
- AI
- Software that learns patterns from examples or uses rules to perform tasks such as recognizing images or generating text. It can make mistakes and does not understand every situation as a person would.
Evidence and its limits
The survey reports respondents' beliefs at a particular time and depends on sampling and question wording. It is not a measured probability of catastrophe or a substitute for examining specific risk pathways.
Grace and colleagues · 2024 · Researcher survey
Thousands of AI Authors on the Future of AI ↗ (opens in a new tab)Reports respondents' beliefs about future AI, not measured probabilities or a consensus prediction.
Broader frameworks and background
These resources provide context. They do not establish every claim in this concern.
A question to discuss
Which assumptions explain our disagreement, and which actions make sense even while the uncertainty remains?
Keep track of different views, what needs checking, and any next step in your own discussion notes.
Open discussion notes →Principles for the conversation
Before you finish · reflect together
Virtues and principles to consider
Virtues are qualities we practice in how we treat people and make choices. They are not scores for judging someone. Use these ideas to explore what a thoughtful response could look like.
Hope
Hope means seeing worthwhile possibilities and taking steps toward them, even when the outcome is uncertain. It does not require ignoring harm or promising that everything will work out.
Name the improvement you want in people's lives, then identify one realistic action and the conditions it needs. Keep room to revise the approach when evidence changes.
Ask together: What better outcome is worth working toward, and what can we do next?
Humility
Humility means recognizing the limits of our knowledge and being willing to learn from others. Expertise can be valuable without answering every question.
State where evidence is weak, invite criticism, and listen to people with different experience. Leave room to revise a decision when new information appears.
Ask together: What might we be missing, and who could help us see it?
Courage
Courage means acting on a well-grounded responsibility even when it is uncomfortable. It can mean trying something useful, admitting a mistake, or challenging pressure to move too fast.
Make it possible for people to raise concerns without punishment. Support those who point out harm, and be willing to pause a popular project when evidence gives a good reason.
Ask together: What important action are we avoiding because it is difficult?
Editorial ratings
These are editorial judgments from the register, not measured probabilities or predictions about your situation.
About the register and scoring →More ways to explore
Search for more writing
For more specific or recent material, open a search below or ask for a web summary. AI-generated summaries can contain mistakes; open the original articles to check them.