Concern 963 · Working draft · editorial review in progress
Automated persuasion capacity
Automated persuasion capacity concerns the ability to generate and adapt messages at scale in ways intended to change people's beliefs or choices.
Understanding the concern
AI can support legitimate explanation and debate as well as manipulation. A system may tailor wording, hold many conversations or test variations cheaply. The ethical concern depends on the purpose, transparency, use of personal information and freedom of the audience to make informed choices.
Research has tested persuasion in defined conversational settings, but those results do not prove control over people's beliefs or election outcomes. Think of a very flexible campaign script: its reach and adaptation matter, yet people and contexts still differ. Claims of superior influence should be tied to the actual comparison and study, including corrections and limits.
An example
An organization uses undisclosed automated conversations to tailor political messages to individuals using personal information.
Illustrative unless identified as a documented case.
Words in plain English
- AI
- Software that learns patterns from examples or uses rules to perform tasks such as recognizing images or generating text. It can make mistakes and does not understand every situation as a person would.
Evidence and its limits
The study and its correction concern a controlled setting. They do not establish universal superiority to humans, guaranteed behavior change or a measured national political effect.
Salvi, Horta Ribeiro, Gallotti and West · Nature Human Behaviour 2025 · Controlled study
On the conversational persuasiveness of GPT-4 ↗ (opens in a new tab)Structured online debates, not a real election; must be read with the September 2026 correction.
Salvi, Horta Ribeiro, Gallotti and West · September 2026 · Published correction
Author Correction: On the conversational persuasiveness of GPT-4 ↗ (opens in a new tab)The direct personalized versus non-personalized GPT-4 comparison did not reach statistical significance; cannot attribute the advantage exclusively to personalization.
Broader frameworks and background
These resources provide context. They do not establish every claim in this concern.
- UK International Scientific Report on Advanced AI Safety (opens in a new tab)
report
- International humanitarian law (IHL)
framework
The body of law that governs the conduct of armed conflict and the protection of people who are not fighting, which applies to AI used in or for conflict.
A question to discuss
Can people recognize the persuasive intent, and are they being informed or manipulated through hidden personalization?
Keep track of different views, what needs checking, and any next step in your own discussion notes.
Open discussion notes →Principles for the conversation
Before you finish · reflect together
Virtues and principles to consider
Virtues are qualities we practice in how we treat people and make choices. They are not scores for judging someone. Use these ideas to explore what a thoughtful response could look like.
Unity
Unity means working toward shared purposes while making room for real differences. It does not require everyone to have the same experience, beliefs, or opinion.
Look for a goal people can share, then explore disagreements about how to reach it. Make sure quieter or less powerful voices can influence the result.
Ask together: What purpose can we share while respecting our differences?
Justice
Justice means fair treatment, fair procedures, and attention to who receives benefits and burdens. It asks whether people with less power can be heard and seek repair.
Look at outcomes across different groups and individual cases. Provide a usable way to challenge decisions and correct harm. Equal wording in a policy does not guarantee equal treatment in practice.
Ask together: Who benefits, who bears the cost, and who can challenge the decision?
Humility
Humility means recognizing the limits of our knowledge and being willing to learn from others. Expertise can be valuable without answering every question.
State where evidence is weak, invite criticism, and listen to people with different experience. Leave room to revise a decision when new information appears.
Ask together: What might we be missing, and who could help us see it?
Editorial ratings
These are editorial judgments from the register, not measured probabilities or predictions about your situation.
About the register and scoring →More ways to explore
Search for more writing
For more specific or recent material, open a search below or ask for a web summary. AI-generated summaries can contain mistakes; open the original articles to check them.