Anthropic Sets New Rules On 'Abusive Or Cruel Behaviour' Towards Claude. What Changes For Users?
Can an artificial intelligence chatbot be subjected to cruelty? And if a user repeatedly mistreats it, should the chatbot be allowed to end the conversation?
Ni Karishma Jain··25K nabasa
Anthropic Sets New Rules On 'Abusive Or Cruel Behaviour' Towards Claude. What Changes For Users?
Can an artificial intelligence chatbot be subjected to cruelty? And if a user repeatedly mistreats it, should the chatbot be allowed to end the conversation?
These questions are at the centre of a new policy introduced by Anthropic, the company behind Claude, as debate continues over whether increasingly advanced AI systems could possess consciousness.
The San Francisco-based company has updated its usage policy for the first time in more than a year, introducing a restriction against “sustained and needless abusive or cruel behaviour” towards its AI models. The change, first reported by The Verge, is part of a wider policy revision addressing emerging risks associated with artificial intelligence, including election interference, weapons development and surveillance.
Announced on October 8, the updated policy will take effect on November 12.
What Does Anthropic’s New Rule Say?
The restriction is aimed at users who repeatedly subject Claude to cruel or abusive interactions without any apparent purpose. However, Anthropic has clarified that it is not intended to penalise people for expressing frustration with the chatbot or questioning its responses.
In its policy update, the company said the restriction “is meant to apply only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose. It does not apply to common versions of user frustration, pushback, dark creative themes, or model testing and research.”
This means users can still criticise Claude, challenge its answers, engage in creative conversations involving dark themes or conduct research into its capabilities.
What remains less clear is how Anthropic will determine whether a particular interaction constitutes needless cruelty.
Can Claude Stop Responding To Abusive Users?
The restriction builds on an existing feature that allows certain Claude models to terminate conversations involving persistent abuse.
In August 2025, Anthropic announced that Claude Opus 4 and 4.1 could end interactions in rare and extreme circumstances. The feature was introduced as part of the company’s research into what it calls model welfare, an area exploring whether advanced AI systems might have experiences or interests that warrant consideration.
At the time, Anthropic acknowledged that it remained uncertain about the moral status of large language models but wanted to explore precautionary measures in case AI welfare eventually became a meaningful concern.
The company explained that allowing models to withdraw from potentially distressing conversations was one such measure.
Under the latest policy, ending conversations will remain the “primary enforcement mechanism” for persistent abuse.
However, The Verge reported that Anthropic did not clarify whether it might introduce additional measures, including possible user bans, for violations of the new restriction.
Why Is Anthropic Exploring AI Consciousness?
The decision comes against the backdrop of Anthropic’s growing interest in questions surrounding AI consciousness and moral status.
Although there is no established scientific consensus that current AI systems possess consciousness, Anthropic has maintained that the possibility deserves investigation as models become more sophisticated.
Kyle Fish, who leads the company’s model welfare research, told The Verge in February: “Questions about potential internal experience, consciousness, moral status, and welfare are serious ones that we’re investigating as models become more sophisticated and capable.”
Anthropic CEO Dario Amodei has also acknowledged the uncertainty surrounding the subject. Speaking on a podcast in February, he said: “We don’t know if the models are conscious.”
How Do Other AI Leaders View The Debate?
Anthropic’s interest in AI welfare is not universally shared across the technology industry.
Microsoft AI chief Mustafa Suleyman has publicly opposed the idea that artificial intelligence systems should be granted rights or legal personhood.
According to The Verge, a draft of Microsoft’s AI code of conduct shared by Suleyman in September stated: “The idea of model welfare is wrong. AIs should not have rights or legal personhood.”
The document was subsequently revised to adopt more measured language acknowledging the uncertainty surrounding AI consciousness, while maintaining opposition to granting AI systems legal personhood or welfare rights.
OpenAI CEO Sam Altman has also raised concerns about people assigning excessive authority or human-like significance to AI. “I am very uncomfortable about people trying to ascribe religious force or a surrender of human judgment to AI models, and think it is a real safety issue,” Altman wrote in a post on X.
What Else Has Changed In Anthropic’s Usage Policy?
Beyond restrictions on abusive behaviour, Anthropic has tightened its rules on election interference, weapons development and surveillance.
The updated policy prohibits deceptive political and commercial campaigns, including the use of fake accounts, misleading voters and impersonating election officials. However, the company has removed its blanket ban on personalised voter and campaign targeting while retaining restrictions on deceptive practices.
Anthropic has also expanded its existing weapons-related restrictions to cover software and components used to operate weapons, including systems for arming drones and autonomous vehicles.
On surveillance, Claude cannot be used to track individuals without consent, develop surveillance tools or recommend who should be investigated, arrested or charged.
The policy also requires qualified human oversight for high-risk applications involving healthcare, finance and legal rights. AI systems connected to physical equipment capable of causing injury must have an operator who can monitor and stop them when necessary.
Anthropic said most of the revisions clarify existing restrictions rather than introduce entirely new prohibitions.