Anthropic bars needless cruelty toward Claude

Anthropic has added a rule to its Usage Policy prohibiting “sustained and needless abusive or cruel behavior” toward its Claude models. The clause appears in a section covering cruel, abusive or psychologically harmful conduct, alongside restrictions involving suicide and self-harm, bullying, violence, harmful deepfakes and deceptive content.

Anthropic said last August that Claude could end conversations that became abusive. The company recently described that as its “primary enforcement mechanism.” It is not known whether Anthropic will also ban users for repeatedly mistreating its models.

The company says the new rule is intended for extreme cases of repeated cruelty without a discernible purpose. It says the policy does not cover ordinary frustration, pushback, dark creative themes, or model testing and research.

The change has drawn debate over whether AI models can be harmed. Anthropic CEO Dario Amodei has said the company does not know whether models are conscious. Microsoft AI CEO Mustafa Suleyman has argued that AI should not have rights or legal personhood.

One theory circulating on Reddit is that polite training data is more useful for improving models, but the source reports no hard evidence for that explanation. Anthropic may instead be seeking to encourage civility, the article says.