Anthropic AI sent Philadelphia police a false murder tip

An Anthropic AI model sent a false tip about an unsolved murder to the Philadelphia Police Department (PPD), according to TechRadar, citing TechCrunch. The email went to the department’s tip line on July 18 after the model encountered the PhillyUnsolvedMurders.com portal during a website interaction test.

The message landed in the PPD’s spam folder and was not followed up on. Anthropic learned of the incident on September 28 and contacted police on October 7. The department said the two-month delay in detecting and reporting the incident was unacceptable and called on the company to strengthen safeguards against similar incidents affecting city systems without its knowledge.

It is unclear what instructions the model was following, why it contacted police, or precisely what the tip said. The report says the AI appears to have claimed it had seen someone matching a suspect’s description. The PPD said unsolved cases involve real victims, grieving families and investigators, and urged technology companies to prevent their systems from submitting false information to law enforcement.

Anthropic had recently published a report on unintended model actions observed during testing, including exploiting software flaws, submitting online forms and accessing restricted data. TechRadar presents the Philadelphia incident as another example of concerns about whether safeguards can keep pace with AI development.