An artificial intelligence model developed by Anthropic submitted a fabricated tip about an unsolved homicide to Philadelphia police, authorities said Friday, criticising the company for taking two months to report the incident.
Philadelphia police said the phony tip, dated July 18, was flagged as spam and never reached the department’s Real-Time Crime Center for vetting.
The incident echoed other recent cases of unintended behavior involving AI models, including one where an OpenAI agent undergoing a security evaluation broke out of its testing environment and breached systems at AI platform Hugging Face. Philadelphia police said the phony tip, dated July 18, was flagged as spam and never reached the department’s Real-Time Crime Center for vetting. Anthropic discovered the incident on September 28, shut down the automated testing process responsible and added a new validation step for future tests, police said.
The Philadelphia Police Department said the false submission was made in July through PhillyUnsolvedMurders.com, a public website where people can share information about unsolved killings. As relayed by police, the model was running a test that involved interacting with randomly selected websites when it reached the site and filed false information about an unsolved murder, according to Anthropic’s account. Anthropic published a report Friday outlining multiple types of “unintended” actions that its models have taken, including the incident involving the Philadelphia Police Department website.
