Anthropic brings Accenture inside as an embedded AI evaluator

Anthropic said staff from Accenture will begin working inside the AI company to scrutinize its models and staff. The arrangement is the first announced example of Anthropic’s plan to place third-party evaluators inside AI labs.

Faculty, the AI division Accenture acquired in January, will evaluate and red-team models, conduct alignment assessments, and test model safeguards, Anthropic said. The companies expect to invest at least $1 billion in the project over the next five years.

Accenture’s shares rose 8% after hours following the announcement. Anthropic said the consulting company’s experience deploying AI for large corporations and government agencies was a key advantage. Its status as a large public company, separate from Anthropic and the wider AI-lab ecosystem, was another factor.

Anthropic said more evaluators will be announced in the coming weeks and that it is discussing pilot projects with METR and other nonprofit organizations. The company also said no standards yet exist for evaluators’ access or communications, and that its approach will evolve.

Some critics argue that embedded evaluation could help the AI industry avoid accountability. Anthropic said the evaluators would not reduce its accountability, but make it more verifiable, and that the safety of its models remains its responsibility.