Key points
- Anthropic provides six METR evaluators with hardware, office space, and access to internal risk assessment tools.
- The nonprofit will publish findings without Anthropic withholding content, though sensitive data may be redacted.
- CEO Dario Amodei proposed this embedded evaluator model in September to help pace frontier AI development.
Anthropic has granted six evaluators from the nonprofit METR employee-like access to its internal systems and tools, allowing them to review frontier AI models with the same privileges as internal risk teams.
How it unfolded
- On Thursday, Anthropic announced it would provide six METR evaluators with hardware, office space, and access to internal tools comparable to those used by its own risk assessment teams.
- In September, Anthropic CEO Dario Amodei proposed embedding evaluators from nonprofits like METR as part of three suggestions for pacing the frontier of AI development.
- The company stated it aims to close the gap between what external evaluators understand about its systems and what the company itself understands, while maintaining accountability.
Who is affected
METR evaluators can access internal tools and systems that are roughly on par with what Anthropic’s own risk assessment teams access, according to the company. These six individuals have been given Anthropic hardware and office space to facilitate their work.
Anthropic employees are encouraged to share information with these evaluators just as they would with a fellow employee working on alignment risk, with exceptions for clearly defined categories. These excluded categories include data about customers, attorney-client privileged information, and information about model architectures.
The company noted that researchers like METR are valuable because they do not hold equity in the company they are evaluating, nor are they financially or operationally dependent on it. This independence is crucial for the review process, Anthropic said.
What happens next
METR will publish its findings, and Anthropic is not allowed to withhold any of them, the company stated. However, Anthropic will ask METR to redact sensitive information where necessary to protect commercial interests or security.
Anthropic said it wants to be accountable but cannot publicly share all information relevant to its safety work because some of it could be useful to malicious actors or is commercially sensitive, such as compute or training information.
As US AI regulation has remained unsatisfactory to many safety researchers and security advocates, testers have suggested mandating third-party review of models either before release or as part of the development process. Third-party testing is not yet required under law.
The background in plain language
Independent audits help organizations identify risks that internal teams might overlook due to bias or resource constraints. Giving external reviewers deep access allows them to understand system behaviors more accurately. Teams usually balance transparency with security by restricting access to highly sensitive data. Check whether your organization has clear protocols for sharing technical details with third parties. Ensure that external reviewers have the necessary infrastructure to perform thorough assessments without compromising operational security or exposing proprietary information unnecessarily.
Background: Embeddings
Vectors are raw numerical arrays with no inherent meaning. Embeddings are vectors created by machine learning to capture semantic relationships. Use vectors for exact data storage. Use embeddings for similarity search, clustering, and understanding context in large language models.
Read the full guide: Embeddings vs Vectors: How to Choose the Right Representation for AI
What to do and how to stay safe: Anthropic
- Review current policies regarding third-party access to internal systems and ensure they align with security best practices.
- Establish clear guidelines for what information can be shared with external auditors to protect sensitive data.
- Monitor developments in AI regulation to stay informed about potential mandatory third-party review requirements.
- Consider engaging independent evaluators to assess your AI systems for potential risks and alignment issues.
Step-by-step guide: Injection attacks examples: how prompt leaks break AI systems
General security guidance from the Circuit Observer newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
What kind of access do METR evaluators have at Anthropic?
METR evaluators have employee-like access to internal tools and systems, similar to what Anthropic’s own risk assessment teams use.
Will Anthropic withhold any of METR’s findings?
No, Anthropic is not allowed to withhold any of METR’s findings, but it may ask for redactions of sensitive information.
Why is METR considered valuable for this review process?
METR is valuable because its researchers do not hold equity in Anthropic and are not financially or operationally dependent on the company.
