An Update On Independent Evaluators
Anthropic is granting "employee-like access" to independent evaluators like METR to assess its AI models and safety practices, fulfilling a commitment to stronger third-party oversight.
Intelligence analysis by Gemini 2.5 Flash
Anthropic is enhancing its commitment to independent AI safety evaluations by providing deep access to third-party organizations such as METR, SecureBio, and Accenture. This initiative, following the White House Accord on Super Intelligence, aims to foster public accountability and close the information gap between AI developers and external auditors, ensuring rigorous risk assessment…
Imagine a toy company that makes super smart robots. To make sure these robots are safe and don't accidentally cause trouble, the company invites special independent inspectors to come inside their factory. These inspectors get to see everything, just like the company's own workers, and then they tell everyone what they found, good or bad, so everyone can trust the robots are being made carefully.
Analysis
Anthropic's recent announcement marks a significant step towards greater transparency and accountability in the development of advanced AI models. By granting independent evaluators, specifically a team from METR, "employee-like access" to its internal systems and information, the company is attempting to address the inherent challenges of assessing complex AI risks. This approach acknowledges that external parties need deep insight into training methodologies, safeguard functions, and even potential failure modes like "alignment faking" to provide meaningful evaluations. The company's commitment to allowing METR to publish its findings, with only narrow, pre-defined categories for redaction, further underscores a push for genuine external scrutiny, rather than mere rubber-stamping.
METR
METR, a nonprofit dedicated to evaluating frontier AI models, is at the forefront of Anthropic's new independent evaluation strategy. Six METR employees have been integrated into Anthropic's operations, receiving company hardware, office space, and access to internal tools and systems comparable to those used by Anthropic's own risk assessment teams. This level of access is crucial for evaluators to gain a comprehensive understanding of the models' capabilities and potential risks, moving beyond superficial reviews. The agreement stipulates that METR will publish its analysis of Anthropic's risk levels, incidents, or practices, with Anthropic retaining only limited rights to redact information in specific, pre-defined categories such as customer data or attorney-client privileged information. Importantly, METR is free to publicly characterize any redactions and state whether it believes they were justified, ensuring a high degree of transparency even when information is withheld.
White House Accord
This initiative by Anthropic is directly linked to broader industry commitments, including the White House Accord on Super Intelligence. Last month, Anthropic, along with other leading AI companies, pledged to partner with independent external auditors or evaluators as part of this accord. The company's current actions demonstrate a tangible effort to fulfill this pledge, moving beyond mere statements of intent to concrete implementation. The analogy drawn to bank supervision, where regulators assign resident examiners to work inside large banks, highlights the perceived systemic importance of frontier AI and the need for similar robust oversight mechanisms. This framework aims to proactively identify and mitigate risks that could potentially threaten the public, mirroring the financial sector's approach to safeguarding against systemic failures.
Three Tiers
Anthropic is developing a structured, multi-tiered approach to engaging with independent evaluators, reflecting a scalable and progressive model for oversight. The company envisions three distinct tiers of access and publication rights, with the goal of eventually having several evaluators operating at the highest tier. The first tier involves narrowly scoped risk investigations, where Anthropic decides the level of access and retains approval rights over publication. SecureBio is currently operating at this level, reviewing specific chapters of Anthropic's Risk Reports. The second tier grants evaluators more extensive information access and the freedom to publish without company approval, though redactions for specific categories of information are still possible, with evaluators able to comment on their justification. The third tier, exemplified by the METR engagement, represents the deepest level of integration, offering "employee-like access" and publication rights similar to the second tier. This tiered system allows Anthropic to build trust and ensure partners have the necessary resources and expertise before granting full, embedded access, while progressively increasing the scope and independence of external scrutiny.
Key points
- Anthropic is providing "employee-like access" to independent evaluators, starting with a team from METR, to assess its AI models and safety practices.
- This initiative fulfills a commitment made as part of the White House Accord on Super Intelligence, aiming for stronger third-party assessment.
- METR will publish its risk assessments, with Anthropic having limited, pre-defined rights to redact information, which METR can publicly characterize.
- Anthropic is developing a three-tiered system for evaluator engagement, with METR representing the deepest level of access and publication rights.
- Other evaluators like SecureBio and Accenture are also engaged, with plans to potentially increase their access and publication rights over time.
This move could significantly enhance public trust in AI development by demonstrating a genuine commitment to external scrutiny and accountability. Increased transparency and independent validation of safety practices may lead to more robust and responsible AI systems, fostering a safer technological future.
Despite the deep access, the potential for redactions and the inherent complexity of AI risks mean that evaluators might not catch all subtle or emerging dangers. There's also a risk that the scope of evaluation could remain too narrow, or that the industry as a whole might not adopt similar rigorous standards, leaving gaps in overall AI safety.
