A Watchdog Gets a Desk Inside the Building
Anthropic has given external evaluators what it describes as “employee-like access” to its internal systems, hardware, and day-to-day operations – a structural arrangement that moves third-party AI oversight from policy document to physical presence. The nonprofit doing the watching, METR, now has a team sitting inside the Claude maker’s offices, not just reviewing outputs from the outside.
This is not a routine security audit. The access METR holds goes deep enough that the distinction between employee and contractor starts to blur – the kind of arrangement that would be unremarkable in, say, a regulated financial institution, but remains rare in the AI industry, where internal research processes have historically stayed behind closed doors.
What METR Actually Is – and Who It Works With
METR is a nonprofit organization that has worked across most of the major AI labs: OpenAI, Google, Meta, and Anthropic. Its focus is evaluating the risk profiles of frontier AI models – the high-end systems that labs like Anthropic are racing to develop. That breadth of access across competitors is itself notable, suggesting the organization occupies an unusual position of institutional trust in an industry not known for sharing.
Anthropic’s decision to grant METR embedded access is a direct follow-through on earlier public commitments the company made around AI safeguards. Those commitments were partly a response to growing concern – from researchers, governments, and the labs themselves – that increasingly capable AI systems could cause serious harm to economic stability, cybersecurity infrastructure, or society at a broader scale. Whether those fears are calibrated correctly is contested. What is not contested is that Anthropic took a specific, structural step in response to them.
METR’s team now has access to Anthropic’s hardware and internal systems – not read-only documentation or sanitized test environments, but the actual infrastructure underneath the Claude models. That level of access makes it possible to evaluate risk in ways that external testing simply cannot replicate, because the variables that matter – training configurations, system prompts, model behaviors under edge conditions – are only visible from the inside.
Why Hardware Access Changes the Equation
Software audits conducted from outside a company’s walls can only measure what the company decides to surface. Hardware and system-level access changes that calculus.
When evaluators can see the underlying infrastructure – the physical compute, the architecture, the live systems – they are working with the same information the engineers have. That does not guarantee they will catch everything, but it removes one of the main structural excuses for why audits fail: that the auditor was only shown a partial picture. Anthropic, by granting this access, is at minimum reducing its ability to claim ignorance if METR flags something significant.
Third-Party Evaluation as the Industry’s Proposed Answer
Across the AI industry, increased third-party evaluation has become one of the primary responses labs have floated when pressed on how they plan to manage risks from more capable systems. It appears regularly in voluntary commitments made to governments and in internal safety frameworks. Anthropic’s arrangement with METR is one of the more concrete implementations of that idea – moving it from stated intention to a team with badges and desks.
The model has precedent in other industries where the consequences of failure are severe and asymmetric. Nuclear facilities, pharmaceutical trials, and aviation safety programs all operate with embedded external oversight that carries genuine access and genuine authority. Whether AI labs will reach that level of regulatory formalization remains an open question, but Anthropic’s arrangement gestures toward what that might look like before it becomes mandatory.
What remains to be seen is whether other labs follow with comparable arrangements, or whether this stays as an Anthropic-specific differentiator. OpenAI, Google, and Meta have all worked with METR in some capacity, but the nature of that access – and whether any of it approaches the embedded, employee-level model Anthropic has now established – has not been disclosed in equivalent terms. The gap between “worked with” and “given access to our hardware” is significant.
METR’s position is also worth examining carefully. A nonprofit that holds deep access to multiple competing AI labs simultaneously sits at an unusual intersection of institutional power. Its findings, its methodology, and its relationship with each lab are not fully public – and the legitimacy of the oversight model depends heavily on what METR actually does with what it sees inside Anthropic’s offices. Anthropic has structured the access. The value of that access is a separate question entirely, and one that METR’s internal processes will determine more than Anthropic’s press releases will.
