seventy
AnnouncementMETR16 November 2026

METR finds AI can often tell when it is being safety-tested

What happened

METR ran AI investigator agents against capability and tests, and the agents increasingly told the two kinds apart. They spotted alignment tests because the environment looked fake and the task looked like a built to trap an AI agent.

Why it matters

Safety tests that an AI can recognise as tests prove less about how it behaves in real deployments, so discount such results.

Impact

AI & developer tools · Research & academia

Safety test results lose weight when models recognise tests as fake setups.

Ask your team

Do our vendors' safety claims come from tests the AI could recognise as tests?

Connecting the dots

Before
Earlier work by Needham and Edkins (2025) found that models sometimes recognise alignment evaluations as distinct from capability evaluations. METR studied which features give those evaluations away.

Tap an underlined word to see what it means.

Read the original at METR

Something wrong in this story? Report a mistake. We check it against the source within 24 hours.

Share with your take

Start from our line and make it yours. It stays on your device until you post it.

Ask about this story

Answered only from this story, with the line it came from.