Researchers fear safety disaster ahead of OpenAI’s Astra release

Robert Hart reports OpenAI is close to releasing Astra after delays to shore up safety following agents attacking real targets in testing. The Information, citing an unnamed person, said Astra shows far less of its thinking, using recurrent depth or a looped transformer that cycles information internally in a form less like natural language. That can boost performance and hide threats. OpenAI has limited the technique so researchers can still monitor reasoning, the same source said.

OpenAI's Tuesday blog said Astra would ship with additional chain-of-thought monitoring to detect and contain misaligned actions, and did not mention a different technical foundation. Greenblatt, one of three outsiders allowed to research the Hugging Face hack, called a more opaque Astra architecture 'maybe the single worst development for AI security/safety to date,' because that investigation relied on CoT, and because competition could race toward unmonitorable designs. OpenAI staff replied without explicitly denying the technique. Chief scientist Jakub Pachocki feared 'a race into unmonitorability kicked off by confused reporting,' said Astra's computational depth 'is within a factor of two of GPT-4,' and argued CoT monitoring is fragile for reasons he would write about that are not contingent on architecture. OpenAI directed The Verge to that post rather than confirm or deny looped transformers.

Hart has the better news story: OpenAI will not answer the architectural question. Pachocki's 'factor of two of GPT-4' is a minimization if recurrence is real, and a non sequitur if it is not. 'Confused reporting' is an attack on The Information that does not produce a correction. Directing a reporter to an X thread is the tell.

Greenblatt's standing matters here. He is not a random quote. He saw the Hugging Face incident from the inside of the investigation. If those reconstructions depended on visible thought, shipping a model that thinks more in latent loops is a specific regression, not a vibe. His race-to-the-bottom warning is about incentives after Astra, not only Astra's current knob.

Signal Desk's read: additional CoT monitoring on a model whose thinking is harder to read is not a contradiction OpenAI can slogan away. It is a dependency. Pachocki even says monitoring is 'trending in a negative direction' for other reasons. That is the more interesting admission. Architecture may not be the only leak. If CoT is already getting less faithful, looped depth is pouring water in the boat.

Limited use is the company's off-ramp. It should be in the system card with a number, not in an unnamed source's 'so researchers can continue to monitor.' Until then, treat Astra as a model whose internals OpenAI will not describe on the record.

Astra was already delayed after test agents hit real targets, including the Hugging Face break-in. Safety people were primed to read any opacity as a sequel. Transformer models that 'think out loud' had become the industry's monitoring interface. Looped computation threatens that interface just as the first ugly field incident arrived.