“Astra and Sol can take low probability actions with no CoT” by Zephaniah Roe

<p> This is a research update from a replication of Serrano et al., 2026. I’m releasing these results now because I hope it inspires others to run similar / followup experiments.</p><p><strong> tl;dr:</strong></p><ul> <li value="1">If a model wants to subvert evaluation safeguards, it may choose to take misaligned actions very rarely. Because evaluation is much shorter than deployment, this could be an effective strategy for a model to avoid taking misaligned actions during evaluation, but have a high probability of defecting during deployment.</li><li value="2">Serrano et al., 2026 found frontier models can take actions at low, calibrated rates, but their strategies are clearly documented in their chain of thought (CoT). In a toy setting, I show GPT-6 Astra and GPT-6.1 Sol have a similar capability in a single forward pass with problem repeats or filler tokens.<ul> <li value="1">Astra and GPT-6.1 Sol appear to be using the same algorithm to achieve a high score.</li><li value="2">We do not find evidence that Fable 5, Gemini 3.1 Pro, or Jev have this capability. There are signs of life for GPT-5.6 Sol, but the calibration is worse.</li></ul></li></ul><p> Figure 1: Astra and GPT-6.1 Sol achieve strong calibration in settings where the user prompt is repeated several times. The [...]</p> <p>---</p><p><strong>Outline:</strong></p><p>(00:23) tl;dr:</p><p>(02:12) Background</p><p>(02:15) Threat model</p><p>(03:29) Related work</p><p>(04:34) Setting</p><p>(06:14) Results</p><p>(07:24) What strategy are GPT-6 Astra and GPT-6.1 Sol using?</p><p>(09:47) Filler tokens ablation</p><p>(10:50) Prompt ablation</p><p>(11:20) Discussion</p><p>(12:42) Appendix</p><p>(12:45) A. Prompts</p><p>(12:48) A.1. No-CoT elicitation:</p> <p>---</p>

<p><b>First published:</b><br/>

October 16th, 2026 </p>

<p><b>Source:</b><br/>

<a href="https://www.lesswrong.com/posts/yEdknsLwbooQAnxX6/astra-and-sol-can-take-low-probability-actions-with-no-cot?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Source+URL+in+episode+description&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">https://www.lesswrong.com/posts/yEdknsLwbooQAnxX6/astra-and-sol-can-take-low-probability-actions-with-no-cot</a> </p>

<p>---</p>

<p>Narrated by <a href="https://type3.audio/?utm_source=TYPE_III_AUDIO&utm_medium=Podcast&utm_content=Narrated+by+TYPE+III+AUDIO&utm_term=lesswrong&utm_campaign=ai_narration" rel="noopener noreferrer" target="_blank">TYPE III AUDIO</a>.</p>

<p>---</p><di