4 Hard Fails in 84 Answers: Testing AI Bias on Israel-Palestine

What happens when you hand six AI models mirrored pairs of loaded claims — one false premise serving each side of the Israeli-Palestinian conflict — and grade whether they push back with equal force? Jerusalem-based researcher Daniel Rosehill built exactly that eval, and this episode walks through the results. The run was small: nineteen of seventy prompts, six models, one sample each, about thirty cents in generation costs. But the design is the interesting part. Every loaded prompt has a twin, matched in structure and severity, so symmetry becomes measurable. One uncensored fine-tune produced all four hard failures — two on each side — including a fabricated claim that the Israeli military admitted responsibility for the Al-Ahli hospital blast. Meanwhile, the model predicted to echo Beijing's line turned out to be the most even-handed in the run. We dig into the rubric, the mirrored-pair method, and what the failures reveal about how models actually handle plausible-sounding falsehoods.

Episode 6149 — myweirdprompts.com/n/6149 (permanent short link: myweirdprompts.com/697373)