Why we make four AI models argue before we ship

We stopped letting one model decide anything expensive.

The tool that makes that practical is Claude Octopus, an open source plugin for Claude Code. It sits on top of whatever AI command line tools you already have and treats them as a panel rather than a shelf of alternatives you pick from. On this machine that means four readers: Claude hosting, OpenAI’s Codex and Google’s Gemini behind it, and Ollama running a local model for the cheap high-volume passes. Ten provider integrations ship with it. You need one to start.

The mechanic worth understanding is the consensus gate. Ask Octopus to review a design and it does not put the question to Claude and stop there. The same brief goes out to every provider you have configured, in parallel for research and adversarially for review, and then agreement gets scored. Fall below seventy five percent and the disagreement surfaces to you rather than being averaged into one confident-sounding answer. That number is doing most of the work in the product.

One detail buried in the source tells you the authors have been burned. The research skill carries a hard execution gate and the wording is blunt: the host model has to shell out and genuinely call the orchestration script, and it is forbidden from answering off its own knowledge and presenting the result as multi-provider research.

Somebody had to write that rule down, which should tell you how tempting the failure mode is. A model asked to cross-check itself will produce something that reads exactly like a cross-check.

I have watched Claude do this in a session. There is no interesting dishonesty in it. Nothing in the context gives the model a strong signal that the second opinion never arrived, so the confident summary comes out either way, formatted just as neatly as the real thing would be.

The consensus gate is one mode among several, and most of the skill is picking the cheapest one that fits the decision. /octo:debate puts the providers head to head on a single question and scores where they land. /octo:council convenes three, five or seven personas drawn from a roster of 32, and you choose the temperament: balanced, adversarial, or red team. /octo:factory takes a written spec and runs discover through deliver unattended. Behind those sit 54 commands and 63 skills, including a security pass that runs an OWASP scan and drafts the remediation rather than just naming the finding.

We live on the bottom two rungs. Council comes out for the decisions we would otherwise argue about for a week, and nothing has gone to the factory yet, because I want to be able to point at the reasoning when something eventually goes wrong.

Why now?

The ground under you keeps shifting. Anthropic previewed Claude Mythos in April and shipped stable Mythos 5 and Fable 5 on the ninth of June, while Opus 5, Sonnet 5 and Haiku 4.5 landed across roughly the same stretch and Codex and Gemini pushed their own updates through it. Every release changes something about default behaviour. How long the model writes, whether it hedges before committing, how much structure it volunteers that nobody asked for. Build a process that leans on one model’s judgement and you inherit its current defaults, including whichever ones changed the morning the new version went live.

Mythos is worth sitting with for a second reason. Access to Mythos 5 is restricted, and Fable 5, the public sibling, ships with safeguards that route certain requests to a less capable Opus. The model answering you is not reliably the model you addressed. Build a process around a single endpoint and that routing decision happens somewhere inside your workflow, silently, on requests you never get told about.

Take Opus 5, which hosts most of our work and which I rate highly on reasoning. The default register runs long. It over-explains. Left alone it restates your question before answering it, then hands back four paragraphs where two would have done the job. All of that is steerable and it complies when you tell it to be terse, but defaults are what you get on the days you forget to steer, and across a few hundred sessions that shows up in the token bill and in how much of the output anyone actually reads. I would call it a tuning preference rather than a defect. I would still like it changed.

A panel takes the edge off. Codex and Gemini read the same brief carrying different defaults, and those tendencies cancel instead of compounding. When a verbose model and a terse one land on opposite sides of a security boundary, the disagreement itself is information you can act on. A single model at full confidence gives you its training, delivered smoothly.

Underneath sits a four-phase workflow, discover through define through develop to deliver, with a gate between each phase. That matters less to me than the panel does. Multi-provider dispatch is not rare. An opinionated argument about when to stop and check is rarer.

None of this comes free. Every extra provider costs a call, adds latency, and gives you one more thing to keep configured and authenticated. For a small change Claude on its own is the right answer, and Octopus says as much in its own documentation: use the native path first, escalate when the decision earns a second reader. We escalate for architecture calls, security review, and anything where being confidently wrong would cost a week of rework.

It is MIT licensed and sits at github.com/nyldn/claude-octopus.