A year ago, I wrote a piece called AI Is Not Your Guru, about a filmmaker who put on a robe and a fake accent, invented a holy man named Sri Kumaré, and taught a devoted flock the profound wisdom of absolutely nothing.
My point then was that boardrooms were doing the same thing with AI — chanting acronyms like mantras, treating technology like theology, waiting for prophets when what they needed were practitioners.
And I thought I was done with that essay.
Then Dr. Ezekiel Emanuel showed up in JAMA arguing that autonomous AI will soon outperform your physician, and I realized the sermon has a second act.
You can see his perspective live on Morning joe here.
Because think about it: there are really only two people in your life you should never, ever outsource to the most probable answer. Your priest and your doctor. One handles your soul, the other your body, and both jobs share a defining feature — the moment that actually matters is the moment the textbook fails. Last year we covered why the model isn’t qualified for the pulpit. Today, let’s talk about the exam room.
And this time, I have scars. Literally.
The story begins two decades ago, when my blood pressure decided it wanted to be famous…
Not “cut back on the salt” famous. Famous like a case study. The numbers kept climbing, and every physician I saw reached for the same answer, because it was the right answer — statistically. Malignant hypertension. It’s what the textbook says. It’s what the last thousand patients who looked like me had. They titrated the metoprolol up. Then up again. The blood pressure ignored all of it with the serene indifference of a cat being given instructions.
Then came the TIA. A mini-stroke.
I remember trying to write and watching my own hand fail to do it — a small, private horror I don’t recommend.
And here’s where the story turns, because it turned on one person. A young hospitalist, Dr. Stonerock, who looked at a patient failing every standard therapy and refused to accept the consensus answer. She bucked the attending, bucked the workup protocol, bucked the entire gravitational pull of the institution, and went hunting. What she found was a cardiac pheochromocytoma — a catecholamine-secreting tumor wedged next to my heart. How rare is that? Pheochromocytomas occur in less than 1 percent of all individuals with high blood pressure, and of those, the cardiac variety is so uncommon that fewer than 50 cases had been reported in the medical literature.
A one-percent diagnosis inside a one-percent diagnosis. The 1% of 1%.
The average answer — the consensus answer, the most probable answer — was going to kill me. A human being who went off-distribution saved my life.
Keep that story in your pocket. We’re going to need it.
The Emanuel Provocation
Dr. Emanuel — oncologist, bioethicist, a man who has never met a comfortable assumption he didn’t want to set on fire — co-authored a JAMA paper asking whether autonomous AI will exceed AI-aided physicians. His answer, roughly: yes, and soon.
The authors argue that AI is likely to outperform not just human doctors but doctors aided by AI, and project that autonomous AI will be ready for real-world cognitive medical tasks in some, maybe many, workflows by 2030. The sharpest blade in the paper is this claim: when AI alone is better at a task than humans alone, human-AI hybrids actually degrade performance — so as AI improves, having humans in the loop will likely worsen patient care.
Let me say something that will surprise people who know how I feel about breathless AI claims: I commend him. Genuinely. His co-author calls the paper a wake-up call for health system leaders with their heads in the sand, and heaven knows healthcare leadership could use one.
The status quo is not some safe harbor we’d be reckless to leave.
An estimated 795,000 Americans die or are permanently disabled by diagnostic error every year, per Johns Hopkins researchers publishing in BMJ Quality & Safety. That’s the population of Seattle, harmed annually, by the system we already have. Anyone defending human-only medicine as the gold standard is defending a gold standard with a body count.
So Emanuel is asking the right question. He’s just arguing about the wrong noun.
His paper — and nearly every vertical benchmark now being waved around boardrooms as gospel — frames the contest as model versus human. The model is treated as a sealed oracle: an advanced Google you pour a case into and pour a diagnosis out of. It’s Sri Kumaré all over again — the robe, the beard, the aura of borrowed authority — except this time the flock is a hospital board. And that framing isn’t just incomplete. It’s the single most expensive category error a healthcare administrator can make in 2026.
Because nobody deploys a model.
You deploy a harness.
What’s a Harness, and Why Should a Hospital CEO Care?
A harness is the software wrapped around the model — the tools, the memory management, the orchestration logic, the supervisory agents, the rules of engagement that turn a raw model into a system that does work. And in the last month, the evidence that the harness is the dominant variable — not the model — went from “practitioner folk wisdom” to “measured, published, repeated.”
Exhibit A. Nvidia published research showing that with a custom harness — one tuned to handle memory well and equipped with a “supervisor” component — Claude Opus 5 scored 100% on ARC-AGI-3, the interactive reasoning benchmark of instruction-free 2D games.
Without the harness, the same model scored 30% — which was still the best result of any model tested. Read that again. Same model. Same weights. Same “brain.” The difference between 30 and 100 was software.
Nvidia’s VP of product put it plainly: the world interprets an agent almost as an API of the model, but an agent is the model plus the scaffolding around it — the harness, the tools, the runtime.
Exhibit B, and I admit I enjoy this one. OpenAI, flustered by its models’ sub-10% scores on the same benchmark, ran its own study and found that tweaking just two harness settings tripled its scores. Two settings. Tripled. If the model were the whole story, that would be impossible.
Now — the exhibit that should be stapled to the front of Emanuel’s paper.
The Medical Proof Already Exists
Microsoft built something called MAI-DxO — a diagnostic orchestrator. Not a model. A harness. It simulates a panel of physicians with distinct roles, proposes differential diagnoses, and strategically selects high-value tests. Then they benchmarked it on 304 of the New England Journal of Medicine’s nastiest clinicopathological cases — the diagnostic haunted house where careers go to be humbled.
The results, from the paper itself: the off-the-shelf o3 model, working alone, achieved 78.6% accuracy at a simulated cost of $7,850 per case. Wrapped in the MAI-DxO orchestrator, accuracy rose to 85.5% — while a differently-tuned configuration hit 79.9% at just $2,397. Same underlying model. The harness made it both more accurate and dramatically cheaper.
And critically, these performance gains generalized across models from the OpenAI, Gemini, Claude, Grok, DeepSeek and Llama families. The harness wasn’t a trick that worked on one model. It was the point.
(Yes, the physicians in that benchmark averaged around 20% — but before anyone drafts the press release, Microsoft’s own researchers flagged that the cases were complex teaching cases, with no healthy patients or mild conditions, and it’s unclear how the system performs on routine cases or how often it produces false positives. Eric Topol made the same point about Emanuel’s paper: the studies weren’t real-world medicine, so “AI beats doctors” remains unproven. Benchmarks are maps. Patients are territory.)
Back to That Hospital Room
Here’s why this isn’t an academic quarrel about nouns.
A large language model is, at its mathematical heart, a machine for producing the most probable continuation. The consensus answer. The top-K. And the top-K answer for a patient presenting with refractory hypertension is malignant hypertension — the exact diagnosis that was quietly failing me while my blood pressure climbed toward a stroke.
A naive deployment — the “advanced Google” deployment, the model-as-oracle deployment implicit in the autonomous-AI framing — is a machine for delivering the average answer with superhuman confidence and infinite patience. It would have kept me on metoprolol right up until the second stroke. The one that finishes the job.
But the top-K is not a law of nature. It’s a dial. And the harness owns the dial.
A side note, it seems like my entire life these days with AI involves making good top-K decisions. :-)
A well-designed harness does in software what my hospitalist did in defiance: it institutionalizes the search for the 1% of 1%. One agent goes deliberately broad — generate the improbable differentials, the zebras, the case-report material. A second agent interrogates each candidate against the actual evidence: rule it in, rule it out, order the test that discriminates. A supervisor watches the whole thing and refuses to let the system anchor.
Nvidia found exactly this: the breakthrough component was a supervising agent — “almost like a CEO” — that nudges the working agent when it goes off direction, hits a dead end, or re-treads old ground.
That is my hospitalist, rendered in software. That’s not the model being smart. That’s the system being designed by someone who understood that consensus is where rare diagnoses go to die.
Emanuel’s “humans in the loop degrade performance” argument, whatever its merits, is aimed at a human bolted onto the outside of a black box, second-guessing its outputs. Fine.
But the harness isn’t a human in the loop. The harness is the loop.
And declaring the model autonomous doesn’t remove the harness — it just means somebody built it without you asking who, or how well.
What Administrators Should Actually Do
So here is my note to every health system executive currently being handed a vendor deck with a benchmark score in 72-point font:
Ask what harness produced that number.
Not which model. Which orchestration, which memory strategy, which supervisory logic, which tools. If the vendor can’t answer, the number is decorative. The same model can score 30 or 100, can cost X or 2X, can anchor on the consensus or hunt the zebra — depending entirely on the software you never see. A benchmark that treats the model as a black box is measuring a car by weighing the engine.
Understand that “autonomous AI” is a procurement category error.
You are never buying a model. You are buying a model and someone’s opinions about search breadth, verification, escalation, and cost — compiled into software. Even the most general-purpose consumer harness on the market — Claude Desktop is probably the best example — is built, by default and by design, to start from the model as the asset and work outward toward your question. That’s a perfectly good strategy for drafting a memo. It is not the strategy you want when the question is “why won’t this patient’s blood pressure come down,” where the right architecture starts from the question and forces the system to go wide before it converges.
And stop treating vertical benchmarks as gospel.
I mean that word precisely — we’ve been down this road before. Gospel is exactly the wrong genre for an engineering artifact. Benchmarks are gospel the way a crash-test rating for the engine block is gospel. What you deploy is the whole vehicle.
Dr. Emanuel is right that the wave is coming, and right that pretending otherwise just delays the hard conversations about regulation, liability, and payment. But the hard conversation isn’t “machine versus physician.”
It’s this: the intelligence may be in the model, but the judgment — the search strategy, the skepticism, the refusal to anchor — lives in the harness.
We just watched the same model go from 30% to 100% because someone built better scaffolding around it. That’s not a rounding error. That’s the whole ballgame.
I’m alive because one physician refused to accept the most probable answer. The great project of clinical AI is not to build a model that’s usually right. Usually-right is what almost killed me. The project is to build systems that are engineered, deliberately and auditably, to find the 1% of 1% — every time, for every patient, including the ones who don’t happen to draw the intrepid hospitalist on call.
A year ago the lesson was: the guru is you, at your desk, actually using the tools. This year’s corollary: the doctor of the future is not a model. It’s a system — and somebody has to build it with the same stubborn, evidence-demanding, consensus-defying judgment as the woman who saved my life. Your priest and your doctor. The two jobs where the average answer is never good enough.
Sorry Zeke - the answer is far more nuanced than you make it out to be.
The model will see you now. Make damn sure you know who built the harness.
P.S. To our dear Dolly Parton, rest in peace darling, we will always love you.