The Gist: AI is wrong some of the time, and it always will be. So the first question about any AI project is not how simple the task is. It is how big the prize is compared with the cost of being wrong. Most enterprises have this exactly backwards: they start with small, simple tasks to reduce risk, and collect tiny prizes and very public failures. Code took off because the prize is huge and the profession already assumes its workers are wrong. Add an electorate that just voted against data centers and an industry spending about a trillion dollars a year on infrastructure, and the math has to change soon, or the bubble will.
Uh oh. Sid watched a documentary again.
Combine that with one of the more challenging customer questions I got at AI World: “how do I help my company adopt AI?” And a new Substack is born…
This time nobody had to make me. Alex Gibney’s Musk premiered at Venice and opened in theaters in October. It runs three hours and forty-five minutes, with an intermission, which is the first time I have seen anything in the Musk universe admit that a human needs to take over halfway through.
A note to the reader - I’m still halfway through it. It’s long.
A big part of the film is about Tesla Autopilot, and one scene will not leave me alone.
In 2016, Tesla posted a video of a Model X driving itself from a house in Menlo Park to Tesla’s office in Palo Alto.
It stops at red lights. It goes on green. The caption opens with a line I cannot get out of my head:
The person in the driver’s seat is only there for legal reasons.
Musk tweeted that Teslas needed “no human input at all” to get from city streets to the highway.
Then, in a 2022 deposition, Tesla’s director of Autopilot software testified that the drive followed a predetermined route using 3D maps that customers did not have. Drivers had taken over during test runs. And on one attempt to park itself in Tesla’s own lot, the car hit a fence. Gibney puts that deposition on screen.
Then came the next ten years. NHTSA reviewed 956 crashes, concluded that Autopilot did too little to keep drivers engaged and invited “foreseeable misuse,” and Tesla recalled about two million cars in December 2023. In 2025, a Miami federal jury held Tesla partly liable for a 2019 crash in Key Largo that killed Naibel Benavides Leon, who was 22, and awarded $243 million. Tesla is appealing.
The film links Autopilot to more than 60 deaths.
Tesla says its systems are intended for a fully attentive driver.
And this February, under pressure from the California DMV, Tesla stopped calling it Autopilot in California. The product is now called Full Self-Driving (Supervised).
Read that name again. Ten years later, the person in the driver’s seat is still only there for legal reasons. The caption was never a boast. It was a spec.
(Disclosure: I work at Oracle, which has a more than passing interest in AI infrastructure. These views are mine, not Oracle’s. I put that in bold because I like my job.)
The Worst Trade in Tech
Here is the part the documentary does not say out loud, and I think it is the most important idea in AI right now.
Look at the trade supervised self-driving offers. The prize: you do not have to steer. But you still have to watch the road, hands ready, prepared to take over in a split second. That is the job minus the fun part. Anyone who has taught a teenager to drive knows that watching is harder than driving. The cost of a mistake: a funeral.
And the human it competes with is already very good. In 2024 Americans drove about 83 million miles for every traffic death (1.20 deaths per 100 million miles). That is the bar: tens of millions of miles between fatal mistakes, so that you can stop steering.
A tiny prize and a catastrophic downside, against a human who almost never fails. That is the worst risk-to-reward trade in technology, and we built the most famous AI product in America on top of it.
Now go look at your company’s AI roadmap. I’ll wait.
The 10% Machine
AI is guaranteed to fail. I have blogged about this many times. Not occasionally, and not as a bug someone will fix next quarter. It is a probability engine, and probability engines are sometimes wrong. That is how they work. In August I wrote about the average machine: models drift toward the most common answer, which is usually right and occasionally, confidently, wrong.
So let’s say it fails 10% of the time. Pick your own number. It is not zero, and it is not going to be zero.
Now remember that work has steps. At 90% per step, a seven-step task succeeds less than half the time. Ten steps, about 35%. Twenty steps, about 12%. That is why, on the Remote Labor Index, the best agent automated 2.5% of real freelance projects. It is also why a swarm of agents handing work to each other is not a safety strategy. It is compound interest on the 10%.
So the first question about any AI project is not “can it do the task?” It is this:
Run Autopilot through it. The prize is not steering. The mistake nobody catches is a funeral. It fails, no matter how good the demo looks.
Notice the two levers. You can make the prize bigger, or you can make mistakes cheap to catch. The best AI projects do both. The worst do neither, and most enterprises are starting with the worst.
Why Code Won
Here is the counterintuitive part. The first place AI really took off was not the easy stuff. It was one of the hardest jobs in the building: writing software.
Run it through the fraction.
The prize is enormous. Software is leverage. A feature written once runs a million times.
The human baseline is not perfect. Human programmers ship bugs all the time. The AI’s error rate is in the same ballpark as the person it is helping, so it does not look worse by comparison.
Mistakes are cheap to catch. This is the one nobody talks about. Software engineering is the only profession that spent fifty years building infrastructure on the assumption that the worker is wrong. Compilers, type checkers, unit tests, code review, continuous integration, staging environments, feature flags, and the greatest invention in the history of labor:
git revert.
Drop a fallible AI into that and it inherits the safety net. Drop it into a dark intersection at 62 miles an hour, and there is no git revert.
Nobody had to write a “human in the loop” policy for code. The loop was already there. It was called a pull request.
The Drive-Thru Fallacy
Now watch what most enterprises do. They make a list of AI candidates and sort it by difficulty. Start simple, they say. Reduce the risk.
So they automate the drive-thru order, the password reset, the refund question and the FAQ. Here is the problem. Simple tasks are simple because humans already do them nearly perfectly, for very little money, so the prize is small. And they face the customer, so every failure happens in public. You have picked the smallest prize with the most visible downside. You have built Autopilot for the help desk.
Bad Behavior #1: The Drive-Thru. McDonald’s tested IBM’s voice ordering at about 100 drive-thrus from 2021, then shut it off in 2024 after viral videos of orders gone wrong: a large amount of butter, ice cream topped with bacon, and hundreds of Chicken McNuggets. Regular readers will remember the 260. The prize was part of an order-taker’s shift. The cost was a meme.
Bad Behavior #2: The Policy Inventor. Air Canada’s chatbot told a grieving passenger he could claim a bereavement discount after he flew. The real policy said no. Before a tribunal, Air Canada argued the chatbot was “a separate legal entity that is responsible for its own actions.” The tribunal disagreed and ordered C$812 in damages and fees. At least Tesla kept a person in the seat for legal reasons. Air Canada tried to make the chatbot the person.
Bad Behavior #3: The $1 Tahoe. In December 2023, a user talked a Chevrolet dealer’s chatbot into agreeing to sell a 2024 Tahoe for one dollar. Its reply: “That’s a deal, and that’s a legally binding offer – no takesies backsies.” No human sales team has ever closed that fast.
Bad Behavior #4: The Self-Roasting Help Desk. In January 2024, after a system update, DPD’s chatbot refused to connect a customer to a human, swore at him, called DPD “the worst delivery service in the world” and wrote a poem about it. DPD switched it off. To be fair, it was the most honest customer service exchange in the history of parcel delivery.
Bad Behavior #5: The Digital Labor Salesman. Klarna bragged that its assistant did the work of 700 people, then its CEO conceded that cost had been too big a factor and quality suffered. Commonwealth Bank of Australia handed 45 jobs to a voice bot and then reversed itself. The voice bot remains very confident.
Bad Behavior #6: The Rubber Stamp. You know this one, because you have probably approved it. A “human in the loop” clicking Approve on 400 AI outputs an hour. The person in the chair is only there for legal reasons. NHTSA has a phrase for where that design ends up: foreseeable misuse.
Every one of these was chosen because it looked safe. Every one was a small prize. Every one failed where customers could watch.
Your Software Playbook Is the Problem
Why do smart enterprises keep doing this? Because for forty years, it was exactly the right way to adopt software.
You piloted the simplest process, proved it, then scaled. It worked because traditional software is deterministic. If the expense system calculated tax correctly on Monday, it calculated it correctly on Tuesday, and every day after. Pass the pilot once and you had proven it.
Consumers learned to love software the same way. The calculator never lies. The button does the same thing every time. We built trust through repetition.
AI breaks both contracts. Passing the pilot proves nothing about the next thousand runs. The simple task is exactly where the human was already nearly perfect, so the machine looks worse by comparison. And customers trained by forty years of reliable software now meet a button that is right nine times out of ten and never says which time is the tenth.
That is not a product people learn to love. It is a product people learn to revile.
So turn the playbook upside down. Sort your list by the size of the prize and the cost of a mistake nobody catches. Sort by simplicity last. Start where the human baseline is weak, the prize is big, and mistakes are cheap to catch.
Capability Is the Big Prize
So where are the big prizes? Not in doing the same job faster. That is still the same job, and it is the first thing the next model eats. The big prize is the job your people could not do at all.
The accountant runs a forecast on a tabular foundation model, then uses a linear solver that used to be out of her league, and optimizes the budget instead of just reporting it.
Every biochemist is suddenly good at bioinformatics.
Every medical researcher can take part in machine learning for drug discovery and protein folding.
And no, before you say it, the agent cannot just write oodles of Python. Writing the Python was never the hard part. The hard part is everything underneath: abstracting the compute so she never sees a cluster, abstracting the data so she never reads a schema, and carrying the business context and tribal knowledge, like which revenue table finance actually trusts.
Here is why this matters for the fraction. Capability is the only strategy that moves both sides at once. It makes the prize bigger, because new work is worth more than cheaper old work. And it puts an expert in the driver’s seat: someone who knows the answer well enough to catch the 10%. That is the opposite of a person who is only there for legal reasons.
If your ROI needs a layoff, it is not an ROI. It is a severance plan. After Oracle AI World I wrote that nobody hugs a product that is coming for their job.
P.S. Nobody votes for one either.
Now Add Voters
The Drive-Thru Fallacy is a bad trade for the enterprise. It is a worse trade politically. The tasks companies automate first (the order-taker, the call-center agent, the junior analyst) are the jobs voters can see. The company saves a little. The worker loses everything. And the failures happen in public.
Last Tuesday, voters got their first real chance to answer.
Congress flipped. Democrats took the House and the Senate, 54 to 44 as of Wednesday morning, with James Talarico winning in Texas and Roy Cooper in North Carolina.
Texas changed landlords. SemiAnalysis expects Texas to host 41% of new U.S. data center capacity in 2028. Texas just elected a governor who wants new data centers paused until the state writes rules on power, water and property rights.
The towns were not close. There were 41 local data center measures, and in 28 of the 38 with results, support for banning or pausing reached at least 70%. Defiance, Ohio capped new data centers at 25 megawatts. Yes, Defiance. No editor would let me make that up.
Voters said it mattered. In NBC’s exit poll, 56% called AI and data centers a crucial or major factor in their vote.
You cannot vote on whether your boss buys an agent. You can vote on the building it lives in. The data center is the only part of AI with a street address. Republican strategist Rob Stutzman told Politico that the fight is really about AI safety and job protection, and that the data center ban is “the tool they have to fight.” OpenAI’s answer, per Axios, is to rebrand data centers as “AI compute campuses.” You cannot rename your way past a zoning board.
So the cost side of every replacement project just grew a new line: political risk. Warner and Hawley’s AI-Related Job Impacts Clarity Act would put layoffs blamed on AI on a federal form. House Democrats want to tax tokens at a rate that rises with unemployment. A token, I should point out, is whatever each vendor’s tokenizer says it is. In 1784 Britain taxed bricks by the thousand, so a man named Joseph Wilkes started making enormous ones, known to this day as Wilkes’ gobs. Never tax a unit the taxpayer defines.
And the industry wrote the prosecution’s script itself. When Anthropic’s CEO warned that AI could wipe out half of entry-level white-collar jobs and push unemployment to 10 to 20%, it was meant as a warning. Josh Hawley cited unemployment of as much as 20% when he filed the layoff bill.
A warning from the seller reads like a product spec.
Steve Kornacki would remind you that 54 is not 60, and the veto pen stays where it is until 2029, so none of this becomes federal law soon. It does not need to. Moratoria, zoning boards and disclosure rules do the work locally, and every one of them raises the cost of being wrong.
How Bubbles Actually Pop
Now the money. UBS projects that the big cloud and AI companies will spend about $1 trillion on capital expenditure in 2026, roughly double 2025. Amazon, Alphabet and Microsoft are estimated to be putting about 102% of their cloud revenue back into it. Meanwhile, MIT’s NANDA team found 95% of organizations getting zero return from their AI projects.
Somebody has to pay that bill, and in the end it is enterprises. (My customer)
If they keep spending their AI budgets in the Autopilot trap, buying small prizes with public failures, and voters keep raising the cost of every attempt, the revenue never shows up to pay for the trillion.
Bubbles do not pop when the technology fails. The railroads worked. The fiber worked. Bubbles pop when the use cases cannot pay the bill. In August I warned about AI stagflation, with valuations outrunning real value. The midterms just shortened the fuse.
So yes: unless we rationalize this soon, the AI bubble will burst, and a lot of money will go with it.
Let Me Cross-Examine Myself
Isn’t Waymo proof that self-driving works? Yes, and it proves the point. When the prize is an entire taxi business, you can justify spending well over a decade and billions of dollars to buy the extra nines inside a geofence. That is what raising the prize looks like. Nobody is going to spend that to automate a password reset.
Won’t the error rate keep falling? Probably, task by task. But as models improve we hand them longer tasks, and compounding eats the gains. The number is not going to zero.
Isn’t the election a vote against AI? No. In CNBC’s October survey, 63% of voters opposed a data center within 10 miles of home, yet only 32% trusted Democrats to handle the issue and 24% trusted Republicans. Voters are not against AI. They are against paying for the parts of it they can see.
Could the bubble just deflate slowly? Maybe. Per-token prices have been falling about tenfold a year, and stocks rose more than 1,000 points the morning after the election. Cheap tokens can make a lot of bad math look good for a while.
What to Do on Monday
Here is what actually works, and it is about as mystical as a pull request.
Score every use case on the prize and the cost of a mistake nobody catches. Score it on simplicity last.
Never automate a task humans already do nearly perfectly for a small prize. That is the Autopilot trap.
Start where the human baseline is weak and the prize is big. Code was first. It will not be the last.
Make mistakes cheap before you make the AI smart. Tests, reviewers, rollbacks, audit trails. Build the
git revertfor your domain.Put an expert in the loop, not a bystander. If the human is only there for legal reasons, you have built Autopilot.
Budget tokens per accepted outcome. Count the failed attempts. Every extra step is compound interest on the 10%.
Delete the FTE row from the business case. If the project dies without it, you have your answer.
Then run what I call the Defiance Test. Imagine defending your AI roadmap in a school gym in Defiance, Ohio, to the people paying the electric bill. Whatever you would be embarrassed to say in that gym, stop doing now.
In Closing: One Last Story
Go back to that caption:
The person in the driver’s seat is only there for legal reasons.
Ten years later, it is the most accurate description I know of how many companies plan to use their people around AI. Present, liable and idle.
Flip it. Put your people in the driver’s seat of something much bigger than a commute. Let the accountant run the optimization and the biochemist run the pipeline, and let the AI take the cheap risks on the big prizes.
Boy, am I tired of leaders asking me, “So Sid, which simple tasks should we automate first?”
Wrong question. What’s the prize?
Now, if you’ll excuse me, I need to go to a drive-thru and order six Chicken McNuggets.
Wish me luck. Last time I got 260.
P.S. If your first instinct is to forward this to your Head of AI Strategy to “action,” you have missed the point, same as last time. Open your AI roadmap. Sort it by the size of the prize instead of the size of the task. If password resets are at the top, start over. That’s it. That’s the strategy.