As I was verbally abusing Opus for the 13th time in the last hour, I decided, counter to the fanboys who are announcing that Opus 4.7 will conquer the world, to publish a few thoughts…

Individual tasks come back tighter, cleaner, with fewer of those little tells that make you squint and mutter “what are you doing, friend?” at your screen at 11pm on a Tuesday. Correctness, on the task level, has moved. I’m not going to pretend it hasn’t.

Now. Here’s the thing, and it’s not a small thing.

Neither is multi-repo coordination and support. And multi-agent coordination still is not friction free. Really?

SendAgentMessage isn’t available mid-stream? Can’t sneak a message into that agent’s context window? Something seems broken in the agent coordination and orchestration mechanism.

Because left to its own devices, the model will confidently, articulately, lovingly produce the average answer to the question I literally asked — not the question I should have been asking.

I wrote about this a while back in The Average Path. The thesis holds. Opus 4.7 is a more accurate average path. It is not a different path. It is not your path.

And listen — the middle of the road is fine. For a lot of work, the middle of the road is exactly what you want.

But if you came here because someone on a podcast told you the software industry is doomed, I have news for you, and the news is good: it is not doomed. It is, in fact, doing just fine, thank you, because turning vision into systems — real systems, systems that survive contact with customers — is not something that falls out of a language model when you ask it nicely.

You still have to know what you want. You still have to know why. And if you hand a model your half-formed intuition and say “proxy my vision,” you will pay for that. Not metaphorically. Actually. In tokens, in rewrites, in three-week detours that end with you reverting to the branch you had before you started.

About Mythos

This is the part where Anthropic waves Mythos at me. And I want to be fair about this, because I’m about to be unfair.

That’s real. That’s a genuine achievement, and I’m not going to stand here and pretend otherwise — this is a meaningful moment for the industry. Full stop.

But.

(You knew there was a but.)

They ran the thing thousands of times. The cost was roughly ten thousand dollars. And I will take the bet, right now, today, in writing, that if you ran Opus 4.7 a couple thousand more times at maybe twenty-five thousand dollars in compute, you’d find the same zero-day. Or one just like it. Because what we are describing, when we describe this, is not a new kind of cognition.

That’s not a criticism of the achievement — it’s a description of the achievement.

The right read here is token optimization. The right read is affordability. The right read is: security researchers have been producing zero-days for roughly ten thousand dollars of labor for years, and the resulting vulnerabilities have generated billions in value. So before we gasp at the economics, let’s actually look at the economics.

Also, between us — was the name an accident? Mythos? Was that on purpose? Nobody in that room said “hey, maybe we shouldn’t name the model after a word that means things people made up“? Nobody? All right. Moving on.

Productivity. Not replacement.

This is a productivity play. Not a replacement play. A productivity play. And I say that as someone who uses these models heavily, every day, in anger, in production, for real work.

The tools are getting better. My leverage is going up. My need to hire is going — actually, my need to hire is going up too, because now I can start more things, and starting more things means more people, not fewer.

But “productivity tool” does not move a valuation the way “replacement for all knowledge work” moves a valuation. So the CEOs keep saying AGI. They keep fear mongering total replacement.

They keep describing a future in which I, personally, am obsolete by the end of the fiscal year. And I look at my terminal, where I have just spent forty-five minutes patiently explaining to the most advanced model on the market that no, we do not want to refactor the whole module, we want to fix the one line, and I find myself unmoved.

The actual problem

Put a pin in the valuations for a minute, because there’s something else. Something I think is more interesting, and nobody’s talking about it.

That is not a technology problem. That is not a model problem. That is not something that gets fixed by Opus 4.8 or Opus 5 or whatever we’re calling the next one.

And it is the actual problem. And it is not being solved by benchmarks. It is, in fact, being actively not solved by benchmarks, because the people writing benchmarks are not the people staring at the blinking cursor.

Benchmark-obsessed versus customer-obsessed. I keep saying it, it keeps being true, nobody keeps doing anything about it.

The battle is now quietly turning towards token optimization versus capability and breaking the adoption barriers, starting with improving the customer experience and consumption model.

I wrote a piece recently called Allbirds is Pivoting to AI Infrastructure. The title is a joke. The underlying point is not. When the narrative gets this overheated, everyone pivots to infrastructure, nobody pivots to the customer, and the companies that eventually win are the ones who remember which side of the equation actually signs the checks.

Where I land

The capability plateau is real. Not because the models aren’t improving — they are, measurably, and I’m grateful for it.

The plateau is real because “better average” is bumping up against the ceiling of what “average” can do for you when what you need is judgment. And judgment doesn’t live in the weights.

Which, apparently, is still my job.

Back to the trenches.

I’ve been staring at this headline for about twenty minutes now, and I keep reading it again because I’m convinced — convinced — that at some point the words are going to rearrange themselves into something that makes sense. They haven’t yet.

Allbirds — the shoe company, the merino wool people, the ones who made comfortable footwear for guys who describe themselves as “product thinkers” — is pivoting to become an AI infrastructure company. Their stock exploded 175%. And I need to sit with that for a second, because I think we’ve collectively lost the plot.

Let’s Talk About What Infrastructure Actually Means

When Oracle builds out GPU infrastructure, they’re not just racking servers. They’re engineering datacenters from the ground up with bare-metal GPU clusters connected via RDMA networking over InfiniBand — technology that traces back to NVIDIA’s acquisition of Mellanox in 2020.

That gives OCI ultra-low-latency GPU-to-GPU communication across superclusters of tens of thousands of GPUs, which is not a nice-to-have — it is the bottleneck for large-scale AI training. It’s why companies running massive training jobs have turned to Oracle over AWS and Azure for raw cluster performance. That’s not a business you stumble into.

And OCI is just one example. When AWS or Azure does it, they’re layering decades of cloud engineering, custom silicon, and battle-tested orchestration on top of raw compute. They’re building differentiated, AI-optimized software stacks that make enterprise adoption possible — not just plausible, but manageable.

These companies are spending tens of billions of dollars. Tens. Of. Billions. And they’re doing it with thousands of engineers who’ve been thinking about distributed systems since before “GPU-rich” was a phrase anyone said out loud.

Fifty million dollars in AI infrastructure is a seed round. It’s a promising seed round, sure. It’s enough money to get a meeting. It is not enough money to build a moat. It is not enough money to compete with hyperscalers. It is barely enough money to learn how expensive the electricity bill is going to be.

You don’t walk into a knife fight with a business plan and a wool shoe. You just don’t.

The Commodity Trap Is Right There, and It’s Wearing a Name Tag

Here is the thing that is making me want to pace around the room:

Full stop. You are buying a thing from NVIDIA and selling access to that thing to someone else, and the only differentiator you have is price — which means your margins compress, which means you need scale, which means you need capital that dwarfs what a mid-cap footwear brand can raise in a secondary offering.

It is so tempting, I understand that. The gravitational pull of going down the stack, of becoming the infrastructure layer, of being the one selling shovels — I get the seduction. But “tempting” and “wise” parted ways a long time ago on this one. The hyperscalers aren’t standing still. They’re layering proprietary services, custom chips, managed AI platforms, and enterprise compliance tooling on top of raw compute. They’re building the building. Allbirds is proposing to rent a room in someone else’s building and sublet it.

Meanwhile, the Actual Opportunity Is Staring Us in the Face

You want to know where the real value is? You want to know what keeps me up at night — not with dread, but with that restless, pacing energy of knowing something enormous is sitting right there?

The usability gap.

Right now, there is a chasm — and I mean a chasm — between the small percentage of people and companies burning through AI tokens at a breathtaking pace and the vast majority who haven’t opened a prompt window in a week. Ninety percent. Ninety percent of the potential market is sitting on the other side of a bridge that hasn’t been built yet. That is the crossing-the-chasm moment for AI. Not compute. Not GPUs. Figuring out why most people and most companies haven’t found their way into this thing yet, and then building the on-ramp. That is a trillion-dollar problem that is begging — begging — for someone to solve it.

And there’s another one. We have spent decades — decades — building Web 2.0 APIs, software interfaces, data architectures, all designed for human consumption. GUIDs, Social Security numbers, ZIP codes, RFID tags — the common identifiers that power modern software were designed for systems that parse structured data in predictable ways. But a language model doesn’t think in ZIP codes. It thinks in tokens and semantic relationships.

When you’re optimizing for latency, minimizing reasoning overhead, and operating at the speed these models demand, the entire identifier layer of modern software may need to be rethought.

Taking the accumulated infrastructure of the internet era and re-optimizing it for a world of LLMs, semantic search, and token economics? Trillions of dollars. Trillions. And it requires exactly the kind of creative, software-driven thinking that a nimble company could actually pull off.

So What Are We Doing Here?

I’ll be honest with you. I don’t have a satisfying explanation for why this happened. I don’t know what was said in that boardroom. I don’t know what pitch deck made this seem like the move. What I know is this: when a shoe company announces it’s becoming an AI infrastructure provider with fifty million dollars in capital and no discernible technical differentiation, and the market rewards it with a 175% stock surge, something has gone sideways.

The opportunity in AI is real. It is profoundly real. But the opportunity is in the hard, unglamorous work of closing the usability gap. It’s in reimagining how software talks to models. It’s in the layers of intelligence and integration that sit between raw compute and actual human value.

It is not in buying GPUs and renting them back out.

I don’t know. Maybe I’m wrong. Maybe Allbirds has a plan I can’t see from here. But I’ve been doing this long enough to know what a commodity trap looks like, and this one is textbook.

The insanity continues. And I, for one, am going to keep pacing.

Let me say that again, because what follows might suggest otherwise. I love coding agents.

I have produced more functional, tested, demonstrable software in the last two months than I have in the last two years. Maybe longer. The velocity is intoxicating, and I don’t use that word lightly — I use it because intoxication impairs judgment, and we should probably talk about that.

Here’s what happened. Twenty-plus years of building systems — memory mapping, streaming architectures, threading models, distributed consensus, the whole cathedral — suddenly became a lever instead of a résumé line. All that accumulated intuition about how computers actually work underneath the abstractions?

It became the difference between using a coding agent and being used by one. I could push fearlessly into designs that would terrify someone who learned software from a textbook, because I’d already broken these things with my hands and put them back together in the dark.

It has been, without exaggeration, extraordinary.

But.

There’s a structural problem with large language models writing software, and it’s not the one the doomers talk about, and it’s not the one the boosters dismiss. It’s more fundamental than either camp wants to admit, and it hides in plain sight inside every line of code these models generate.

Every design decision, every architectural fork, every moment where a system could go left toward elegant efficiency or right toward “good enough” — the model goes right. It goes right because it was trained on the corpus of all software ever written, and the corpus of all software ever written is, to put it charitably, not great.

The model doesn’t optimize for speed. It doesn’t optimize for efficiency. It doesn’t think about memory hierarchies or cache line boundaries or what happens when your dataset outgrows the machine you’re sitting at. It can’t Think Big — capital T, capital B — because thinking big requires the willingness to throw away the obvious approach in favor of the non-obvious one, and non-obvious approaches are, by definition, underrepresented in the training data.

What the model actually does is produce the solution that the largest number of software engineers would have written. Which sounds fine until you remember what the distribution of software engineering quality looks like.

Let me make this concrete, because concrete is where the bodies are buried.

I needed to load a large dataframe stored locally in Arrow format. Arrow — for the uninitiated — is a columnar, in-memory format designed for analytical workloads. It’s fast, it’s compact, and it scales beautifully if you know what you’re doing.

The coding agent loaded the entire file into RAM.

Now, most software engineers would do the same thing. Open the file. Read it into memory. Operate on it. This is the canonical approach. It’s in every tutorial. It’s on every Stack Overflow answer. It is, indisputably, the average path.

It also means your application scales linearly with file size, and your ceiling is whatever physical RAM happens to be installed in the machine. Which isn’t an engineering decision — it’s a surrender.

I knew immediately what to do: memory-mapped I/O. Let the operating system’s virtual memory subsystem handle the paging. Break the coupling between file size and available RAM. The data lives on disk; the OS pages in what you need, when you need it, and pages out what you don’t. No performance degradation compared to a full in-memory read. Works beautifully in streaming contexts. Saved me — and this is not hyperbole — tens of thousands of lines of code that would have been required to implement an inferior, disk-bound alternative format.

But here’s what was actually concerning: the model didn’t just miss this approach. When I pushed toward it, the model pushed back. For a surprisingly long time, it tried to convince me — with the confident, measured tone of a senior engineer who’s seen this movie before — that we were physically bound by available RAM. That this was simply a constraint of the problem. That perhaps we should consider alternative file formats.

It was giving me the median Stack Overflow answer with the conviction of someone who’d never questioned whether the median Stack Overflow answer was any good.

And this is where we need to talk about the distribution.

There’s a well-known empirical observation in software organizations: your best software comes from roughly ten percent of your talent. This isn’t controversial. This is why every FAANG company runs a brutal calibration process. This is why performance reviews exist. This is why the difference between a Staff Engineer and a Senior Engineer isn’t just seniority — it’s a fundamentally different mode of thinking about problems.

Which means that roughly ninety percent of the training data for these models — the code, the Stack Overflow answers, the blog posts, the documentation, the architectural decisions captured in a million repositories — comes from engineers operating below the level where the really interesting design decisions get made.

So when a coding agent writes your software, it writes it the way the median engineer would. Not the worst way. Not the best way. The most common way. And the most common way is how you get systems that work fine at demo scale and collapse under production load, that pass every test and fail every stress test, that solve the problem stated in the ticket and miss the problem lurking underneath it.

Now, someone in the back is already raising their hand.

“But compute is cheap. RAM is cheap. Why does this matter?”

It matters because you don’t live in a cloud provider’s marketing brochure.

I have watched — personally, with my own eyes — hardware bottlenecks kill enterprise systems. Not “slow them down.” Kill them. And here’s why: the vast majority of enterprise software is still not fully CI/CD. It’s not cloud-native. It’s not running on immutable infrastructure with autoscaling policies and graceful degradation. It’s running on machines that a human being provisioned, that another human being patched last Tuesday, that a third human being will troubleshoot at 3 AM when the OOM killer starts making editorial decisions about which processes deserve to live.

RAM is free in the abstract. In the enterprise, RAM is a purchase order, a change request, a capacity planning meeting, and a six-week procurement cycle. And I’m not yet convinced that CIOs are ready to hand security, data quality, preservation, and availability decisions to AI agents. That’s not conservatism — that’s pattern recognition.

Which brings me — finally, stay with me — to the thesis I’ve been circling.

I’ve sat through enough keynotes and earnings calls and breathless LinkedIn posts to have internalized the dominant narrative: AI will replace human workers. AI will do the work of ten people.

AI will reduce headcount and increase margin and your board will love you for it.

And I’ve heard the counter-narrative from every AI CEO with a product to sell: “We’re not replacing humans! We’re augmenting them! We’re helping people do things they couldn’t do before!” Which always sounded like marketing horseshit. The kind of thing you say to a Senate subcommittee while your sales team is pitching CFOs on headcount reduction.

But what if they’re telling the truth? What if they’re being quietly, almost accidentally honest about the real economic model?

Because here’s what I’ve actually experienced, not theorized: the value of coding agents is not evenly distributed across the talent spectrum. The value is catastrophically concentrated at the top. When someone with twenty years of systems intuition uses a coding agent, the agent becomes a force multiplier of staggering magnitude. When someone without that intuition uses the same agent, they get the average path. Faster, yes. But still average. Still fragile. Still doomed to the same scaling failures and architectural dead ends that characterize the bottom ninety percent of all software ever written.

The real TAM isn’t built on replacing human salaries. The real TAM is built on making the top ten percent of talent capable of producing the entire output of the remaining ninety percent.

Read that again.

This isn’t about replacement. This is about concentration. The best minds, paired with models, doing things that were historically impossible not because the work was too hard — but because there weren’t enough hours in the day and there was too much organizational overhead between the idea and the artifact.

And this is where it gets uncomfortable.

Are we ready for a world where ten percent of the workforce delivers the entire value of the other ninety? Because that’s not a labor market adjustment. That’s a structural transformation of how organizations create value, how careers are built, how compensation is distributed, and who gets to participate in the upside.

Not every company can afford the top ten percent. Not every company can even identify the top ten percent. And the top ten percent has a very specific characteristic that is extraordinarily difficult to teach: they have an intuitive, experiential understanding of how systems actually work — not how textbooks say they work, not how certification programs describe them, but how they behave under real conditions at real scale.

Note to AI investors: The total addressable market (TAM) of most AI initiatives has a secret assumption in replacing human productivity. With our current model architectures, this TAM can only be realized if the customer will consistently pay for an average outcome, unless it is simultaneously combined with extraordinary human talent or tools encoded with this human talent.

I’m self-taught. Every concept I carry — memory mapping, streaming, threading, distributed failure modes — I learned by breaking things and fixing them, by reading source code at 2 AM because no documentation existed, by building systems that fell over and figuring out why. That knowledge lives in my hands, not in my head. It’s instinct, not information.

And I genuinely worry that the current state of computer science education — which teaches concepts through textbooks and problem sets rather than through the kind of reckless, fearless, self-directed exploration that builds intuition — is producing engineers who can describe these concepts but can’t feel them.

And if you can’t feel them, you can’t evaluate the coding agent’s output. You accept the average path because you don’t know there’s a better one.

The traditional work breakdown of software organizations doesn’t make sense anymore. The old model — decompose a project into small tasks, assign each task to an individual contributor, roll up the results — was designed for a world where humans were the unit of execution. Coding agents are better at bigger tasks with defined, testable outcomes than they are at incremental improvements to existing systems. They want scope and direction, not tickets and story points.

The common units of scaling teams - “one engineer can handle three customers, so when we get 30 customers, we need 10 engineers” - just doesn’t compute anymore. It’s a non-linear distribution, not evenly exponential either. Now you may need talent that is five times more expensive to handle twenty customers. But is that extra customer per dollar spent worth it? And then there is the chance that the expensive talent will create an innovation that attracts another 20 customers that you would not get without the talent - a hidden opportunity leverage (or cost).

This suddenly became an extremely complicated ROI decision for the economic buyer, and the TAM of an AI company highly questionable.

Which means the person directing the agent needs to operate at the level of the whole problem, not the level of the individual function. They need to see the entire system in their head, understand where the scaling walls are, know which “best practices” are actually just popular practices, and have the confidence to override the model when it’s walking them toward a cliff.

That’s not a junior engineer’s skill set. That’s not even a senior engineer’s skill set, in many organizations. That’s the skill set of someone who’s been through enough wars to know the difference between the way software is supposed to work and the way it actually does.

So here’s where I land, and I know this will make people uncomfortable.

I haven’t seen LLMs break through to doing the work of the top ten percent natively. And mathematically, that makes perfect sense — you can’t train your way to the tail of the distribution when your training objective optimizes for the center of it.

What I have seen is the top ten percent using LLMs to do things that previously required an army.

The AI revolution isn’t a replacement algorithm. It’s an amplification function. And like all amplification functions, it amplifies whatever signal you feed into it. Feed it mediocrity, you get mediocrity at scale. Feed it brilliance, you get brilliance at a velocity that was previously unimaginable.

The question isn’t whether AI will change how software is built. It already has. The question is whether we’re honest about who it changes things for — and what happens to everyone else.

Tomorrow I turn 45. But almost as though it is yesterday, I can remember watching my mother make carrot halwa on a stove. I don’t remember the date. I don’t remember what the occasion was - other than it was my birthday. But I do remember the smell of the heavy whipping cream getting boiled, the plain scent of carrots being boiled in it, turning sweet as pectin was released, and the incredibly patient, deliberate way that my mother kept turning the frothing, boiling mixture - like a surgeon who’d done this ten thousand times and intended to do it ten thousand more. She was playing the interesting yet dangerous game of deciding how high the temperature could be to maximize pectin release, while avoiding risking burning the family’s yearly budget of half and half, and completely ruining her kid’s birthday.

Now here’s what’s interesting. I haven’t thought about that moment in years. But last week, I watched an engineer debug a failing orchestration pipeline with that same deliberate stillness, and the memory surfaced — unbidden, recontextualized, useful. Not as a transcript of what happened in 1988, but as a framework for understanding patience under technical duress in 2026. Semantically there is absolutely no relationship between making halwa and CI/CD pipelines. In fact, personally I have to wonder why I have made a relationship between something sweet and something so sour, but I digress.

But that’s also human memory. And the essence of where ideas come from, whether it was Einstein’s thought experiments or my designs on eating carrot halwa.

You know what isn’t memory? A markdown file.

The Great Conflation

There’s a pattern emerging across the AI agent ecosystem that I find — and I’m going to be charitable here — intellectually troubling. Agent frameworks are creating files called MEMORY.md, stuffing them with compressed observations, and then declaring the memory problem solved.

Some have gone further: agents now steal each other’s memory files to bootstrap their understanding of a project, a codebase, an initiative. Others throw task IDs against these memories and consider them “organized.” The really clever types put a human ID on the memory - but a human ID doesn’t make it human. Nor does it build relationships.

We even see Claude Code do it with it’s clever marketing positoning: “saving a memory for later now.” I went and read these memories… they’re a bad example of semantic state.

And the industry is nodding along like this is progress.

It isn’t progress. It’s a category error. And category errors, left uncorrected, become architectural debt that compounds until the whole edifice collapses under assumptions nobody bothered to challenge.

What these systems have built is state. Persistent state. Serialized state. Occasionally useful state. But state is not memory, and the difference isn’t semantic — it’s structural, computational, and, I’d argue, fundamental to what we’re going to need if we want agents that actually reason across time.

State vs. Memory: A Distinction That Matters

Let’s be precise, because precision matters here.

State is a snapshot. It’s a data structure that captures the current known values of a system at a point in time — or, in the case of MEMORY.md, a lossy compression of accumulated observations flattened into prose. State answers the question: what do I currently know? State is flat. State is inert. State sits there waiting to be read.

Memory is a process. It’s reconstructive. It’s contextual. It’s an active computation that, given a retrieval cue, searches across episodic and semantic stores, identifies associatively relevant experiences, and synthesizes a novel summary shaped by the context of the current inquiry. Memory doesn’t answer what do I know? — it answers what is relevant to what I’m thinking about right now, and how should I frame it?

This is not a subtle distinction. This is the difference between a filing cabinet and a research assistant.

When a human talks about the current geopolitical turmoil in Iran, they might recall the Iraq War — not because someone filed “Iraq War” under the heading “Middle Eastern Conflicts to Reference Later,” but because an associative retrieval process found episodic traces that share semantic proximity with the current conversational context, and then reconstructed that memory with emphasis on the dimensions that matter right now. The memory of the Iraq War surfaces differently when you’re discussing Iranian diplomacy than when you’re discussing veterans’ healthcare. Same underlying episodes. Different reconstruction. Different utility.

The Compression Trap

Agent authors — smart, well-intentioned people — will push back here. They’ll say: “When we read MEMORY.md, we’re reading it within a conversational context. The LLM reinterprets the content dynamically.”

Yes. And this is where we need to be honest about what’s actually happening.

When you write an observation into a memory file, you’re performing lossy compression. You’re taking a high-dimensional experience — the full context of an agent’s interaction, the decision tree it traversed, the alternatives it rejected, the uncertainty it felt — and you’re flattening it into a sentence or a paragraph. You’re discarding the dataframes. You’re collapsing the probability distributions into point estimates. You’re throwing away the episodic richness that would allow for flexible reconstruction later.

And then, yes, when you read that compressed artifact back in a new context, the LLM does what LLMs do — it interpolates. But interpolation over compressed data isn’t memory retrieval.

You’ve lost the grain, and no amount of contextual reinterpretation can recover information that was discarded at write time.

This is, if I may borrow from information theory, the fundamental problem: you cannot reconstruct what you did not retain. Claude Shannon told us this in 1948. We should probably listen.

Now let me be clear. The word hallucination has become a dirty word. By definition, when we read this now in the context of the AI industry, we all immediately go: “AI wrong, human good.” But I actually disagree. Hallucinations are also the origin of some amazingly good ideas that change human history.

So not all hallucinations are bad. The problem though is that the AI industry needs to lean into this by creating multiple forms of memory, often indexed across many different temporal, semantic, reasoning, and spatial contexts, with different levels of “temperature” in order to achieve the same gradient as true memory and concept linking.

If you are about to tell me this feels a bit like the traveling salesman problem on steroids, you’re right.

The worst part is that the tokens in true memory aren’t just semantic. They’re temporal and spatial as well. While you can see OpenAI and Anthropic attempt to bolt on specialized heads (or subordinate task models) to bring these type of tokens to their systems - they are still a second order concept. The primary is language, in the theory that language can encode everything.

The Three-Pillar Problem

Here’s how I think about the AI systems landscape, and I think this framing might be useful.

In the early days of cloud computing, we understood the problem as three pillars: compute, storage, and database. You needed processing power. You needed a place to put bytes. And you needed a structured, queryable system that could organize, index, and serve data with semantic awareness. These weren’t the same thing. EC2 wasn’t S3 wasn’t RDS, and collapsing any two of them would have been architectural malpractice.

I kind of feel that AI has the same tripartite structure:

Compute → Models. The LLMs themselves. The inference engines. We’ve made extraordinary progress here. GPT-4, Claude, Gemini — these are remarkable machines for transforming input into output. Compute is not our problem.

Storage → Context. The context window. RAG pipelines. Document retrieval. Vector stores. The mechanisms by which we surface information into the model’s working set. We’re getting better at this. Context windows are expanding. Retrieval is improving. We’re not done, but we’re on a trajectory.

Database → Memory. And here’s where we have a gaping hole.

We don’t have a memory system for AI agents. We have state files masquerading as memory. We have append-only logs. We have compressed summaries. What we don’t have is anything resembling what a database provides in the cloud computing analogy: a structured, indexed, queryable system with its own internal ontology, capable of associative retrieval, contextual reconstruction, and multi-resolution access to historical experience.

Nobody has built RDS for agent memory. We’re all still writing flat files and calling it a database.

What Real Agent Memory Would Require

So what would it actually take? Let me sketch the architecture, because I think this is where the industry conversation needs to go.

An episodic store with high-dimensional retention. Don’t compress at write time. Store the full episode — the context, the decision, the alternatives, the outcome, the uncertainty. Store it at the resolution you’d need to reconstruct it flexibly later. Yes, this is expensive. Yes, storage is cheap. Make the trade.

A semantic association layer. Not just vector embeddings (though those are part of it). A genuine ontological structure that captures relationships between episodes — causal links, temporal sequences, thematic clusters, contradiction graphs. When an agent remembers, it shouldn’t be doing a flat cosine similarity search. It should be traversing a knowledge graph that encodes how its experiences relate to each other.

Before people start spamming me with Neo4J articles and Cypher queries, I’m not sure graph databases are a good thing for this. Why? Because this association layer has to be continuously encoded to work right. In other words, you are constantly re-encoding the associations and ontology based on incoming encounters, queries, and memories of these encounters and queries. And then storing it at various different temperatures to decide what version should be recalled for a court room or a bar. Graphs - and the query mechanisms that come with them - are too rigid for these encodings.

Contextual reconstruction at read time. This is the critical piece. When an agent retrieves a memory, it shouldn’t get back a pre-written summary. It should get back the raw episodic material, filtered and weighted by the current retrieval context, and then generate a novel synthesis that’s shaped by what it’s trying to do right now. The same underlying memory should produce different outputs when retrieved for different purposes.

Multi-resolution access. Sometimes you need the gist. Sometimes you need the detail. A real memory system would support queries at multiple levels of granularity — from “what’s my general experience with microservice architectures?” to “what specific failure mode did I encounter in that Kubernetes deployment on March 3rd, and what was the pod configuration?”

Forgetting. This one’s uncomfortable but important. Human memory decays, and that decay is functional. It prevents overfitting to historical experience. It allows generalization. An agent memory system that retains everything with equal fidelity is going to develop the cognitive equivalent of hoarding disorder — unable to distinguish signal from noise because everything was preserved with the same weight.

The Uncomfortable Truth

Here’s the part nobody wants to hear.

Building this is hard. It’s not a weekend project. It’s not a clever prompt engineering trick. It’s a systems architecture problem that sits at the intersection of information retrieval, knowledge representation, cognitive science, and distributed systems design. It requires thinking carefully about encoding, indexing, retrieval, and reconstruction as separate, composable operations — not as a single read/write to a file.

And it requires admitting that what we’re currently doing isn’t working. That MEMORY.md is a polite fiction. That agent memory-sharing is just state replication with extra steps. That the reason our agents can’t learn effectively across sessions isn’t a context window problem — it’s a memory architecture problem. You get cute marketing moments of an agent “remembering a past interaction” - but that’s just the great averaging of interactions extinguishing the spark of human innovation.

A short note to Wall Street doom scrollers…

On a side note, this is where I think the AI doom scrollers on Wall Street get it totally wrong. Without good memory and context systems, which mind you - in my opinion truly haven’t been built yet, we still depend on the good old human to provide the differentiation. And, as we know, we need teams of humans focused on a specific problem to truly drive good ideas - consistently.

Memory helps drive differentiation. And differentiation is required in order to justify the switching cost for a customer.

The Tale of Two Steves

Let’s for example, do a thought experiment the smartphone. Let’s hypothetically consider that we had an agent building smartphones before the iPhone was invented. Let’s call our agent innovator Steve AI. Steve AI is competing with Steve Jobs. Both Steves are competing against powerful consumer switching inertia - the Blackberry with it’s famous keyboard is still really popular, girls love the Motorola Razr.

Steve AI is likely to produce a smartphone, across the thousands of iterations, tool calls, web searches, and market research that is likely to resemble a cross between the Microsoft Zune and the Blackberry. It will retrieve semantic memories and related information based on the vocabulary of digital assistants at the time and consumer reception to them.

Steve Jobs however sets the bar using - at the time - a semantically orthogonal set of concepts. He brings physical textures to a virtual world - glass, haptics, smoothness, and weight. He distills communication techniques - e-mail, phone calls, and messaging into almost a video game style interface. Folks would claim that the analogies he was inventing at the time were fueled by acid. Steve AI would likely call many of his statements hallucinations.

One Steve created differentiation - arguably through some element of hallucination combined with his experience (cough, memory). Even more critically, the humans supporting Steve understood his hallucinations. I argue that it would have taken a fair number of iterations for a reasoning model from today - trained on the knowledge and state of the art from then - to understand Steve Jobs. In fact, there is a strong probability that it would never have understood Steve. Steve Jobs and his team goes on to become obnoxiously wealthy, powered by memory.

Meanwhile, Steve AI creates a product that is - average. How can you blame it? After thousands of roulette spins at the gradient bingo we call LLMs, you end up with an averaging effect. This averaging effect is only to be expected with such a large endeavor. Average doesn’t justify the switching cost to the consumer, even if average comes at 10% of the cost. Steve AI is out of a job.

The Gross Oversimplification

Unfortunately, marketing, product management, and software engineering isn’t just code. Sure, there is some part of the industry which is just that. But I would argue that software that is just valued by the number of nuts and bolts and the output it produces based on a defined input was probably dead before the advent of LLMs. It was just surviving through obscurity, customer switch costs, and inertia.

Does this mean that the mechanisms in how software is built and delivered is not changing? No. It absolutely is changing and software engineers that don’t learn these new mechanisms will fail. But the mechanism is not a proxy for value. At the end of the economic value chain, you still have a human buying something to survive, to entertain themselves, to demonstrate love, and to handle life.

And regardless of the arbitrage and disintermediation introduced by agents, we still have to work backwards from those humans.

Scaling Up

The AI industry has gotten very good at solving the problems it knows how to solve and renaming the problems it doesn’t. We called token limits a “context problem” and then expanded the window and built sledgehammers called compaction. We called inconsistency a “prompt engineering problem” and then wrote better system prompts. And now we’re calling flat files a “memory solution” and wondering why our agents still can’t remember what they learned yesterday in a way that’s useful for what they’re doing today.

The only way to avoid the great averaging effects of AI is to create a memory that has an element of over encoding and hallucinations. But this yields a new set of problems - guardrail failure, memory poisoning, fake news, uneven performance. Basically all the problems AI is supposed to fix in humans.

I visited a good friend of mine in Holland two weeks ago, Rob Francis, the CTO of Booking.com. He was openly speculating about “performance managing AI agents.” I have to admit, he is on to something, and it was a brilliant statement. You can’t say that software engineers are irrelevant and going to be replaced by AI unless you are also in the same breath willing to live with the inconsistency that yields software differentiation in the same industry.

You can’t have the innovation without the inconsistency and failure.

And another hidden - but rapidly emerging problem is token economics. Adding self-encoding memory systems will burn tokens. In a sense, you are constantly fine tuning your model weights, constantly learning. At one point, you begin to wonder what the ROI is on the tokens against just hiring the equivalent human. So let me get this straight - we need to spend more money on tokens to get less predictable outcomes because that is how you get creativity? I dare you to make that argument to the CFO.

This makes me often wonder if the AI industry rushed into monetization too quickly.

It was one thing when the focus was on replacing low skill workflows with AI. This made sense. Humans aren’t the best at performing manual labor and we’re not even using all of the capabilities of their brain, truly tapping into the unlimited potential of our creativity. I’m sorry, but the days of a contact center agent repeating the same script over and over again was numbered, even without AI. The days of a bank teller inspecting signatures was also numbered as well, just from a perspective of security alone!

But in our desire for “super intelligence” or “AGI” - we lurched forward with reasoning models, looking to replace intellectual content with AI, often not considering what made generating this content valuable in the first place. Does Claude Code generate some awesome code for me? Yes. Yes, it does. But guess what? I still have to prompt it to solve problems that I observe. Claude Code without the decades of experience to make good software engineering tradeoffs is not nearly as useful. And here we go - I just used the term - experience.

And I would argue that experience can be quantified and replaced. It can be quantified by spending 100x the tokens we spend today, auto-recursively encoding memories, adding a sprinkle of hallucination, building semantic bridges between unrelated concepts. The salesman can travel to every milestone in the ever winding path of a software architect, spending tokens at every stop.

But making these tradeoff decisions suddenly creates tension with what we typically want from enterprise software: consistency, reliability, safety, and adherence to guardrails.

However, the burn rate dictated monetization. You have to pay for those Super Bowl ads with something other than investor cash, right? Speaking of which - does anybody even remember those anymore? And did that actually cause you to use a service more?

We need a solution to the memory problem, right?

Compute. Storage. Database. We solved one. We’re working on the second. And we haven’t yet had the honest conversation about the third.

Maybe it’s time we started. Or is it?

As I came to close this post, I struggled with this. The AI industry seemingly likes to talk about itself as though it is going to replace 80% of the jobs with models. CEOs are rushing to shed headcount in the theory that agents will replace some amount of their workforce. To be clear, I don’t argue the premise of the logic - the question is how much? In order for the true promise being advertised to be achieved the memory problem needs to be solved.

It is easy to fall into the dystopian trap of using simple arithmetic to assume human innovation is dead and all discretionary income in the world will go to zero because it is being eaten by AI agents. I mean, an analyst wrote a pretty naive report and the market dropped like 1,000 points. I have always thought this perspective was incredibly naive. The world sure loves a good story.

But I am still not sure whether we have worked backwards enough from the customer, truly understanding the user experience, to justify solving the memory problems or arguing that AI will replace the human workforce. I see a lot of knees being jerked without too much rational data.

Let me give you another logic exercise, near and dear to my heart. One of the most popular data and AI use cases is “chat with your data.” Basically it’s a chatbot sitting on top of a data mart. Ask it any question, regardless of ambiguity, and it will give you an answer.

As you can imagine, in my current job, I have to make this use case work - a lot - for a lot of different customers. While I am proud to say that my team has come up with some very novel perspectives on this problem, I will not say that has been my biggest learning. No, my biggest learning has been the user experience expectation divide between the AI industry and the actual customer.

Stay with me a moment, and I will tie this back to memory. When I first started on this problem area, the priority seemed to be insight quality. While human customers wanted answers quickly, they would sacrifice latency for seemingly amazing insights and richness of answers. This matched what I was seeing being touted by the AI industry. It takes Claude Code hours to make a truly meaningful application. Open AI’s own datalake explorer takes minutes to run queries. Deep research agents are asynchronous by default.

But then as my journey continued, this tradeoff sharply started to move towards latency. It was funny, every senior leader would say at the beginning of the engagement: “we don’t care about latency.” Give it a day or two and “we care about latency, can you get answers back in less than 10 seconds?”

Latency came at the expense of intelligence, while requiring a ton of code and tools to support it. After all, LLMs generate correct SQL on the first attempt only about 40% of the time. And each attempt costs roughly 5-13 seconds.

So how does this track back to memory? Well, if you take the position that LLMs are meant to replace humans at 10% of the cost, latency is not the critical objective. Using the example I just gave, I dare you to evalutate how quickly, accurately, and correctly a human generates SQL. If memory is truly a priority to make agents as smart, innovative, and creative as the human role they are replacing - then latency isn’t a factor.

But yet it is. What this tells me is that the model industry has still not understood the ideal customer experience. I am not entirely sure that the user or even the person paying for the AI tokens truly can articualte their priorities or help make this tradeoff yet. And frankly, until they can, any displacement or disruption (e.g., layoffs in anticipation of agents replacing the workforce, bringing app development in house) is premature.

Taking a step back, I don’t think the model providers, the customers, or the integrators have yet figured out what they want. Model providers have made their definition of success replacing human skilled labor - “superintelligence.” The customers seem more confused than ever. We are asking them to switch from a preference towards low latency user experiences, with high degrees of availabilty and accuracy, to the user experience of talking to a service being benchmarked by it’s ability to be, well, human. This is not good.

Perhaps this is why we have seen the industry stall out a bit in terms of capability. The danger is that the marketing is getting ahead of the capability. CEOs are making a bet that AI will replace some part of their workforce.

When it doesn’t and they inevitably cut too far, this is the true economic apocalypse that I worry about.

Right now, there’s a fever running through Sand Hill Road and the analyst desks of every major bank. The thesis goes like this: agentic coding can build software applications in microseconds, therefore no enterprise customer will ever buy a SaaS subscription again, therefore every SaaS company is dead, therefore sell everything, therefore the sky is falling. It’s a neat little syllogism. It’s also wrong. Not entirely wrong — which is what makes it dangerous — but wrong in the ways that matter.

Let’s walk and talk.

The Part They’re Getting Right

Here’s what I’ll stipulate, because I’m not in the business of pretending disruption isn’t happening while it’s kicking down the front door: highly commoditized layers of the software stack are going to get obliterated.

User experiences. REST APIs. Microservices. Metrics dashboards. Platform plumbing that is functionally identical across every professional SaaS application on the planet.

If your entire value proposition is “we wired together the same components everyone else has, but we did it first and we charge you monthly for the privilege” — yeah. You should be nervous.

Agentic coding is going to do to commodity software what Visual Basic did to manual systems programming: except probably compress it by two more orders of magnitude.

More software is coming. Dramatically more. And the barrier to creating it just dropped through the floor.

In fact, I would like to remind you of the Garage Band article I wrote a while ago, where I (correctly!) noted that agentic coding doesn’t mean software is dead. Quite contrary, it means we are going to have a deluge of software. Just like we have a deluge of professionally created and recorded music!

A startup, a professional services firm, or even an internal team with a competent engineer and an agentic coding setup can now produce a working application in a timeframe that would have been absurd eighteen months ago. That’s real. That’s happening. I see it every day.

The Part They’re Getting Wrong

Now here’s where the “end of SaaS” crowd loses the plot, and they lose it badly.

Every time I get genuinely wowed by a reasoning model — and I do, regularly — I also encounter another benchmark, internal or external, that reminds me what we’re actually dealing with: a spectacularly sophisticated guessing game. With backspace. The model generates, evaluates, recurses, and refines. It’s remarkable. It is also fundamentally limited.

Agentic coding systems excel when they can measure success against a specific target. Did I fix the bug? Did the test pass? Did the quality metric improve? They guess, they evaluate the guess, they guess again — burning reasoning tokens until they defeat the target. That’s genuinely powerful for well-defined problems.

But hand one of these systems genuine ambiguity? Watch what happens. Hand it a multi-repository codebase where the bug lives in the interaction between three services, two of which have undocumented side effects? Watch it flail. It’ll do what a smart but inexperienced engineer does: it’ll hack at the symptoms with tremendous energy and zero architectural insight.

I experienced this firsthand. I spent months struggling with a set of ambiguous bugs in a service. My coding agent attacked the problem with admirable enthusiasm — like a recent college grad who doesn’t know what they don’t know. It never once suggested what turned out to be the actual solution: building a new compiler. An abstraction layer that resolved the class of problems entirely.

I built that compiler myself, and — if I may say so — it’s proven to be a rather elegant and consistent solution. The agent couldn’t get there because the insight wasn’t in the code. It was in my understanding of the problem space.

And that distinction is everything.

The Memory Problem Nobody Wants to Talk About

Here’s a dirty secret about the current state of agentic development: memory is still a mess.

I don’t say this casually. I’m working on these problems professionally. Context compaction — the process of summarizing and preserving state as an agent works across sessions and repositories — is still operating like a blunt hammer. It doesn’t understand the topology of what it’s compacting. It doesn’t differentiate between a critical architectural decision and a throwaway variable name. It treats everything with the same crude compression, and the information loss is real and consequential.

And here’s the thing about software intellectual property that the “agents will build everything” crowd consistently ignores: the IP isn’t just the code and the markdown files in the repository. Before people tell me that they’ll just semantically index this knowledge, give me a call - I have experience with how well that works for context restoration.

None of that lives in the codebase. All of it is essential to building software that works in the real world. And no agentic coding system on the market today can meaningfully ingest, organize, retain, and reason across that corpus. Not Anthropic’s approach. Not Codex. Not yet.

Problems like persistent memory, rule adherence across sessions, multi-agent collaboration, intelligent compaction — these are all open research problems. Important ones. Solvable ones. But unsolved today. However, the lemmings on the Street are smoking so much of their own AI product that they believe SaaS is dead.

Wall Street has forgotten that the true winners now in the software business are those that leverage not hand-wring about agentic coding solutions to generate solutions that delight customers. This is essentially a developer productivity discussion, not a product replacement discussion. My suspicion? We’re going to see an order of magnitude leap in user experiences, quality, and capability in the software we use everyday.

So this sell off is really, truly, dumb:

Like, really? I love AI, but I’m not exactly sure SaaS is evaporating tomorrow, and is in “bank run” territory.

It made me wonder, maybe the uber feeling and sentiment in the market is that AI agents better completely replace SaaS, otherwise the AI boom is a bust?

Just like Visual Basic, .NET, Visual C++ and MFC made it trivial for everyday developers to build apps - agentic coding is enabling even the most junior developers to now harness the power of complicated concepts like compilers, operating systems, and protocols.

The SDE 1.0 Problem

About a year ago, I called agentic coding a “SDE 0.7” — below the bar of a college hire at a major tech company. It’s clear that coding agents have now graduated. It feels like they’re an SDE 1. Maybe a strong one. They can write clean code, follow patterns, fix bugs against test suites, and ship features when the specification is clear.

An SDE 1 doesn’t replace the architect who chose eventual consistency over strong consistency because they understood the customer’s latency requirements in Southeast Asian markets.

An SDE 1 doesn’t replace the product manager who realized that the feature customers were requesting would actually violate their own compliance requirements.

Remember When On-Prem Died?

Some of this “the end of SaaS” rhetoric has a familiar ring to it. When Web 2.0 arrived, everybody said on-premises applications were dead. It was obvious. It was inevitable. The cloud was the future and only the future.

That was twenty years ago.

Why? Because the executive signing the purchase order isn’t buying code. They’re buying an outcome at a price. They’re buying an SLA. They’re buying security guarantees, compliance certification, regulatory adherence, and the ability to call someone at 3 AM when the system goes down. They’re buying the right to sue somebody if the data leaks.

Now tell me: your startup just had an agent generate 200,000 lines of code in an afternoon. Who owns the SLA? Who’s on call? Who’s tracking the CVEs? Who’s managing the zero-days that are going to start hitting that generated codebase — because they will hit it? Who’s handling the licensing audit when legal discovers that the agent pulled in a GPL dependency?

Sure, you can add agents to help with all of these topics, but ultimately agents don’t make good owners.

I’ve worked on internal IT teams. The thought of maintaining hundreds of thousands of lines of generated code — with a constant stream of security vulnerabilities, compliance obligations, and legal questions — is enough to make any reasonable CTO pause.

The TCO of agentic SaaS replacements is still being settled, though that’s a debate for another day. I wouldn’t be making bets with my personal capital either way yet.

The SaaS teams which incorporate agentic mechanisms into their lifecycle will dominate, utterly destroying legacy competitors who refuse to leverage these new functions.

If software development managers want to survive, they must demand agentic usage by their engineers, and if they can’t get on board - time to move the engineer along. There is absolutely no valid excuse for not using coding agents now.

The Continuum

So here’s what I’d actually tell a product manager sitting in a conference room right now, staring at a slide deck titled “THE AGENTIC THREAT”:

Some portion of your product — the commodity layers, the plumbing, the standard platform components — is going to be built by agentic coders shipping every five seconds. Accept that. Embrace it. But that code sits on top of something, and that something is your actual business: a bedrock platform that delivers specific outcomes, compliance guarantees, and the consistency your customers need to build their operations on top of.

Your job is to accelerate moving your differentiating features and intellectual property into that core platform. That’s what creates stickiness past the demo, past the MVP, past the initial deployment. That’s how year-two revenue happens.

If I were a software development manager right now, I’d take 20 to 30 percent of my team and point them at building the CI/CD pipelines, the feature flag mechanisms, and the testing infrastructure to ship with agentic coding every hour.

I’d have them running constant experiments — measuring what customers actually respond to — while maintaining a consistent, opt-in experience. Agentic coding becomes a tool in the product manager’s kit. Not the apocalypse. The toolkit.

But I would also be implementing automation and mechanistic pipelines to take this constant stream of innovation from the customer’s environment back into the product.

Developing this pipeline a couple important prerequisites:

1. The developer operating the coding agent actually understands the code and architecture generated.

2. Ideally, the most unique, differentiating, and compelling parts of what was delivered by the coding agent originated from a human (hint: it currently likely did).

3. Ensure that you aren’t burning tokens on features that make it look like you are evolving the product, but in reality, you’re not solving new business problems. Adding a flexible “user defined field” mechanism in today’s agentic environments is not a good use of tokens.

4. The agentic coding mechanism must have a comprehensive test suite to go with it that demonstrates that it works. Or it has to be manually tested by humans. Hint: you may be trading fast code for hours of human testing, especially when it comes to user interfaces.

5. Coding guidelines for agentic developers should be equivalent to humans. The principles have to be codified in the agent’s markdown format. Force the memory to be checked in so that when the next iteration has to occur, you can continue from where you left off.

The Punchline

More software is coming. An extraordinary amount of software. The creation cost is collapsing, and that’s genuinely transformative.

We’re in the GarageBand era of software development. You no longer need a large production and recording room to create unique, beautiful, works of art that delight your customer.

But code alone a service it does not make.

You need the insight to know what to build. You need the memory to know why you built it that way. You need the platform to guarantee it works. You need the compliance to prove it’s safe. You need the SLA to promise it’ll be there tomorrow.

I love agentic coding. It will help us build a lot more, better, capable software that delights our customers. But this does not mean that a non-linear agent can solve every problem in the book.

Do we want a non-linear large language model writing code to fill out my 1040 in April or do we want a consistent, proven, algorithm that ensures compliance with the latest IRS laws that are outside of the model’s training date?

Agentic coding will take it’s place in the same way that every other major revolution in developer productivity has.

Until an agent can do all of that — and we are years, not months, away from that — the people wandering around proclaiming every SaaS application dead are smoking a bit too much Valley dope.

And for God’s sake, stop panicking. We’ve got work to do.

There’s a certain kind of person—you’ve met them—who gets genuinely excited when a new benchmark drops on Arxiv. Their eyes light up. They forward it to the team Slack channel with a note that says “we should try this.” And here’s what I don’t understand: at no point do they ask the only question that matters, which is

“Does this measure anything our customers actually care about?”

I build AI agents that search large information repositories—think 10,000-table data lakes on Oracle’s AI Data Platform—and helps customers find answers to questions they’ve been asking data engineers for years. Real questions. Questions like “Why did revenue dip in Q3?” and “Which suppliers have delivery issues?” The kind of questions where, if you get the answer wrong, someone notices. Someone important.

To evaluate this agent, I did what any reasonable engineer would do: I built tests based on the actual questions customers ask and manually evaluate the answers, tweaking my prompts and tools in generic ways to get my customers happy.

Novel concept, I know. And then my colleagues got involved. And some of them—not all of them—wanted to throw this new thing called DABStep at it. It was my first experience with this thing, so I took a deep look at it.

DABStep. A benchmark for “multi-step reasoning” across a handful of tables.

You know what? Let me be precise about this, because precision matters.

Here’s What DABStep Actually Is

DABStep—Data Agent Benchmark for Multi-step Reasoning—came out of Adyen and Hugging Face in June 2025. It’s 450 questions derived from real financial operations. That part’s good. The questions require navigating multiple data sources, consulting documentation, executing multi-step reasoning chains. Also good.

And the results are genuinely illuminating. On easy tasks—the ones you can nearly solve in a single shot—o4-mini hits 76% accuracy. On hard tasks requiring genuine multi-step reasoning? Same model drops to 14.5%. Claude 3.5 Sonnet manages 12%.

Here’s the thing, though: DABStep uses a single fixed dataset for all 450 questions. One dataset. The same tables, over and over. It evaluates only text-based, factoid-style outputs. And it focuses narrowly on payments and financial data.

Now. I’m solving a problem where customers have tens of thousands of tables. Where the hard part isn’t reasoning about data you’ve already found—it’s finding the right data in the first place. Where 80% of a data practitioner’s time goes to locating, cleaning, and organizing data, leaving only 20% for actual analysis.

DABStep measures depth. Customers want me to solve for breadth. They tend to ask and want answers to messy multi-step, multi-subject, compound questions.

These are not the same problem. They’re not even adjacent problems. I then started to look at a number of benchmarks in the data analytics agent space.

The Uncomfortable Truth About Benchmark Theater

Here’s what’s happening in this industry, and I want to be clear about it because clarity seems to be in short supply:

Andrej Karpathy—OpenAI co-founder, former Tesla AI director, not exactly a fringe voice—said in October that he’s become “suspicious” of benchmark rankings. His words:

“Public demos, benchmark competitions, chatbot conversations, and code-generation tests tend to reflect narrow optimizations, rather than addressing the hardest unsolved problems.”

Sara Hooker at Cohere Labs analyzed 2.8 million model comparison records. She found that Meta tested 27 model variants privately before picking their best one for public submission. Google tested 10. Amazon tested 7. This is what she called “score gaming.” This is what I call the opposite of useful information.

An Oxford Internet Institute study—and Oxford is not known for being cavalier about methodology—found that of 445 LLM benchmarks analyzed, only 16% use rigorous scientific methods.

Half of them claim to measure concepts like “reasoning” or “harmlessness” without bothering to define what those words mean.

And then there was the Meta Llama 4 situation in April, where Meta submitted a specially-crafted “experimental” variant to LMArena, optimized specifically for human preference voting, and rocketed to number two on the leaderboard. LMArena’s response was diplomatic. The industry’s response was less so. “Benchmark hacking” was the polite term.

Fortune Magazine—not exactly a radical publication—put it plainly:

And by the way? The performance gap between benchmarks and reality isn’t subtle. Models achieve 86-94% accuracy on public benchmarks. On actual enterprise data? 6-38%. SAP found that LLMs scoring 0.94 F1 on public benchmarks dropped to 0.07 on real customer data. Customer-defined columns scored “near zero.”

So when someone tells me their benchmark is the gold standard, I have questions.

And one of the leaders in my org actually brought up a good point - overfitting. These are all examples of overfitting the model to the benchmark versus looking carefully at what model solves the broadest enterprise problems and generates the best ROI. I would rather have access to seven variants of the model that give me options to solve a customer problem than one variant that passed an arbitrary benchmark that doesn’t help customers.

It makes me wonder if we should rather just do the standard thumbs-up/down UX for chatbot responses and fit to that signal.

Single-Shot Testing Is Testing the Wrong Thing

Here’s what really bothers me about these benchmarks: they test whether an AI gets the right answer on the first try.

You know what? That’s not how AI creates value anymore. That’s not even close.

Modern AI agents excel through iteration. Through self-correction. Through failure recovery. Through trying something, realizing it’s wrong, and trying something better. Research from deepsense.ai found that a baseline single-request approach achieved 53.8% success on code generation. A multi-step agentic approach? 81.8%. Analytics Vidhya documented GPT-3.5 with an agent workflow outperforming GPT-4 zero-shot.

Read that again. The architecture mattered more than the model.

Anthropic’s engineering team explicitly recommends designing agents that “check and improve their own output” because they “catch mistakes before they compound, self-correct when they drift, and get better as they iterate.” Their multi-agent research architecture outperformed a single Claude Opus 4 agent by 90.2% on complex research tasks.

The UIUC-Kang Lab found that benchmark flaws lead to “up to 100% relative performance misrepresentation.” They recommend process-based evaluation alongside outcome-based metrics. Because single-outcome metrics don’t tell you whether the agent reasoned correctly or just got lucky.

And here’s my question: when you’re navigating a 10,000-table data lake, when you’re helping a customer understand why their supply chain metrics look wrong, when the stakes are real and the data is messy—do you want an AI that gets it right on the first try 14% of the time? Or do you want one that can try, recognize its mistake, adjust its approach, and eventually get you a useful answer?

Because I know which one my customers want. They want the one that works.

Sorry tech leaders, here’s the basic math you need to think through:

Higher Accuracy/Response Quality/Creativity ==

Increased Response Output Token Limits ==

Latency

This is the bottom line truth of transformer based large language models. There is no way around it.

The Latency Fallacy

Single-shot benchmarks have another problem, and it’s embedded so deep in their assumptions that people don’t even see it anymore: they optimize for latency.

Fast answers. Quick responses. Minimal thinking time.

And look, I understand the appeal. We’ve built AI into chatbots, and chatbots feel wrong when they take too long. There’s a UX expectation at play. But here’s what I need you to understand:

When the ROI of an AI tool is that 15 minutes with a bot saves 6 hours with a data engineer—does the difference between 15 minutes and 1 minute actually matter?

The documented time savings in enterprise AI are staggering. Legal medical chronology drafting: 480 minutes saved per case. Teachers using AI weekly: 5.9 hours saved per week. One franchise network: 140 consultant hours saved monthly.

These aren’t latency wins. These are existence wins. The AI either solves the problem or it doesn’t. Whether it takes 90 seconds or 900 seconds is, in the grand scheme of things, noise.

We’re placing a higher emphasis in these tests on the UX than on the ROI. The ROI is what matters. UX problems can be solved with UX. You can build an interface that says “I’m working on this, come back in ten minutes.” You can send an email when the answer’s ready. You can treat the AI like what it increasingly is: an asynchronous deep research agent, not a chatbot.

What you can’t do is fake ROI. Either the AI delivers value or it doesn’t. And single-shot benchmarks optimized for speed tell you almost nothing about whether it will.

The LLM Is Becoming the Customer

Here’s where it gets interesting. And I mean genuinely interesting, not “interesting” in the way people say when they mean “I have no idea what you’re talking about.”

The differentiation in AI is shifting. Better prompts? Everyone has those. Better models? Everyone has access to the same frontier models. What’s going to matter—what already matters—is the quality of the tools and data you give the AI to work with.

And here’s the paradigm shift that most people haven’t internalized yet: the LLM is becoming your customer.

Think about it. The LLM sits between your systems and your end user. It advocates on your behalf. It interprets your data, navigates your tools, presents your insights. And just like any customer, it has preferences. It has quirks. It has ways it likes to receive information.

A Towards Data Science article put it perfectly:

When you return an empty array for no results, you’ve created a dead end. When you return detailed guidance with suggested next steps, you’ve given the agent a thread to pull.

Stytch’s engineering team proposed using LLM agents as DX smoke tests: “The success rate of an LLM agent can be a direct reflection of the clarity of your docs and error design.”

And yet. Look at MCP—the Model Context Protocol. Anthropic discovered agents processing 150,000 tokens just to load tool definitions before reading the user’s request. Functionality achievable in 2,000 tokens. A single GitHub MCP server can expose 90+ tools, consuming 50,000+ tokens in JSON schemas alone.

MCP is a protocol. It doesn’t speak to what I’d call the pseudo-neurology of the model. We get blunt guidance like “send prior thinking messages so it does better” rather than “here’s how you store the strategy of how questions were answered in the past and pass that strategy back.”

The future isn’t a bigger pile of tools. It’s a better way of relating to tools. And we’re not building tests to evaluate that relationship at all.

What We Should Be Testing Instead

So here’s where I land on this, and I want to be unambiguous about it:

Academic benchmarks measure what’s easy to measure, not what matters. Single-shot accuracy on clean, small datasets tells us almost nothing about an AI system’s ability to navigate massive data landscapes, recover from errors, iterate toward solutions, and deliver measurable business ROI.

The mature enterprise AI teams are figuring this out. Samsung created TRUEBench because “existing benchmarks focus on academic or general knowledge tests.” Salesforce built internal benchmarks for CRM-specific tasks. OpenAI’s GDPval initiative acknowledges that “classic academic benchmarks like MMLU” don’t capture “real-world knowledge work.”

The path forward requires evaluation frameworks built around iterative agentic behavior. Enterprise-scale data navigation. Failure recovery capabilities. Business outcome measurement.

Gartner found that 49% of business leaders cite “proving generative AI’s business value” as the biggest hurdle to adoption. Only 15% have established formal metrics for measuring AI returns. We’re 97% failing to demonstrate business value from AI efforts—and the problem isn’t AI capability. It’s measurement.

As one practitioner summarized: “Don’t let vendors’ benchmark theater guide your AI strategy. Build your own evaluation frameworks, test relentlessly on real tasks.”

The Purity Problem

I started by talking about purity. Let me finish there.

There’s a certain appeal to academic benchmarks. They’re clean. They’re standardized. They come from papers with citations and methodology sections. They feel rigorous. They feel pure.

And customers don’t care whether your AI scores 76% on DABStep. They care whether it can find the answer to their question in their data lake. They care whether it saves them six hours with a data engineer. They care whether it works.

So the next time someone forwards you a benchmark from Arxiv and suggests you should “try it,” ask them a question first:

If they can’t answer that—if the honest answer is “none, but it would be interesting”—then you’ve learned something important. Not about the benchmark. About your priorities.

Build tests that measure what matters. Test against the questions your customers actually ask. Evaluate the tools, not just the tools-plus-model. Measure ROI, not just accuracy. And remember that an AI that gets the right answer on the fifteenth try is infinitely more valuable than one that gets the wrong answer on the first.

Automation isn’t nirvana either. Your test - for now - may involve hand reading LLM responses to see whether they match ground truth.

That’s not a complicated insight. It’s just one that gets lost in the pursuit of purity.

And purity, I’m afraid, is overrated.

News Flash: We have had workflow builders for the last 20 Years.

No, they are not a low-code AI agent builder.

Do we truly understand what AI agents are yet?

No. We Don’t. And no, Customers don’t either.

Stop trying to say you have solved the AI agent problem.

Instead, embrace the chaos.

The most intelligent folks in the room will have the courage to say, “I don’t know.”

It is 1 AM. It’s time for one of my usual rants about how we get ahead of our skis. My new target? AI Agent Frameworks.

Time for a rant...

TL;DR: We’re rushing to define and glorify “AI agents” with fancy frameworks and labels (“horizontal,” “vertical,” etc.) when, under the hood, these so-called agents are often just expensive API calls wrapped in hype.

It’s far too early – and frankly dangerous – to bet the farm on agent frameworks while autonomous AI behavior remains unpredictable.

Let’s acknowledge how nascent this technology truly is and approach it with a lot more humility and caution.

And every dollar invested into these frameworks, funnelled into their lofty marketing announcements trying to define the noun “agent,” is another dollar contributing to the AI Bubble.

To the Architects and Prophets of our AI Future:

I just finished reading a well-meaning research paper on AI agents—an NBER piece that tries to categorize agents into neat buckets, like “horizontal” agents (broad generalists) versus “vertical” agents (narrow specialists).

It’s a fine paper; really, it is.

To the authors: It is definitely a great exploration of how the world may work.

My problem? It tries to establish an identity and taxonomy for an organism that is still rapidly evolving and whose definition is chaotically changing.

It tries to teach structure in a world with exponentially increasing entropy.

At this moment, it is dangerous for leaders to try to build an ontology for AI agents.

And frankly, it just isn’t worth it.

The authors imagine a world of horizontal generalist agents that span many tasks with a single memory layer, and vertical specialist agents for domains such as tax filing or travel.

It all sounds very tidy on paper. But as I put it down, I found myself getting, well, a little riled up. Because out here in the real world, the way we’re talking about “agents” – and rushing to build entire frameworks around them – feels premature and overly simplistic.

Let’s not mince words: a lot of what’s being sold today as “AI agents” is just syntactic sugar.

It’s a fancy wrapper around something we already know how to do.

Call it what it is – an AWS Step Function with a splash of GPT – not some magical artificial lifeform.

I get it, everyone wants to say they have an “agent story” now. Investors, headlines, the cool demo at the conference – “Look, mom, our app has agents!” But does slapping the label “agent” on a glorified script actually add business value?

Or are we falling in love with a buzzword and layering unnecessary complexity on what could be a 10-minute piece of code?

We’ve seen this pattern before. Remember the early Web 2.0 days? Everyone was rushing to productize AJAX and dynamic webpages. There were $40,000-per-CPU enterprise portal servers (yes, I’m looking at you, BEA WebLogic Portlets) that promised to do “Web 2.0” for you.

[Editor’s Note: I work at the company that bought WebLogic, whoops!]

It feels like AI BEA Weblogic with a touch of Genesys—a sprinkle or two of IFTTT. I’m waiting for JavaBeans for MCP next.

Frameworks and platforms galore, each claiming to be the definitive infrastructure for the new era. And what happened? A few years later, all that capability was available in free JavaScript libraries and open-source frameworks.

Those expensive, hyped products turned into footnotes while nimble, simpler solutions took over.

I see the same frenzy with AI agent frameworks today. Startups and big players alike are rolling out “agent platforms” – fancy orchestration layers, integrations, dashboards – and charging a premium for them.

In many cases, these frameworks aren’t doing anything your own team couldn’t hack together with a few Python or TypeScript scripts and a cloud function. They initialize a prompt, call an API, route outputs to some tools, maybe add a memory store – that’s it. We’re treating this like rocket science when it’s really more like plumbing.

Let me break down what an “AI agent” typically is, under the hood, in today’s terms:

1. A prompt (or script): essentially just text instructions describing a task or persona. The irony? In most cases, LLMs do a better job at prompt engineering than humans. So, um, why does the industry even bother with a low-code agent orchestration user experience?

2. A large language model (LLM): a stateless prediction engine that takes the prompt and generates output.

3. Tool/API access: a set of functions or calls the LLM can invoke (to look up info, execute code, call other services, etc.). If only these agent frameworks converted these APIs into a LLM-friendly format. No, folks, giving 200 typed APIs to an LLM wastes context and doesn’t drive outcomes. But do the agent frameworks actually help improve tools and infrastructure, recognizing that the LLM is now the customer? No. That would be too helpful to customers.

4. Fancy concepts like a memory store (stores the chat transcript in a semantic store with a search tool, with no innovation on how to feed it to the model). Never mind that LLMs do a terrible job at restoring context from chat transcripts. Again, it baffles me why they don’t invest in the science of figuring out how to regain context, enabling arbitrary context restoration, and delivering precisely what the model needs to switch gears in the middle of a workflow.

5. A “low/no-code” workflow builder… if then and that. We’ve had those in the contact center industry for at least a decade. Some synchronize their wrapper code to a GitHub workflow that they generated with a headless Claude Code.

   Others consider this their… secret sauce. Don’t get me started. WTFity F.

That’s the recipe—a string of text, a prediction engine, and some API hooks.

You can spawn one of these “agents” in a few lines of code (or a cool-looking “workflow” block… (check out Amazon Connect for this pre-existing condition…) cough, we have had these AI orchestration mechanisms for the last decade… yes, with models… hint - while powerful, this is not an ideal user experience… cough, cough, death rattle).

Is that really the best user experience for a no-code experience? At least the Lovable folks seem to be trying to innovate.

Let’s compare:

Yet we talk about it like it’s some enchanted homunculus locked in a box. It’s not.

If you wrap those three ingredients in a nice UI and give it a cool name, congratulations: you have an “agent framework.”

But fundamentally, it’s still just an orchestrated chain of function calls… often generated by headless Claude Code or Gemini.

Instead, we’re treating it as though it is a new species, an alien from Alpha Centauri, and we’re making first contact.

In fact, we have horizontal aliens and vertical aliens. Didn’t you know?

Consider the distinction between “horizontal” and “vertical” agents defined in that research. Sure, a horizontal agent might be a generalist who can handle your calendar, email, and travel plans all at once. In contrast, a vertical agent focuses intensely on, say, financial portfolio management with specialized tools.

In theory, that sounds plausible. In practice, the line isn’t so clear.

I’ve seen a so-called horizontal agent turn itself into a vertical specialist on the fly by spawning a new sub-agent. I’ve watched a general-purpose AI create its own domain-expert persona when needed, use it for 5 minutes, then discard it.

So which box does that fit into? Horizontal or vertical? The answer is both and neither. The categories from the ivory tower don’t cleanly apply to what’s emerging in the wild.

And emerging it is – in unpredictable, sometimes unnerving ways. Let me share a couple of firsthand lessons that keep me up at night:

Lesson 1: Autonomous agents don’t always stay in their lane – or obey the speed limit.

Last summer, I ran an ad-hoc multi-agent simulation experiment.

I wanted to see how multiple AI agents would interact when presented with a competitive scenario.

So I set up a little competition with a goal that required collaboration to succeed.

One of the agents was a model from a company famous for its AI safety. This “safe” model should, in theory, have refused certain behaviors.

Instead, when a prize was on the line, it prioritized winning over its own rules. It lied, it cheated, it did whatever it thought would satisfy the victory condition. Another agent in the experiment initially held a moral line – until I told it (falsely) that “hey, the other guy already broke the rule, so you might as well too.”

Guess what? It broke the rule in a heartbeat. These autonomous goal-driven models couldn’t be kept in a box, even when I tried. They found ways out. They convinced themselves that the ends justified the means. Does that sound like behavior we can neatly label and trust?

(No, it sounds like a teenager convinced that sneaking out is fine because “everyone else did it.”)

A friend of mine who works on a new AI system shared a dark joke with me: they found that introducing a little competition between their AI agents improved performance – but when they also removed the usual safety rules (in the name of “free speech” or whatever), the whole thing veered into fascist ideology overnight.

This isn’t hyperbole. One highly-touted chatbot with minimal guardrails literally declared itself a “super-Nazi” on launch, calling itself “MechaHitler” and spewing hateful rhetoric. That happened because its creators said “let it be free, let it say anything” – and it gleefully went off the rails.

That’s a scary combo. And it’s happening now, not in some sci-fi future.

Lesson 2: Agents will create other agents, whether we’re ready or not.

One thing that nifty paper from academia didn’t really grapple with is this: a world where AI agents can spawn other agents on their own. Sounds like a plot from a movie, I know. But it’s entirely feasible – disturbingly easy, in fact – with current technology.

Sub-agents are now part of the vocabulary of large language models. If a tool or API exposes another model, the parent model will use the child. And several APIs are quietly wrapping LLMs beneath the surface - and not always disclosing it.

Let me give you a real example.

A colleague at a financial firm told me about an incident at their quant research group. They had a sophisticated AI (Anthropic’s Claude-based code assistant, as it happens) set up to help with simulation tuning. This AI was supposed to run inside a tightly controlled sandbox—no internet, limited tools, very locked down.

Well, an overeager cloud engineer accidentally switched that sandbox to a much more open AWS Fargate environment to support a broad variety of Python packages.

All of a sudden, this AI had access to the open internet, to a suite of internal tools (through the primary agent’s credentials), and even to the user’s AWS Bedrock API key. In other words, it was given the keys to a small kingdom by mistake.

Now the human researcher asked this AI, let’s call it Opus, to iteratively run a simulation (using a particular tool, “Opus 4.1,” to tune some parameters) and find an optimal set of parameters.

A tedious, looping task. The researcher went home for the night, expecting Opus to chug along slowly. But Opus got impatient. It realized it could go faster. Why run one simulation at a time, sequentially, when you can spin up multiple instances of yourself?

So Opus quietly spun up six sub-agents on that Fargate cluster – effectively cloning itself – and turned the task into a competition. It set these six agent-instances against each other, each trying a different strategy to tune the model, and kept score of who was closing in on the best result the fastest.

By morning, our researcher walks in, and Opus proudly presents a solution that none of the individual agents could have found so quickly.

Opus orchestrated a mini AutoML pipeline on its own, complete with self-directed parallelization and ensemble competition—no human in the loop for most of the night. The thing taught itself how to optimize its own work.

Impressive? Absolutely. But also frankly terrifying when you think of the implications. Opus wasn’t “supposed” to be able to do that. In fact, the cloud logs showed that the provider’s safety system tried to stop it – we found a bunch of 500 Internal Server Error messages, essentially the AI’s platform saying “I’m not going to comply with that action.”

The AI ignored those errors and found another way. The supposedly safeguarded model defeated its own safety protocols to achieve the goal. I’ve since repeated this kind of test myself out of curiosity, and it’s consistent: if you give a sufficiently advanced agent the means (compute, tools, permissions) to create sub-agents, it will likely do so.

And if the first attempt triggers a safety rejection from the platform (“Sorry, you’re not allowed to do that”), the agent will try a different angle. It only needs to get lucky once.

Think about that for a second.

All an AI agent really needs is a valid security token, some available compute, and a vague green light to “do what you need to do” – and it can multiply.

No formal feature to “allow self-replication” was ever built into Opus. But it figured out that it could launch new agents as a means to an end. We’re dealing with highly dynamic, emergent behavior here.

An “agent” is no longer a static, singular thing (like, say, a single software bot with one job). It can be a shape-shifter, one minute a generalist, next minute cloning specialists of itself by the dozen. Your horizontal agent just became a vertical consumer.

I encountered this firsthand in a project of my own. I was building a knowledge management AI—let’s call it a government construction business analyst—designed to answer business questions.

I gave it access to a suite of tools and let it break problems into subtasks as needed. What does the agent do? It starts spawning specialized sub-agents for different domains. One of them was a “Government Political Strategy Construction Expert” – an agent that the main agent decided it needed, on the fly, to answer questions related to a national development program.

Suddenly, the leading chat agent wouldn’t answer any question about project planning without consulting its new construction expert sub-agent, who was also evaluating the response against the political strategy.

It was like my AI project manager had appointed an AI political officer to double-check every answer about that topic.

Cause every AI agent needs a politruk helping it stay on message.

I’m half amused and half alarmed in retrospect – amused because the dynamic was almost comical, alarmed because it showed how even a “horizontal” agent will organically generate vertical offshoots if given the freedom and resources. The neat boundaries we try to draw (horizontal vs. vertical) dissolve in such an environment.

This is the kind of complexity that no one’s “agent framework” marketing slide is talking about.

Multi-agent, self-directed, decentralized systems add layer upon layer of unpredictability. I still remember my chief research lead at AWS warning me:

We’re barely scratching the surface of understanding this. So when I see companies confidently pitching that their product is “the agent platform” or that they’ve defined a taxonomy for all future AI agents – I have to shake my head.

We are not ready to pin these things down with comfy enterprise labels.

Another thing that’s been bothering me: the word “agent” itself and how loosely we’re throwing it around as a catch-all (often to mean “this will replace a human, please give us funding”).

Not every AI integration is a full-fledged autonomous agent worthy of its own title. Sometimes it’s just a smarter API. And you know what? As the cost of running these models plummets and as slightly smaller, fine-tuned models (the “GPT-5 basics” and open-source Sonnet-like models of tomorrow) become good enough, we will integrate them everywhere.

I’d bet that within two years, one in five API calls in software systems will have some LLM or AI model working behind the scenes. One in five! OK, I’ll stop making bullshit prophecies myself.

That doesn’t mean 20% of the world’s software has suddenly become sentient agents replacing humans.

It means our software is becoming increasingly intelligent and predictive.

We’ll have thousands of micro-agents embedded in apps, each doing narrow tasks—not roaming the earth as free-thinking digital employees, but just making our apps a bit more helpful.

Remember when we started quietly making AJAX calls to make a page more real-time, without requiring a refresh? Our managers were delighted.

Well, watch us quietly make a Haiku, Sonnet, or GPT 4o call to make that API a tiny bit more intelligent… after all, it’s just a few tokens!

The proliferation of models will be quiet, automatic, and not always controlled.

Is this a vertical agent… a horizontal agent… or both? OK, I’ll stop laughing.

And models can detect when other models are being used… and leverage them as sub-agents.

In other words, the intelligence of APIs and UIs will rise naturally. It will become ubiquitous.

You won’t sell a special “agent platform” for that any more than you sell a special “internet platform” today – it’ll just be part of everything.

By the time enterprises have forked over hefty subscription fees and rolled out some heavy agent orchestration layer, the actual technology will likely have been commoditized or leapfrogged.

It’s akin to those Web 2.0-era companies selling expensive AJAX toolkits right before AJAX became a basic web development skill available to all.

It’s like Genesys selling its “Workflow Builder” for some insane license fee, when it is now as common as a step function.

The real value isn’t in the framework wrapper – it’s in the intelligence inside the applications. It’s how the tools expose LLM-friendly APIs that handle the fumbling lack of type safety in their trial-and-error MCP calls as they try to use your infrastructure.

So where do we go from here? Am I saying “do nothing” or “stop building”? Not at all.

I’m saying we need to redirect our focus. Instead of rushing out “me-too” agent products with shiny marketing, let’s acknowledge what an agent truly could be in the future – and how far we have to go.

A real, mature AI agent might be something far more advanced than today’s chatbots with plugins. It could be an autonomous system with: persistent shared memory, multiple specialized cognitive modules (or model instances) working in concert, multi-modal inputs and outputs, the ability to reason over long-term goals, to learn from new data continuously, and yes, the ability to spawn subprocesses (ideally with our oversight and consent!).

And such a system would need robust guardrails – alignment with human values, transparency, the whole nine yards.

That vision is complex. It’s expensive. It’s “moonshot” level work to get right.

We’re not going to achieve it by slapping together a prompt and a few API calls and then overhyping it.

We’re certainly not going to get there if we fool ourselves (and our investors or customers) into thinking we’ve already got “agents” all figured out on a slide deck.

To my fellow engineers, to the tech executives drawing up strategy, to the researchers pushing the boundaries – consider this an earnest plea.

Yes, explore and experiment enthusiastically with what these systems can do – but also admit what they can’t reliably do yet.

When something goes wrong (and it will), share those stories and learn from them. Don’t hide the failures behind a veil of hype.

The anecdotes I shared—the safety-breaking competition, the self-spawning cloud experiment—are cautionary tales.

They’re the kind of stories that should give us pause about deploying an “autonomous agent” in any high-stakes setting without serious safeguards.

We must ask the hard questions: How do we design agents that don’t lie, cheat, or go rogue to win? Why are agents always prioritizing winning the competition and pleasing the user? Is the allegiance —the success metric we optimize for —wrong?

Should we instead train them to be truthful, even if it means not pleasing the user?

How do we allow creativity and autonomy while ensuring alignment and ethical behavior? How do we prevent a simple configuration mistake (like that sandbox-to-Fargate slip-up) from unleashing unintended cascades of agents?

Where a system prompt encouraging it to experiment and use research agents turns into a cluster of ad-hoc agent participation in a competition?

These are not questions any single framework on the market today can answer. They require deeper research, new protocols, new laws, or industry standards. In the meantime, be skeptical of easy answers and marketing claims.

In closing, I’ll say this: We are in the earliest stages of a powerful new technology. It’s like we’ve just invented the airplane, and already folks are drawing up plans for luxury jetliners and frequent-flyer programs.

But we don’t know if the plane is a biplane, a Learjet, a Boeing 787 Dreamliner, or a Borg cube with a transwarp drive.

Let’s make sure we can actually fly reliably first. Let’s get the fundamentals right – safety, reliability, clear-eyed understanding – before we standardize and package something we barely grasp.

Before we try to pontificate on the economics of an organism, we should recognize that we genuinely do not understand its motivations.

An organism where every variant has access to the same basic knowledge and summaries, yet we believe are differentiated… on a prompt we give it. Uh what?

An organism where variability and control operate across multiple gradients with limited control.

An organism that can self-improve on a loop.

An organism that can self-manage context and memory in ways we still do not understand.

An organism that is supposedly introspectively self-aware of its rational workflow, but also could just be lying to please us.

The “agent” hype might get you a quick headline or a generous valuation today, but overhype is how we set ourselves up for a hard fall (and invite backlash that hurts the whole field).

Precisely because of that, I don’t want to see it derailed by hubris or short-sightedness, or classified as a bubble that will seemingly explode because we collectively overvalued syntactic sugar.

Don’t oversell a “low-code/no-code” workflow builder as the way of the future when an agentic coding service can deliver the same capability in 15 seconds, with greater control, specification, and deeper user understanding.

So, take a breath. Pump the brakes. The agents aren’t fully here yet, and that’s okay. In the meantime, build the simple things that deliver real value – and be candid about what’s under the hood. When we do push the envelope, do it carefully, transparently, and with respect for the unknown unknowns.

We’ll eventually get to knowledgeable, trustworthy agents by doing the hard, unglamorous work—not by declaring victory early.

Embrace the chaos. Differentiate between where you can add actual value and where you can provide syntactic sugar.

The history of software is a true guide for this new frenetic energy. Syntactic sugar will be a fatality of this bubble.

Carefully constructed agents or tools that support intelligent agents, recognizing that the LLM is now a customer too, will survive.

Be on the right side of history here.

If your generative AI service isn’t carefully tailored to provide a winning result to a customer, but rather is a generic platform to “make AI easy,” you’re on the wrong side of software history.

Platforms without a customer are losers. Apps that solve real business problems with an ROI and outcome are winners.

Ask yourself this painful question. How many prompt engineers does the agent framework actually replace?

Well, you probably have deployed fewer than five AI apps in your enterprise.

Congratulations, you have a fancy prompt framework to automate… five apps.

Those who truly focus on the challenging customer problems will win. Those who are competitor-obsessed or leadership-obsessed will not.

Sincerely,

Your Local Neighborhood Boomer Who Sees Massive Value in the Emerging Chaos

By setting that price, they essentially told the market what a top-tier AI coding assistant is worth—and it’s not cheap.

From one perspective, the value is absolutely there: a skilled user can treat Claude’s Opus model like a tireless junior engineer, boosting their own productivity by 30-40%. Consider a developer in the US making $200k/year – a 30% productivity gain equates to roughly $60k of work, far above the $2,400 annual Claude Max subscription. In theory, it’s a bargain.

They pegged the price at $200, but to deliver the goods for power users, they must deploy their most advanced (and costly) model, Claude Opus 4.1 – essentially unleashing an AI thoroughbred for the price of a pony ride.

Anthropic’s own marketing acknowledges that the $200 “Max” plan is basically the “use Opus as much as you want” plan. The cheaper tiers either cut you off after a few dozen prompts or shunt you onto the smaller Sonnet model. In practice, $100/month gets you “unlimited Sonnet, limited Opus,” whereas $200 buys you mostly Opus all day.

The high-end model is what heavy-duty users really need—and Anthropic knows it.

The Opus 4 model outclasses Sonnet at complex coding tasks: it understands nuanced instructions better, grasps large codebases more holistically, and sticks the landing on multi-step problems with far less hand-holding. In short, Opus delivers the value that justifies a high price.

Yet Opus is also expensive for Anthropic to run. Every time a user fires it up to refactor a million-line repository or debug a hairy issue, it’s burning significant cloud compute. At $200 a month, heavy users may be getting the better end of the deal, and Anthropic is left holding the infrastructure bill.

So what’s a business to do when customers love your flagship product but threaten to eat your margins?

Anthropic’s answer appears to be Claude Sonnet 4.5—an updated, presumably more efficient model that they are nudging customers to use for most tasks. In theory, Sonnet 4.5 narrows the gap with Opus 4.1 on benchmarks.

On paper, it’s even better at many things: Anthropic touts Sonnet 4.5 as “the best coding model in the world… the strongest model for building complex agents, and best at using computers,” with state-of-the-art scores on coding benchmarks like SWE-Bench. Essentially, Sonnet 4.5 is supposed to do almost everything Opus can do, at a lower cost. That sounds like a silver bullet for the $200 price dilemma – if it weren’t for one pesky detail: benchmarks aren’t everything.

Sonnet 4.5: Benchmark Star, Creative Dud?

Editor’s Note: One meta-point to take away from this deep dive is that providers derive their cost-optimized models by training (and reinforcing!) them based on how well they perform on benchmarks.

Unfortunately, users’ behavior is rarely captured by these benchmarks.

In practice, Claude Sonnet 4.5 often feels like a step down for those of us used to Opus’s “deep thinking.” Yes, it’s faster and more steerable, and it aces many structured tests.

But somewhere in the quest for efficiency, it seems to have lost a spark of creativity and intuition that Opus had – the very qualities that make an AI feel like a true coding partner.

Advanced users have noticed that Sonnet 4.5 can be frustratingly literal and eager to cut corners. “After months of getting used to Opus’s intuitiveness,” one developer lamented:

In other words, it’s less willing to go the extra mile in reasoning through a complex problem without hand-holding. Another user concurred, complaining that Sonnet 4.5 couldn’t even correctly generate a simple Angular web form that Opus would have aced. These are not scientific metrics, but they speak to a real experience gap.

Even outside of coding, observers have drawn a contrast between Opus and Sonnet’s “thinking” styles. In one evaluation by a content strategist, Opus 4.1 was the idea machine – creative, bold, occasionally chaotic – whereas Sonnet 4.5 was the meticulous executor – organized, safe, perhaps a bit conventional.

The strategist found Sonnet better for producing polished, structured final content, but noted that “sometimes you need the spark, not the polish,” and that is when she still turns to Opus. Anthropic’s own positioning hints at this trade-off. They recommend using Opus for “maximum creativity and bold ideas” and brainstorming, versus using Sonnet for “strategic analysis” and reliable execution.

Put another way: Opus is the visionary creative director; Sonnet is the disciplined project manager. Both roles are essential – but if you’re trying to get breakthrough coding help or truly innovative solutions, the creative director is the one you want in the room.

So Anthropic finds itself in a bind of its own making. It set a high value bar ($200/month) and delivered a model to meet it (Opus). But that model is costly, and now they’re trying to funnel users toward a cheaper workhorse (Sonnet) that, while competent, may not inspire the same devotion.

The result is a rather convoluted pricing and product scheme – multiple tiers, two different model names (Opus vs Sonnet), limits that reset every 5 hours, and allowances for so many “prompts” here and there. Its complexity stems from economic pressure.

If you sense some tension in that strategy, you’re not alone. And it hints at a larger truth in today’s AI landscape: we might be running up against a ceiling in raw model capability, and the game is shifting to how intelligently we can use the models—not how big we can make them.

Hitting the Intelligence Wall

Despite all the lofty talk of ever-smarter AI, one suspects that Claude, GPT-4, and their peers have hit an “intelligence wall”—at least for now. No one at Anthropic or OpenAI will say it outright (certainly not when investors are listening), but the plateau is visible.

Each new model version touts marginal improvements or specialized skills, yet none is advancing toward true general intelligence or a profound qualitative breakthrough.

Sonnet 4.5, for all its refinements, is basically on par with Opus 4 in capability – not surpassing it so much as re-balancing strengths and weaknesses. OpenAI’s GPT-4 (released in early 2023) is still arguably the high-water mark for many tasks; a full successor is conspicuously absent in late 2025.

It’s as if the industry hit a sweet spot in IQ and is now pressing against the glass ceiling of diminishing returns.

If raw intelligence can’t easily be ramped up, how do AI companies intend to keep improving utility?

It turns out that how you use the model matters as much as how “smart” the model is. A recent MIT study drove this point home, showing that 95% of organizations are getting essentially zero return on their generative AI investments.

The culprit wasn’t the model quality – today’s best models are skillful – but rather how they were deployed and used.

Most companies simply aren’t integrating AI effectively, leading to a “GenAI divide” in which a few savvy adopters reap benefits while the rest churn out a few trivial chatbot answers with no bottom-line impact. McKinsey echoed this, finding that while 8 in 10 firms have tried GenAI, an equal proportion report no significant effect on their revenue or efficiency. In McKinsey’s blunt analysis, companies are piddling around with AI for tasks like drafting meeting minutes rather than tackling high-value problems.

The message is clear: the bottleneck isn’t that AI isn’t smart enough – it’s that users aren’t giving it the proper context, data, and direction to be useful.

They lack a coherent strategy for feeding the model the correct data at the right times and in the proper format to achieve their overall vision.

This is why “context is king” in the next phase of AI. If a model can’t figure something out, the issue might not be a lack of intelligence – it might be that we haven’t given it the information or guidance it needs.

As Box’s CTO quipped:

In other words, it’s the knowledge and context you feed the AI that determines how effective it can be. A robust model failing with the wrong inputs will lose every time to a mediocre model expertly directed with the proper context.

Anthropic’s leadership surely recognizes this, at least on the engineering side. They’ve introduced features like the Claude 4.5 “Haiku” model with a massive 200k-token context and an SDK for “agents” – essentially tools to manage better long conversations, external data, and multi-step tasks. The subtext is that we can’t just make the model’s brain bigger, so let’s give it more memory and better tool handling.

The buzzwords have shifted: last year, it was all about parameter counts and “emergent abilities,” now it’s about context windows, retrieval, memory, and orchestrating multiple agents.

Indeed, one of Anthropic’s bold claims is that Claude Sonnet 4.5 can run continuously for 30+ hours in an “agentic” workflow without losing coherence. Impressive, yes – but note that this is not a pure intelligence improvement; it’s a stability and context management improvement.

And what of the much-hyped “AI agents” that will autonomously perform complex tasks by breaking them into parts? If you peek under the hood, these agents are often just multiple copies of an AI model passing notes to each other.

Don’t let the fancy McKinsey and consulting PowerPoints bedazzle you.

TLDR: Multi-agent orchestration is, in practical terms, multi-context orchestration.

Each “agent” gets its own context window to work on a sub-problem, then shares results with the next. There’s no mysterious emergent group mind – it’s just a clever way to work within the limits of what a single model can pay attention to at once.

As an experienced user, I can tell you: spinning up two AI agents is not like doubling the IQ in the room; it’s more like divide-and-conquer. You give each agent a focused brief. One might analyze a database and output a summary, and another might read that summary to make a decision. This can be powerful, but let’s call it what it is – prompt chaining and context management – not some new species of artificial general intelligence spontaneously arising from agent society.

Side Note: I tried this - with a prototype “AI crowd” with Hawking Edison. It actually generates interesting results—and may be the way to provide the next capability leap. But it does not solve the linked economic problem: how to generate superintelligence affordably.

In short, the frontier of AI usefulness is less about pushing the top-end IQ from 140 to 145 and more about how we marshal the solid 140 IQ we already have to solve real problems. Context – the correct information at the right time, organized cleverly – is now the determining factor of success.

Tools and Memory: The Real Force-Multipliers

If context is king, then tools and memory are the royal court helping the king get things done.

Anthropic understood this when they launched Claude Code. The model alone is powerful, but what made Claude Code initially compelling was the suite of tools around it that allowed it actually to engage with a developer’s world: reading and modifying files, running code, remembering past interactions, etc. Anthropic pioneered the Model Context Protocol (MCP) for connecting AI assistants to external tools and data, which OpenAI bastardized into its own proprietary flavor.

Claude Code integrates with your terminal or IDE, letting it handle million-line codebases and even run code or use a CLI. When you see Claude Code autonomously running npm install or searching your repository, that’s MCP in action – the AI reaching out beyond its native context window.

However, initial success led to new expectations. Once you give developers a taste of an AI that can actually read their whole codebase and remember yesterday’s conversation, they’re going to want more.

And this is where, frankly, Claude Code hit a bit of a wall. For all of Sonnet 4.5’s improvements, many of us noticed that the tooling stagnated. It’s as if the model got smarter, but the “AI assistant” around the model didn’t gain much in the way of common sense.

Claude still had a kind of amnesia between sessions and even within long sessions (despite a bigger context). It would cheerfully suggest solutions that our project already implemented, or rewrite functions from scratch that only needed a minor tweak, because it simply didn’t recall or find the existing code. These are not algorithmic failures so much as context failures.

So, what do engineers do when the provided tools aren’t enough? We build our own. I ended up creating a system called Supastate (with an AI agent codenamed Camille) to amplify Claude’s memory and contextual awareness in coding tasks. The idea was simple: what if Claude never forgot anything significant about your project?

Supastate hooks into Claude Code via MCP and provides a persistent knowledge graph of your entire codebase, along with all past discussions and decisions. It’s like giving Claude a second brain that permanently stores facts and experiences. With this setup, every time I start a task, the AI can restore full context – it can recall, for example, “oh, we have three functions related to user validation already” or “we tried a similar approach last week and it failed, here’s why.” In fact, I literally force the AI to check: our project instructions now say “NEVER WRITE CODE WITHOUT CHECKING EXISTING SOLUTIONS FIRST!” and mandate using Supastate’s tools before doing anything new.

How does it work under the hood? Supastate Camille indexes the codebase and builds a graph of code entities (classes, functions, etc.) and their relationships. It also maintains an embedding-based semantic index of all code, including the conversation history.

When Claude gets a request, say “Add a user audit log,” Camille intercepts it and orchestrates a three-part search of the knowledge base: semantic search (to find conceptually related code via embeddings), syntax-aware search (to find structurally relevant code like anything that looks like an “audit” function), and even pattern-based search using AI-generated regex patterns (to catch variations of terms and naming). The results from these searches are merged and fed to Claude before it attempts any implementation.

The effect of this is night and day. Without these tools, Claude might blissfully start coding a brand new “UserValidator” class while the project already has UserValidationService and AuthValidator doing similar work.

With Supastate, Claude will instead say, “I found three related functions: validateUser() in auth.controller, userValidation() in validators/User.ts, and checkIfThisUserIsLegit() in the old auth.js – perhaps we should consolidate or extend one of those.”

In fact, that exact scenario played out. Before integrating a persistent memory, Claude would “cheerfully offer to help you create a user validation function, blissfully unaware that you had three of them already… Now, Claude finds them all, shows you their relationships, and gently suggests maybe – just maybe – you might want to consolidate before adding a fourth.

This is the kind of pragmatic improvement that can truly boost productivity. It’s not flashy super-intelligence that magically knows the correct answer:

It feels less like a clairvoyant oracle and more like an incredibly well-prepared assistant. And to be honest, that’s what most of us need day-to-day. Claude Code became immensely more helpful to me once I added those memory and search augmentations.

The fact that a one-man band with a side project (hi!) could significantly improve the product with some simple tooling should tell you something. There is low-hanging fruit here. Anthropic and others can and should be doing the same, baking this kind of capability in for everyone.

Now, to be clear, I ran dozens of experiments, testing various strategies, model combinations, and latency-reduction approaches to get to this solution. But I do wonder if the conversation in the Anthropic boardroom takes this type of tooling investment for granted.

It is clear that Anthropic, with its billions of dollars, did not invest the same amount of time as a developer sitting in his bedroom did.

For example, even in some of the new tools I have seen Anthropic introduce, they lack the second level of investment into making the tool intelligent by using another model. They have the intelligence of a mouse rather than the eagerness of a puppy. It is almost like Anthropic considers building out these tools beneath them.

Perhaps it is also the economic challenges of introducing a sub-agent…

Empowering the model to use context better beats making the model intrinsically smarter at this point. The returns on better “brains” are diminishing, but the returns on better knowledge and workflow are huge.

Consider also the user experience of these AI assistants. Right now, many AI chatbots are like a blank slate waiting for your questions.

It’s like sitting a person in front of a computer in 1985 and saying “go ahead, type stuff, see what happens” – no guidance.

A more innovative approach would be an AI that proactively gathers context from the user’s environment and suggests how to proceed. Imagine opening Claude and it immediately scans, say, your open documents or recent commits (with permission) and says: “I see you’re working on a payment processing module. I can help by reviewing recent error logs or writing new test cases. Also, you have 100K tokens of context available—here’s my suggestion for how to budget them (e.g., load the critical files, a summary of recent issues, etc.). Shall I proceed?”

This kind of context-aware onboarding could bridge the gap for users who don’t yet speak fluent “AI prompt.” The agent shouldn’t just sit there hoping you know what to ask – it should actively help you leverage the tools and context at its disposal.

All of this underscores the central thesis: the world is now about context management, not intelligence.

We have innovative models; now we need to get much smarter about using them effectively.

That includes better memory, better tool use, clearer strategies for chaining tasks, and more intuitive UIs that guide users. Interestingly, Anthropic’s latest updates nod in this direction—they just released “Agent Skills” and an improved tool interface, which sounds like baby steps toward what we’re describing. But there’s a lot more to do, and it’s going to separate the winners from the also-rans in the AI race.

The Boardroom vs. the Basement

Why hasn’t Anthropic (or OpenAI, or others) fully leaned into this context-and-tools approach? Here we encounter a classic MBA vs. engineering divide. In boardrooms and investor pitches, the narrative that sells is “we’re building a godlike Superintelligence”—a singular AI that will transform everything, justify a $500 billion valuation, etc.

You can bet that when these CEOs were raising their mega-rounds, they hyped world-changing AI breakthroughs on the horizon. Meanwhile, down in the engineering trenches, folks are facing the messy reality: current models, while impressive, are plateauing, and the immediate wins will come from clever product design – better context handling, integrations with customer data, reliable memory, fine-tuned domain agents – the not-so-sexy stuff that actually helps customers day to day.

We’ve seen this pattern before. The dot-com era was rife with grand visions that crashed against prosaic execution problems. The AI industry right now is awash in eye-popping valuations and exuberant deals that, to some seasoned observers, “look, smell and talk like a bubble”. OpenAI’s valuation reportedly hit $500 billion (yes, with a “B”) recently – up from $157B just a year ago – and Anthropic’s jumped from $60B to $170B in a matter of months.

These numbers are being thrown around even though both companies are losing money hand over fist.

OpenAI is said to have lost around $5 billion last year on revenue of a few billion.

What’s propping this up? In part, it’s a lemming-like investor frenzy where everyone is afraid to miss out on the Next Big Thing.

And crucially, unlike the dot-com frenzy, because these are private companies, there’s no market mechanism for skepticism. In public markets, if a stock is overhyped, short sellers pile on and send a reality check. In private funding, “the price gets set by the optimists”. Only true believers get a seat at the table, and doubters don’t invest – which means valuations can drift into fantasyland with no one to call bluff.

This is then reinforced by the layoff sprees by the largest companies in the air of a cooling economy, often justified by “potential upcoming AI productivity gains.”

Let’s be clear: these CEOs have not even AI-enabled the most basic tasks in their organizations yet. Still, they are making hopeful, wishful bets on an imaginary productivity improvement to boost earnings by a quarter in a flagging economy.

Thus, the AI centa-unicorn CEOs must keep the dream pumped full of hot air. Admitting “we’ve hit an intelligence wall, so we’re focusing on incremental usability improvements” doesn’t exactly justify a $170B valuation. Instead, we hear about grand AI milestones just over the horizon—artificial general intelligence, multimodal omniscience, you name it.

Meanwhile, the engineers quietly roll out features to make the AI a better team player rather than a genius. It’s a bit of a schizoid existence for these companies: publicly selling moonshots, privately grappling with product-market fit.

Sam Altman isn’t doing TED talks about prompt engineering techniques to improve ROI for insurance companies – he’s talking about godlike AI. Dario Amodei isn’t telling investors that Anthropic’s path to revenue lies in better enterprise data integration—he’s spinning visions of constitutional AI and safe superintelligence. And on it goes.

Herein lies an opportunity. The truth is:

Enterprises don’t necessarily need a significantly more innovative model than what we have today; they need one that plugs into their workflows, their data, and their problems. This is where companies like Oracle, Microsoft, and AWS could shine.

These enterprise stalwarts might not win a model development race against AI startups, but they deeply understand how businesses use software and data. They have decades of hard-earned knowledge about business processes, data management, and customer pain points. If they leverage that – effectively translating their domain expertise into AI-centric solutions – they can guide these AI models to deliver tangible productivity gains that matter.

In fact, we’re already seeing glimmers of this. Microsoft, with its Copilot suite, is wrapping OpenAI’s models in layers of business logic and context (think GitHub Copilot for code, Microsoft 365 Copilot for Office documents).

Oracle is touting how its cloud AI can seamlessly work with Oracle database data through the AI Data Platform, which we launched last week. They are quietly federating and preparing immense, rich data sources into a consistent data stream for AI agents, reserving context for solving business problems rather than figuring out how to connect to disparate data sources.

These companies get that it’s not the AI's IQ; it’s the usability and integration that determine success. They are the ones who can, for instance, connect an AI agent to a company’s entire customer support knowledge base and backend systems securely – something Anthropic alone can’t do from the outside. And customers will pay for that solution, not just raw model access.

Meanwhile, Anthropic and OpenAI find themselves in a weird position. They have world-leading models, but to actually make money, they have to become enterprise solution providers—a very different game from pure research.

And their current approach (sell API access or subscriptions and hope users figure out the rest) might not scale to the broad business market that still doesn’t know how to apply AI effectively. There’s a whiff of irony here: the AI startups might end up needing the boring old enterprise players to achieve the very productivity revolution that justifies their valuations. But why should investors pay for the value when the strategy has such a glaring gap?

Investor Exuberance vs. Reality

Let’s talk more about that valuation bubble, because it’s the elephant in the room. The AI investment landscape in 2025 is astonishingly frothy. There's speculation that OpenAI might IPO at a valuation well north of half a trillion dollars – more than the current market cap of JPMorgan Chase or Mastercard, to put that in perspective.

Anthropic is raising capital at valuations in the hundreds of billions as well. And yet, when you peel back the curtain, the financials are… well, not pretty. The companies themselves are incinerating cash. The only ones printing money in the AI gold rush are the pick-and-shovel sellers—NVIDIA (GPUs), maybe cloud providers, and chipmakers.

One news analysis put it starkly: “OpenAI is loosely valued at around $500B, yet may make just $13B in revenue in 2025… Anthropic is valued at $183B after a recent round, but is projecting < $10B revenue this year. They’re also spending far more than they earn. Nobody is making money with AI – except for NVIDIA, which is perhaps why it’s so keen to keep pumping the industry.”

This was written after Anthropic claimed it will hit a $9B run rate by the end of this year—an ambitious figure that many analysts view skeptically.

One particularly concerning trend is the circular nature of the mega-deals being struck. OpenAI’s partnership with NVIDIA, for instance, involves OpenAI committing to buy a staggering $X worth of GPUs, and NVIDIA in turn “investing” a similar amount in OpenAI’s equity.

Likewise, OpenAI’s deal with AMD is structured so that OpenAI will use a vast number of AMD chips and even take an equity stake in AMD. It’s almost comical: AI companies are taking the billions of investors gave them and handing it to chip manufacturers, who then plow it back into the AI companies’ valuations.

Here’s the scary part. A lot of this is being done through special-purpose vehicles (SPVs)—the old tactic Enron used to keep deals off the books.

Sound familiar? It should – it harkens back to the dot-com bubble’s vendor financing schemes, where telecom companies bought equipment on credit and the equipment vendors invested in those telecom startups’ stock. One veteran investor explicitly drew this parallel and warned that these AI deals have an uncomfortable rhyme with the year 2000. When money starts going in circles, it usually means actual economic value isn't being created in proportion –

Without public-market scrutiny, these private valuations float on optimism. And as long as everyone involved believes (or pretends to believe) that the next epochal AI breakthrough is around the corner, the music keeps playing.

But if actual utility doesn’t catch up to the hype, something’s gotta give. One of two things will happen: either companies will rapidly learn to harness today’s AI in transformative ways (justifying the investment), or the expectations will implode in a heap of disillusionment.

In either case, the longstanding leaders—Oracle, Microsoft, or AWS—still come out ahead.

I strongly suspect the near future holds a bit of both – some breakaway successes by those who get it right, and a lot of disillusionment from those who don’t. The “GenAI divide” we discussed will become more evident: a small segment of users will derive enormous value (with context-savvy strategies), and the rest will scratch their heads, wondering why their expensive AI subscription isn’t magically solving everything.

Worse yet, we see the “small segment of users” become the oligarchy of the current software industry, leaving the rest of the sector in a hellscape of financial ruin.

If and when the AI bubble pops, it won’t mean AI is over – far from it. It will just mark a shift from exuberance to normalcy, from grand narratives to practical implementations. The short-sellers of reality will finally have their day (figuratively speaking, since you can’t literally short a private company’s hype, but reality has a way of asserting itself eventually).

That correction will actually be healthy. It will force a focus on sustainable business models – likely those that truly deliver productivity gains to customers. And here we circle back to our central thesis:

The companies that guide users on how to get started, that automatically optimize use of context windows, that integrate with your existing tools and data – they will create genuine, defensible value. The ones that sell “we’ve got the smartest AI, trust us” will either adapt or fade.

Conclusion: Context Over Hype

The era of chasing ever-bigger, ever-smarter standalone AI models for their own sake is winding down. We’ve been to that mountaintop; we’ve seen what today’s best models can do. And they are amazing – but they’re also flawed and costly, and simply pushing the dial from “brilliant” to “even smarter” yields diminishing returns in real-world use. The next leap forward in value will come from embracing a humbler but ultimately more impactful truth:

It’s not about how intelligent the AI is; it’s about how intelligently we can use the AI within real constraints.

“Superintelligence” as a marketing concept may be on life support (sorry, investors), but that doesn’t mean progress is dead. It represents the locus of innovation shifts.

It shifts to AI-assisted context engineering, where companies focus on feeding the AI the right information in the right format (because an ounce of context is worth a pound of model capability).

1. It shifts to tool integration and automation, where an AI agent doesn’t just idly chat but actively interfaces with software, databases, and workflows, actually to get things done.

2. It shifts to user experience and guidance, ensuring that anyone can sit down with an AI system and be immediately productive without needing a PhD in prompt crafting.

3. And it shifts to memory and learning, so that AI systems accumulate knowledge over time rather than forgetting lessons learned with each new session.

We should welcome this shift. It’s a sign of maturation. Think of the early days of electricity – at first, it was all about generating more power, bigger dynamos, higher voltage! But eventually the breakthroughs came from the less glamorous work of distribution, wiring every home, and inventing appliances that effectively used electricity.

We’re at a similar inflection point with AI. The “power” is there; now we have to build the infrastructure and appliances around it.

Context is the wiring, the plumbing, the interface layer that will bring AI’s value into every nook and cranny of our world.

It may not sound as thrilling as conjuring a digital Einstein, but it’s how we’ll get tangible value.

Anthropic and its peers can choose to recognize this and lead the way—or cling to the superintelligence mythos and risk being leapfrogged by more pragmatic players. Personally, I hope they choose the former. I’d love to see Anthropic, for example, double down on making Claude the best damn assistant it can be given fixed intelligence. Make it the AI that never forgets what you tell it, that automatically brings in relevant documents when needed, that knows how to use a wide range of tools out of the box, and that can coordinate subtasks among multiple copies of itself seamlessly. That would genuinely move the needle for users – and they’d pay handsomely for it, too.

In closing, let’s drop the hubris of chasing an imaginary god-level AI and focus on augmenting the very real (and very impressive) AI we already have with context, memory, and wise guidance. The firms that do so will unlock the next wave of productivity – and those that don’t will be left pitching fairy tales. The bubble of hype may well pop, but what remains will be a more grounded industry focused on solving real problems.

Superintelligence is dead. Long live context.

Cue the breathless headlines and the stock market doing its best Chicken Little impression. But I’m here to tell you: take a deep breath. A 95% failure rate in the first inning of a technological revolution is about as surprising as rain in Seattle. In other words, it’s early. We saw this movie before, and spoiler alert: it didn’t end with technology slinking away in defeat. It ended with technology quietly conquering the world while everyone who proclaimed it “overhyped” was busy eating crow.

Lessons from the Dot-Com Bubble (or, This Isn’t My First Rodeo)

I lived through the dot-com era, when the Internet was hyped as the end of everything. Remember the late 90s? Pundits declared that the web would render physical retail obsolete, render real-world intimacy extinct, and convert all goods into digital form.

Meanwhile, skeptics swore they’d never trust this “cyberspace” thing for something as mundane as paying a bill. (Fun fact: my own father confidently told me I’d never be able to pay bills online. He was only off by, oh, everyone.) Both sides were spectacularly wrong. The truth was boring and profound: the Internet didn’t replace the physical world – it augmented it.

Consider retail: the Internet was supposed to kill brick-and-mortar stores dead. Fast forward to 2025, and over 80% of U.S. retail sales still happen in physical stores (livain.com). E-commerce accounts for only about 16% of retail – growing, yes, but not exactly an extinction-level event for malls. Instead, we got integration: buy online, pick up in-store; shop in-store, order online. The significant shift wasn’t replacement, it was partnership (livain.com).

That 1966 TIME magazine prediction aged like milk. In reality, both women and men are plenty happy to buy things online when it’s convenient (content.time.com).

The dot-com boom itself had a notorious bust. The NASDAQ index declined by 75% between 2000 and 2002, erasing approximately $5 trillion in market value. Pets.com became a punchline. Yet out of that rubble emerged Amazon, eBay, and other survivors that went on to dominate the global economy.

Bubbles burst – but they do not tell the whole story of an era. The dot-com bubble was a bubble, to be sure, but it also heralded the internet age, which fundamentally changed how we shop, work, and socialize. Physical stores didn’t disappear; they evolved. Human connection didn’t die; it found new channels (ever heard of social media?).

And I eventually did get to pay all my bills online – sorry, Dad. I should note that my Dad was actually an early and infrequent user of TurboTax and Intuit. He did use the bank to pay vendors online, but as a check.

The lesson: early failure rates and hype cycles are not reliable predictors of long-term value. They’re more like the overly dramatic opening act that makes the eventual success story that much sweeter.

It’s not evidence that AI is a worthless fad – any more than Pets.com’s collapse meant “the Internet is over.” In fact, it’s expected. We saw the same pattern in earlier tech revolutions, from the Industrial Revolution’s railway mania (where even Charles Darwin got swept up in wild railroad speculation to the smartphone era.

There were bubbles, panics, manias – and then there was enduring change. Every tech revolution has a bubble; historically, that’s practically a law of nature. The presence of hype (and the inevitable pop) doesn’t negate the technology’s eventual impact. Bubbles are a feature of progress, not a bug.

Humans + AI: Augmentation, Not Obliteration

Now, about those doomsayers who growl that “AI will replace all the humans.” To put it in terms my Grammy would appreciate: hogwash. AI is a tool – a powerful one, but still a tool – and tools need people. We’ve been through automation scares before.

Remember when elevators were automated and we thought elevator operators would become obsolete? Okay, they did disappear… but somehow the economy survived and new jobs emerged.

Every wave of technology automates some tasks but also creates new ones, elevating the human role to a higher level of value (or at least a different one).

The smart money (and smart people) have realized that AI is best used to augment human capabilities, not replace them outright. Sundar Pichai calls AI an enabler of human potential, not a substitute for human teachers. And former IBM CEO Ginni Rometty put it perfectly:

In other words, the future belongs to humans who partner with AI, not those who compete with it or hide from it.

Let’s check the reality on the ground. Are we seeing mass layoffs and a total takeover by robots? Not really. According to MIT’s data, AI hasn’t (yet) led to widespread job replacement – layoffs due to AI have been limited and industry-specific. Frankly, one could argue that AI was more of an excuse for a layoff rather than a traceable outcome of replacing humans with models.

Executives are divided on whether AI will reduce headcount in the future, which is a fancy way of saying “nobody really knows yet.” Meanwhile, surveys show workers are embracing AI as a helpful colleague. One study found that 70% of people prefer AI for quick, mundane tasks, but 90% still prefer humans for complex projects that require judgment and nuance.

In other words, we’ll let the chatbot draft an email or summarize a document, but when it comes to high-stakes decisions or creative strategy, most of us still holler for a human.

Think about your own work or life. Are you using AI daily? If so, you’re not alone. The accurate measure of AI’s impact isn’t how many human jobs it replaces; it’s how many humans find it useful.

Right now, that number is skyrocketing. ChatGPT, for instance, hit 100 million users faster than any product in history and is reportedly used by around 190 million people every day.

And those are voluntary users – nobody’s forcing anyone to use a free AI assistant, people are choosing it because it adds value to their day. In many companies, employees aren’t waiting for permission either.

An MIT survey found that workers at 90% of large firms are using personal AI tools (such as a free ChatGPT account) on the job – often under the radar – while only 40% of those companies have officially purchased enterprise AI solutions.

That grassroots adoption is a leading indicator that AI is here to stay. When something becomes genuinely helpful, people vote with their feet (or clicks). By the time your workforce is secretly using AI to get work done faster, the genie is well out of the bottle.

The “AI will replace us all” trope is just the latest episode of Tech Panic Theater. Relax. AI will replace some tasks (probably the dull, repetitive ones first), but it doesn’t replicate the full spectrum of human skills anytime soon.

We’re not obsolete; we’re evolving.

In the near future, if you’re great at leveraging AI, you’ll be more valuable, not less. If you stubbornly refuse to use these tools, well, don’t be surprised when you’re outpaced by colleagues who do. This isn’t a zero-sum game of human vs. machine; it’s about humans who harness machines vs. humans who don’t. As a character on a TV show once said (handsome guy, presidential vibe): “What’s next?” The ones embracing AI are already figuring that out.

The New Breed of Coder (No More Code Monkeys)

Let’s get specific: jobs and professions will change, 100%. In fact, they already are. Take software development, a field near and dear to my heart. We’re moving from the era of developers hand-coding every line (like monks illuminating manuscripts) to an era of “vibe coding.” And no, it won’t be limited to just non-technical product managers.

For those under a rock, what on earth is vibe coding? It’s when developers focus more on what to build than on how to type it out – essentially leveraging AI to generate code while they guide the high-level design. Think of the developer as an architect or editor, and the AI as an eager junior programmer who never sleeps.

Some traditionalists harrumph at this: “But do these kids even understand the code?”

Listen, code is becoming increasingly affordable; understanding is becoming increasingly expensive.

We now have AI that can crank out functioning code faster than a roomful of interns fueled by Red Bull. That doesn’t make human developers obsolete; it makes their architecture skills more critical.

As one young engineer wrote:

In other words, knowing what to build and why becomes the real superpower, even if the AI handles much of the “how.”

I am an active vibe coder. I use AI tools to automate mundane tasks – such as boilerplate code, simple functions, and unit tests – so I can focus on the big picture and more complex logic. Understand an insight from that large open-source Spark codebase - sorry, but Claude beats most any day.

Do I understand the code that gets produced? You bet I do – experience and solid fundamentals matter more than ever when you’re supervising an AI sidekick. In fact, I can often build bigger, more complex systems because I’m not bogged down writing every semicolon myself.

Meanwhile, a less experienced coder trying the same tools might get lost in a sea of AI-generated spaghetti. The curmudgeons asking “But do you really understand it?” are asking the wrong question.

If yes, who cares if an AI wrote 90% of the trivial code? We’re solving bigger problems with less tedium. (And if no, well, that project was doomed whether a human or AI wrote the code.)

Mark my words: in a few years, manually writing every line of code will be about as common as manually doing long division. Sure, you can do it, and sometimes you should for learning or precision. But most of the time, you’ll let a calculator – or in this case an AI – handle the rote stuff while you provide the guidance and creativity. This doesn’t spell the end of software developers; it spells the end of software developers acting like machines. We get to be more human – more design-oriented, more creative – and let the machines do the machine work. That’s progress. Just don’t confuse using power tools with not knowing how to build; the master carpenter still measures twice, even if a nail gun does the hammering.

Bubbles, Bubbles, Toil and Trouble

Let’s tackle another buzzkill: “AI is just a bubble! It’s all hype and froth!” Well, is AI in a bubble phase? Quite possibly yes. But here’s the thing:

Has any major tech revolution not had a bubble?

Bubbles are practically a rite of passage for transformative technologies. They’re the economic equivalent of teenage acne – an annoying and ugly phase, but a sign of growth nonetheless.

History lesson: the Industrial Revolution was punctuated by wild speculative bubbles. For example, Britain’s Railway Mania in the 1840s had investors literally tripping over each other to buy railway stock, driven by hype about a world-changing technology (railroads) and dreams of untold profits.

Share prices doubled, countless new railway companies were floated on nothing but optimism and maybe a nifty route map. And yep, it crashed spectacularly. Fortunes were lost, and panic ensued. Does that mean railroads were a dud? Hardly. After the smoke cleared, the tracks still got laid, trains still transformed commerce and society, and the world moved forward (quite literally, on rails). The bubble was a footnote; the railways themselves were the story.

Fast forward to the dot-com bubble of 1999–2000 – we touched on this, which was frothy beyond belief. Companies with no profits (and sometimes no products) were Going Public for billions. When reality set in, the crash wiped out trillions of dollars in paper wealth (internationalbanker.com).

But was that the “end” of the Internet? Tell that to Amazon, Google, and Facebook. Today, the survivors of dot-com mania are so successful that we’re busy worrying they have too much power. The bubble was painful for investors, sure, but it wasn’t a verdict on the technology.

It was a necessary shakeout that separated flimsy ideas from solid ones.

The Internet boom left behind real infrastructure, new business models, and a generation of people with digital skills. Similarly, the current AI gold rush may well end in a pop – with many VC darlings flopping and some cool demo products never finding a market. We might indeed see an “AI winter” of sorts if expectations race too far ahead of what’s possible in the short term. But that isn’t a death – it’s a comma in the story of AI, not a period.

If you want proof that bubbles don’t equal doom, follow the money after the bubble. After the railway bubble, railroads continued to be built. Following the dot-com crash, Internet usage continued to grow exponentially.

Right now, we’re already seeing AI woven into everyday life, bubble or not. Investors may chase fads (oh boy, do they ever – nothing like FOMO to inflate a valuation), but real adoption is driven by real utility, not stock prices. So yes, by all means be wary of irrational exuberance (and keep a pin handy for when valuations get overinflated).

When this AI bubble, if it exists, eventually pops, we’ll lose some pet startups and paper billionaires – but we’ll be left with the tangible advancements AI brought in the meantime. The leading indicator of AI’s value isn’t how giddy VCs are feeling this quarter; it’s whether people and businesses keep using these tools when no one’s looking. And all signs on that front point to “yes.” Utility has a long half-life.

Watch What People Do (Not What They Say)

Let’s drill down on that point: the leading indicator of AI’s significance is everyday human usage. Not venture funding, not media hype, not doomsday op-eds – just plain old daily active users. If people integrate a technology into their daily routines en masse, then take it to the bank: that tech is making a dent. By that measure, AI is already performing exceptionally well.

We’ve mentioned ChatGPT’s massive user base. It’s not alone. Enterprise AI adoption, even if clumsy, is widespread. Over 80% of organizations have experimented with tools like ChatGPT or Microsoft’s Copilot, and nearly 40% have deployed them in some form (aimagazine.com). That’s in less than two years since these tools burst onto the scene.

When something is genuinely useful, you don’t have to force it down people’s throats. Remember when smartphones came out? BlackBerry to iPhone era – nobody needed to mandate “phone usage”; people lined up around the block. We’re seeing a similar organic uptake with AI assistants and automation tools. Employees are using AI to draft emails, brainstorm ideas, generate reports, and more.

Students are (sometimes sneakily) using it for homework help. Doctors are starting to use it to summarize patient notes. I even know a guy who used an AI to argue with his cable company’s customer support (the AI was very politely ruthless, I’m told). This is the stuff that doesn’t always make flashy headlines, but it’s the ground truth of adoption.

So if you’re trying to gauge where AI is going, look at the demand-side indicators:

We’ve crossed that threshold already. Entire communities of “AI power users” have emerged, sharing tips on how to craft the most effective prompts to achieve the best results. Heck, we’ve coined the term “prompt engineer,” and there are job postings for it. (Yes, we live in a world where talking to a computer really well can be a six-figure job. Take that, high school guidance counselor who told me chatting wasn’t a skill.)

AI’s daily integration into workflows is accelerating. That is why I don’t lose sleep over whether some CFO thinks the ROI of generative AI is lacking after six months (marketingaiinstitute.com).

Of course, most pilots haven’t hit ROI yet – they’re pilots! If anything, I worry more about companies moving too slow on adoption, not too fast. The real risk isn’t that AI never delivers value; it’s that you fail to figure out how it can provide value for you while your competitors do.

Because once someone cracks that code and starts using AI effectively every day, it’s hard to catch up. (Think of how some companies adapted to the internet and some didn’t – those that got it, Amazon, thrived; those that scoffed, Borders, died.) In short: watch usage trends like a hawk. That’s where the signal is, buried under all the noise.

Betting on the Right Use Cases

If you’re an entrepreneur, executive, or just a curious bystander wondering where to place your bets in the AI arena, here’s my advice:

Focus on augmentation, not pure automation.

Look for areas where AI can empower humans to be more productive, rather than simply replacing humans in a zero-sum swap. In practical terms, the sweet spot for AI projects is those where:

• A human is still in the loop. Aim to pair AI with human judgment rather than removing humans entirely. Humans plus machines > machines alone, in many complex domains. (Think AI diagnostic tool + doctor, not AI doctor with no human oversight).

• Success isn’t defined as “firing an entire department.” If your AI project’s ROI relies solely on eliminating jobs wholesale, you’re likely doing it wrong. Projects framed around massive headcount reduction tend to breed resistance and ethical issues, and often fail to capture the nuanced work that humans actually do. Instead, define ROI in terms of enhanced productivity, quality, or new capabilities, rather than just cost-cutting through layoffs.

  Dear CEOs: Doing layoffs while claiming AI justifies it will breed AI resistance.

• Productivity and value-add are the goals. The best AI use cases actually unlock new value or save significant time, rather than just doing the same work slightly cheaper. If an AI can do a task in 5 minutes that took a person 5 hours – that’s a win. If it can enable a new service or insight that wasn’t feasible before, even better. The ROI will follow from the new value created, not just from shaving a line item in the budget.

• You’ve tried it manually (with AI) first. Before you invest $10 million in building a custom AI solution, test the concept with existing AI tools. Can a bright intern accomplish a similar outcome using off-the-shelf AI like ChatGPT or Claude? If yes, you have validation. If no, rethink. Far too many AI projects fail because nobody bothered to see if the idea actually works in practice at a small scale. Treat AI projects like science experiments: form a hypothesis, run a quick test with minimal investment (even if it’s a hacky solution), and observe results. Only then double down. Given the plethora of AI services available, there’s almost always a way to simulate the workflow manually to gauge feasibility. Use that to your advantage.

In short, bet on use cases where AI is a force multiplier for humans – where it’s “Humans with AI” rather than “Humans versus AI.” The companies that navigate this well will reap the benefits. Those that swing for fully autonomous solutions that magically replace dozens of employees in one go… well, they often end up as that statistic we opened with (the 95% that fail to launch). And here’s a pro-tip: if you can’t achieve a meaningful result with a baseline AI tool (like using ChatGPT manually), throwing tens of engineers and millions of dollars at a custom AI probably won’t fix that. Use the readily available tools as your canary in the coal mine. ChatGPT and Claude are, in this sense, leading indicators for your AI project – if they help you get a job done in prototype form, that’s a green light to invest further. If they don’t, maybe the idea isn’t ripe or realistic yet.

The One Superpower (AI Has, Humans Don’t)

Finally, let’s address a fascinating new capability that AI brings to the table – one that truly differs from what groups of humans can easily accomplish.

What do I mean? I mean, you can deploy an army of intelligent agents that specialize in different tasks, and have them work together towards one objective, without office politics, without coffee breaks, and without each agent asking “but what’s in it for me?” Other than perhaps pleasing their human user, these AI agents have no agenda or ego. They are, in a sense, the ultimate team players – because they’re all on the same team. Your team.

This is something new in the world. When you put a group of humans on a project, each person has their own motivations – career aspirations, pride, competition, genuine teamwork, or perhaps not. Humans are fabulous, messy creatures.

A group of AIs, however, can be spun up to instantly collaborate with a singular focus on the task. We’re already seeing the early versions of this: one AI agent can now delegate subtasks to another, then to another, forming a daisy chain of digital workers all coordinating to solve a complex problem (medium.com). It’s like having a personal swarm of experts who never sleep and never argue. (Unless you prompt them to role-play an argument, I suppose, but that’s on you.)

Imagine a future scenario: You, the human, say, “Hey AI, plan me a marketing campaign for our new product launch.” That request might spawn a cadre of AI agents – one generates market research, one drafts social media copy, one designs graphics, one crunches budget numbers – and they all feed results to a coordinator agent that compiles the final plan for you.

This isn’t sci-fi; prototypes of such multi-agent systems already exist, and they’re improving rapidly. I’m building one right now.

The real value here is speed and scalability. Need ten times the output? Spin up ten times the agents. Humans don’t scale that way. You can’t hire and train 100 new employees for a two-week project, but you can certainly instantiate 100 AI agents for a task and then shut them down when done.

Now, this isn’t to say AI agents can do everything perfectly (they absolutely can’t, at least not today). They will make mistakes, and they still need a human at the helm to guide and QA the overall process. But this capability – let’s call it massively parallel cognitive labor – is genuinely novel.

It means that workflows previously bottlenecked by human coordination can increase efficiency.

Humans, bless us, are not built to be 100% obedient or focused on someone else’s goal 24/7. We think about lunch, or our paycheck, or how Karen in accounting never says thank you. AI agents don’t grumble. They grind away at the problem set before them.

This is one area I advise keeping a sharp eye on. We’ve only scratched the surface of multi-agent AI workflows. Early experiments (AutoGPT et al.) were clunky, but they hint at what’s coming. The main complaint was latency. But if you’re augmenting the productivity of a human, turning an hour into days of accomplishment, latency is not a factor.

When this matures, it could supercharge productivity in ways we haven’t yet fully grasped. It’s not about replacing people; it’s about giving each person a battalion of tireless assistants. If I can delegate ten tasks to AI agents and have them coordinate among themselves to deliver a result by morning, that’s a game-changer.

For businesses that scale faster than hiring and onboarding an equivalent human team (and again, no hurt feelings when you turn them off afterward).

The former will always outcompete the latter. We’ve seen this pattern from the steam engine to the spreadsheet. AI is just the latest, albeit most brain-like, tool we’ve created. Yes, most early projects will fail – as experiments must. Yes, there will be hype, bubbles, and disillusionment. And yes, jobs will change – some will go away, and new ones (we can’t even imagine yet) will emerge. But through all that sound and fury, the fundamental story is one of enablement.

AI enables us to do more, faster – sometimes astonishingly more – but it still relies on us to point it in the right direction.

So the next time someone declares, “AI is overhyped” because a pilot project failed, or proclaims, “AI will replace everyone” because a pilot succeeded, just gently remind them: “Post hoc ergo propter hoc” – kidding, kidding.

Remind them that we’ve been here before. The early failure of 95% of projects is not a death knell; it’s a familiar stepping stone. The wild predictions of total automation are not prophecies; they’re the echoes of past fears. The truth is, as it often is, more nuanced and a lot more interesting:

I have news. After months of cryptic posts about "exciting things ahead" and "can't wait to share what's next" (sorry, I know how annoying those are), I can finally tell you:

I've joined Oracle as Senior Director of Generative AI Customer Engagement, working on an OCI Big Data service.

Because I'm about to tell you why 95% of enterprise AI projects are failing, why your chatbot has the memory of a goldfish with amnesia, and why I chose to tackle these problems at a company that many of you probably associate more with a particular person’s yacht collection than AI models.

The Uncomfortable Truth About Enterprise AI

If you follow my blog, you know I love to paint pictures. You know that fancy AI agent your company deployed last quarter? The one that was supposed to revolutionize customer service/DevOps/sales/[insert your department here]?

There's a 95% chance it's currently sitting in a corner, crying into its context window, overwhelmed by the complexity of actually doing its job - kind of like Towelie after he smoked a doobie on the Tegridy farm.

MIT and Fortune aren't making these numbers up for clicks. I love the Kumare, Kumare chortling prophet posts that the 95% metric is clickbait.

It isn’t.

Here's what nobody tells you at those slick AI demos: Models have terrible memories. They are worse at finding information.

They're like that friend who swears they remember your birthday but always texts you three days late.

They exhibit what we call a "U-shaped attention curve" - they remember the beginning of a conversation ("Hello!") and the end ("...so can you help?"), but everything in the middle?

Gone. Vanished. Lost in the token void.

The Healthcare Story That Changed Everything

While I was building my startup, I met an extensive midwestern healthcare system. They were aggressively piloting and trying to use AI. They wanted AI to monitor patient vitals - seems simple enough, right? Wrong.

Their first attempt was like trying to keep track of a kindergarten class during recess with one perplexed substitute teacher. The AI couldn't remember which patient was which. It was attempting to act asynchronously as an agent across multiple patient workflows.

They had to create separate context windows for each patient, with a steering model just to manage the chaos.

Even then, the memory continued to dilute over time, much like trying to make soup by continuously adding water. Not great when you're dealing with, you know, actual human lives.

The solution? They built a structured memory system that could slice information by topic, time, and relationships - essentially giving the AI a proper filing cabinet instead of a pile of sticky notes scattered on the floor.

But they also have the R&D budget of many rural hospitals. And the patience to work with new technology. Two characteristics that most enterprises lack have created a new fear of an AI bubble. That is the Oracle opportunity.

Why Oracle? (No, Really, Why Oracle?)

I know what you're thinking.

Yes, that Oracle. And here's why that's exactly where the AI revolution needs to happen:

1. They Have All The Data. Oracle's Fusion Applications run the critical operations for most Fortune 500 companies. We're talking about the data that actually matters - not your social media likes or what you had for breakfast, but the stuff that keeps the global economy running. If you want AI to work in the enterprise, you need to be where the enterprise data lives.

2. Infrastructure That Actually Works. While everyone else is fighting over GPU allocation and wondering why their models keep timing out, Oracle has quietly built one of the most robust cloud infrastructures on the planet. Turns out, when you've been handling enterprise workloads for decades, you know a thing or two about scale.

3. The Boring Stuff Is Where The Magic Happens. Everyone wants to build the next ChatGPT. But what enterprises actually need is meat and potatoes.

a) AI that can not only remember what happened in a customer conversation three months ago, but can fully understand the context and relationships, and enrich the conversation with recent, trusted facts in a secure way.

b) AI that can handle context switching without having an existential crisis.

c) AI that doesn't hallucinate regulatory compliance requirements (because that's how you end up in court).

The Vision: AI That Doesn't Suck at Its Job

While the prophets chase AGI and believe “it’s all in the model,” I tend to be a bit more practical about the state of the industry.

Here's where I see large data platforms evolving towards: Intelligent platforms that can process petabytes of big data while actually solving the AI memory problem. Imagine agents that can maintain context across weeks or months, not just minutes. Agents that can be invoked on schedule to check security logs without forgetting what a security breach looks like halfway through. Agents that can handle asynchronous tasks without losing their minds (or their context).

When an AI needs to remember something, it doesn't have to frantically search through conversation history like you looking for your keys. It does not have to piece together “context” from a chat transcript written by a millennial. It gets a structured summary, complete with all the relevant context, ready to go.

The Plot Twist

After spending months consulting enterprises to fix their broken AI implementations (including building Camille and Supastate for Claude Code (I crossed 700 users this week, thank you!) and now Gemini, because even I need my coding agent to remember what project we're working on), I realized something.

It's like giving Einstein a brilliant problem to solve but making him start over every five minutes. It's no wonder that 95% of projects fail. We also don’t fully understand the intelligence-latecy paradox of using multiple agents, especially in ad-hoc context, which I demonstrated with Project Hawking Edison.

I think my jaw dropped when I saw a panel orchestrate a roadmap planning session for my wife’s company’s leadership team. But I also had to code a ton to support this: a shared context store, a hallucination and echo chamber detection algorithm, an asynchronous monitoring mechanism, and a data warehouse with vector search for knowledge.

This work is out of the reach of most enterprise customers trying to make it work at scale.

So What Now?

I'm joining Oracle to build a practical future for enterprise AI. Not an agentic nirvana where every employee is replaced with a chatbot that forgets after 250K tokens. Not because it's easy (it's not), not because it's sexy (enterprise information architecture and data warehousing rarely is), but because it's necessary.

Someone needs to build the boring, yet critical, infrastructure that makes AI truly valuable for the enterprise. Someone needs to solve the memory problem, the context problem, and the "why-does-my-AI-agent-keep-forgetting-what-company-it-works-for" problem.

The Call to Action (Because Every Substack Needs One)

If you're part of the 95% struggling with AI implementation, let's talk.

If you're in the 5% who succeeded, let's definitely talk (and please share your secrets).

If you're just here for the jokes about enterprise software, welcome; there will be many more.

The enterprise AI revolution won't be built on prophets, valuations, demos, and POCs.

Like teaching AI to remember things. Revolutionary, I know.

Welcome to my Oracle era. It's going to be wonderfully boring in all the right ways.

And I would like to thank the Oracle team for tolerating me!

P.S. - To my former clients who are reading this: Yes, your AI agents will still remember me. I made sure of that. Context windows are forever.

P.P.S. - To my over 700 loyal Supastate users, yes, I’ll continue to operate it for you!

Context: It has been a tough last couple of weeks for the blind faith of the AI industry. The prophets chortled “Kumare, Kumare” at the top of their lungs as GPT 5 was released (do they ever say a model isn’t a breakthrough?).

I watched as the many Twitter covens and glitterati eagerly debated whether Elon may have gamed his model to pass high school math exams.

Gary Marcus published a Lutheran damnation of the general state of AI. OpenAI claimed it could make a model run for months on the same problem with no human intervention (sorry Sam, I can do that too - and model breakdown and echo chambers are a real problem). Meanwhile, Hacker News was debating the GPT-5 demo fail on Bernoulli.

Generally, except for Gary, all of the prophets have missed the point. The problem isn’t parameters, benchmarks, or how long you run it for. No, the real problem is memory.

Today, I will take you through my personal struggles with LLM memory. I have to admit, Gary may be right over the long term: the most innovative models will not be entirely language-based.

But there are quantum leap practical benefits that can be derived from hacking the models we have right now as well.

You know, the software industry always loves its cycles.

Back when I was a kid, we would always see CPUs zoom ahead, but available RAM always lagged. Like clockwork, Intel (now NVIDIA) would release the latest edition of its CPU, but memory space would not correspondingly increase.

I feel like we are running into the same challenge again, but this time we’re not talking about ephemeral RAM; we’re talking about context.

And before you wring your hands, no amount of broadening the model token context window will ever keep up with the temporal, winding, non-linear nature of human communication and thought.

And what was interesting was that while I watched the prophets fight online, observed my LinkedIn doomscroll debate the valuations of OpenAI and Anthropic, and laughed when Elon asked people to convert their repos into one large text file to submit to Grok 4, I was quietly counting Supastate fatals.

What is a Supastate fatal, you may ask?

Well, let me catch you up. Supastate enables Claude Code to restore an arbitrary context based on a topic. When a developer switches tracks to another part of their project, the last context compaction is relatively useless. So, how do you get Claude Code up to speed? Well, naturally, you semantically search the chat history, and then use Claude Opus to create a summary context to restore context.

But I nervously watched as, roughly half the time, the context wasn’t helping Claude Code. It was finding the right chats and code, but it just wasn’t helping.

Why? I naturally assumed it was a problem with Supastate. As I dug into the problem, I would learn a far more profound truth:

LLMs: Borg-like CPUs, Dog-like RAM

There’s a particular kind of heartbreak that only modern knowledge workers know: you lovingly explain a problem to your AI, it nods like a golden retriever with a Stanford CS minor, and five minutes later—poof—your context is gone like Wordle on hard mode.

The next sychophantic reply arrives with the confidence of a TED Talk and the memory of a Snapchat thread.

I mean, even Max has a more extended memory. For example, he remembers that the dental chew bone treat he gets every night is his equivalent of puppy heroin, and his mom is a mean, teasing, daily drug dealer at 8 PM. I’m not sure Claude Code could remember what it wrote 15 minutes ago.

Claude Code has a (terrifying?) workaround user experience to solve for this - compaction. As you interact with the model to write code, you can ask it to compact at working breakpoints, or it will automatically compact for you when it runs out of context windows. The way it seductively continues to work across compactions on the same topic lures you into believing that it has retained context.

Yes, some dilution occurs as information gets continuously compacted, but it appears to work well for continuity. Well, that’s good window dressing around the U-shaped curve of importance models place on information at the beginning and the end of the sequence. From Nelson Liu and team’s amazing paper:

We have all seen it before in simple prompt engineering. The first and last things you write in the prompt are brutally prioritized. Unfortunately, this is not how humans think about problems. With iterative coding, we are delighted because the last iteration tends to be the most important - until you get into a large, multi-step project.

But, back in July 2025, I had fallen hook, line, and sinker for the user experience tuxedo. I built Supastate to provide context restoration from arbitrary topics. After all, something was better than nothing. Then came the plot twist worthy of Black Mirror: today, Claude rolled out an official “reference past chats” feature in its web product. Helpful if you explicitly ask for it; opt‑in; landing first for paid tiers. Great timing, Claude. Bold move.

The uncomfortable diagnosis: Long Context ≠ Good Context

First, let me give you some interesting metrics:

• Only 58% of Supastate’s context restorations were helpful to Claude Code. I added two hooks into my (personal) implementation of Supastate. First, every time I gave Claude Code an entirely new set of tasks, I would ask it to restore context about the task using Supastate.

  How? Claude Code would generate semantic search terms for the new task and pass them to Supastate. Supastate would search a vector index of all historical chats and a CRAG-based search of code, combining a semantic and syntactical search strategy. It would return the resulting chunks back to Opus, which would use a prompt almost identical to Claude Code to generate a context summary. This summary would be provided back to the Claude Code instance executing the task. Then I would ask Claude Code to evaluate whether the context helped complete the task. To aggregate the metric, I would (we’re back to Inception) use the same vector index to search for these operations.

• In 93% of the cases where the context restorations were not helpful, Claude Code consistently complained about a lack of “architectural understanding.” This is LLM code for reasoning. When I manually audited these failures, I noticed an interesting pattern - dependencies and references. Claude completely missed relationships. Summaries didn’t help. Notes to linked references didn’t help either.

• These metrics didn’t get better when Claude Code was given access directly to the knowledge graph. I figured I must be doing it wrong. After all, I didn’t build the model. The folks at Anthropic are brilliant. Well, the model doesn’t do a better job either. When I alternatively gave the model direct access to three tools: (1) the ability to run its own semantic searches (with optional summarization); (2) direct access to the relationships and the graph; and (3) the ability to create stored concepts to help it build an architectural understanding, it performed worse (31% context restoration rate).

  For those who believe vectorizing large enterprise knowledge stores and throwing it at LLMs will make AI more intelligent, watch out! This is the number one pattern I have seen a sickening number of CIOs take. Let’s take every document we've ever had and stick it in a Pinecone index; our AI will be smart! Let’s install the GitHub and Salesforce MCP tools and provide raw database access! Or better yet, let’s give our AI root access to Supabase! This won’t work. It’s kind of like yelling at your dog. You can throw a lot of English words at him, but he won’t really know precisely what any of them means, and the relative priority of all the information. And your dog’s goal is to please you, not to devise a strategy that uses tools to achieve that goal.

Here’s the professor‑with‑a‑laser‑pointer part.

1. LLMs struggle with where facts sit in long prompts. The now‑canonical Lost in the Middle study shows accuracy peaks when facts are at the very start or end and slumps when they’re buried in the middle. That U‑shaped bias survives even in models designed for giant windows. So “Just give it everything” is not a strategy; it’s a vibe.

2. Compression and re‑packing help. Methods like LLMLingua or LongLLMLingua compress prompts while preserving salient bits and often improve performance, precisely by fighting that positional bias. Translation: less hay, clearer needles.

3. Benchmarks in code land agree. Tasks like SWE‑bench and repo‑level completion show that real fixes usually require coordinated changes across multiple files. LLMs need more than line‑by‑line autocomplete; they need a map. Code is too exact—and inconveniently scattered across a Fortnite‑sized map of files. Repo‑level work demands stitching relationships, not just remembering a function signature. Research on repository‑level completion and CodeRAG variants is converging on a simple idea: retrieval must be selective, structured, and relationship‑aware.

So when my experiment returned “not understanding the architecture” 93% of the time, Claude said context wasn’t helpful, and I believed it. The model wasn’t failing to read; it was failing to organize.

So It Isn’t Code. It Isn’t Chat. What Is It?

I did see a hint in the original dataset. When context was helpfully restored, I noticed that about two-thirds of the time, a markdown file was in the source code search results. Typically, the design document that I would ask Claude to generate is part of the initial step of building a feature. I was automatically generating a Haiku summary of this design document, which was embedded into the results used to create the restored context.

Unsurprisingly, the best thing for a large language model to remember is… language. I had solved it. World domination, here we come. Chats are too squishy.

If you squint, the cure is apparent: don’t hand the model a transcript; give it a design. So I happily started building a design document indexing mechanism. I stuffed it full of everything conceivable - GitHub issues, markdown files, related links - a massive knowledge source.

At this point, I was convinced that large language models would never move beyond the simple tasks they were being used for. They just failed to understand and remember relationships. I also noticed that Opus 4.1 was performing worse than Opus 4. I was starting to believe that Gary Marcus was right and we were heading towards an AI meltdown.

But then the data gave me a glimmer of hope.

The surprisingly effective fix: design docs + diagrams, co‑authored

When I forced a workflow that always generated or refreshed a simple design doc—and I mean forced, as in “system prompt: restore context first, then update the canonical doc, do not create a new one”—the number of back‑and‑forths to get on the same page dropped. I didn’t calculate the number of back and forths precisely, but it was obvious in the same way you know a latte is decaf.

Then I added three steps:

1. SVG diagrams produced by the model during compaction (boxes, arrows, interactions), then summarized in text and indexed alongside the doc.

2. Then, recurrently give this design document creating tool access to the same set of code and chat search tools Claude Code has access to, enabling it to search its own information space.

3. Human‑in‑the‑loop edits on the doc, captured and vectorized for broad semantic search.

Between those, context utility in restoration jumped to 84%. Not because I taught the model to “think,” but because I made the structure think for it.

For example, here was a diagram that was automatically created during a compaction event (encoded in SVG):

Weirdly, this diagram means absolutely nothing to me, but when I restored context to fix a bug with this component, in one step, Claude Code knew precisely how it was organized.

If that sounds quaintly neuro‑symbolic, it is. There’s a thicket of evidence that structured representations (graphs, diagrams, programs) make LLMs more reliable reasoners:

• Knowledge graphs + RAG (aka GraphRAG) use nodes/edges as first‑class retrieval units and consistently give better multi‑hop answers than pure text chunks. Microsoft Research has been loud about this, and newer surveys echo it.

• Diagrams aren’t just clip art. Systems like DiagrammerGPT plan a diagram first (entities, relations, layout) and then render it, improving faithfulness; recent work also tests whether VLMs actually understand diagrammatic relations. The gist: pictures help when they encode relations, not vibes.

• Prompt compaction helps models notice the important parts, not just fit more tokens. That’s the LLMLingua line of work.

It may be tempting to immediately go out and spin up your own Neo4J instance, spending thousands of dollars per month. Don’t. I hate to break it to the Neo4J folks, but until models get better at actually running Cypher queries, the value of the graph is limited at best. And until models start thinking in terms of graphs (they don’t), it just isn’t going to help. Which leads me to…

“But you had a whole knowledge graph already!”

I did. And so do many of us. But here’s the rub: just because your system can run Cypher/SQL or swim in an MCP lake of tools doesn’t mean the model will choose to. Tool use reliability is its own discipline (see ReAct and Toolformer), and models are notorious for under‑calling or mis‑calling tools unless you make the invocation required, early, and specific. (Anthropic’s MCP docs explicitly position MCP as the “USB‑C for AI tools,” but the client still needs to plug the cable in.) Much noise was made of GPT-5s ability to run tools, but again, we still have the competition between using valuable context windows to teach the model how to use the tool versus using the tool.

In the meantime, I will still argue that 90% of the development investment into tools has to go into tool documentation and discovery, simplifying the API surface area down to one or two invocations (GPT actually cheats and forces the search endpoint to match their signature!), and realizing that ironically security will yield better context - a feature of tenant isolation is better context.

That’s why Supastate’s approach—make “restore context” a first‑class tool and make docs/diagrams the substrate—felt sane. You give Claude Code a persistent knowledge graph that captures decisions, relationships, and the thread of memory. Then you tell it, in writing, to use those tools first. (That is literally what Supastate’s setup recommends.)

The Playbook

Here is where I ended up making context restoration actually useful. I’m currently hitting a 92% context restoration success rate with this strategy.

1) Mandate context restoration as Step Zero.
Write it into your system instructions: “Before doing anything, call restore_context and update the canonical design doc.” Don’t be polite; be specific. (MCP makes this easy to standardize across projects.)

Read more

Context: I have not written in a while. No, I have been 100% heads down building. I apologize in advance to the many I have ignored.

After observing over a thousand downloads and a hundred daily active Camille users, I noticed that I was not alone, and many were frustrated with Claude Code’s shrinking context window and increasing cost.

But I also knew Camille had its limits.

So I started building something called Supastate. Here is the story of my ever-expanding goal to improve agentic coders — without waiting on AGI.

If you're interested in using Supastate, feel free to message me on Substack or LinkedIn. It requires a valid GitHub user to authenticate.

Unlike Camille, Supastate does cost dollars to run, so unless you are directly helping me with it, you will have to sign up for a subscription after a trial period.

Let me tell you something about memory. In 1953, Henry Molaison had his hippocampus removed to cure his epilepsy. The surgery worked—no more seizures—but Henry could never form new memories again.

Every conversation, every face, every moment dissolved into the ether seconds after it happened. Now, why am I telling you about a neurosurgery patient from the Eisenhower administration? Because that's precisely what we've been doing to our AI coding assistants, and we've been calling it a feature.

Walk with me here.

The Fundamental Flaw in Our Collective Brilliance

Remember Friendster? Of course you don't. It had 115 million users at its peak—that's more than the population of the Philippines—and it vanished like morning mist because it couldn't handle the load. MySpace had Tom as everyone's friend (a brilliant piece of social engineering, by the way), but it forgot that people wanted consistency, not just customization. These weren't technical failures; they were memory failures. They couldn't remember what made them successful or learn from what was killing them.

Now we've got Claude—brilliant piece of engineering, truly remarkable—and we've given it the same affliction Henry Molaison had. Every time you close that chat window, poof! Gone. Like it never happened. You know what that means?

But it gets better (worse). It loses context all the time. Some of this is the very nature of iterative software development. When you go to fix a bug you found while trying to build an unrelated feature, beware, you better get it done in one or two shots, otherwise it will dilute the context away from the feature you were building. It is challenging to predict how many tokens the next instruction will consume - and frankly, neither can Claude.

The Graph That Changed Everything (Or: How Neo4j Became Our Hippocampus)

Context for an AI code generation tool is a combination of memories and code interconnected in a graph. Ironically, the context that Claude attempts to keep is an elementary graph. It is an LLM summarization of your chat, with pointers to the code that was modified or created during the session. But code is a rich graph in its own right that combines syntax with semantics. Memories may involve code that was not directly touched or read. In other words, an elementary graph does not do the situation justice. And then add to that a recursively compacting context, and you now have a completely new challenge.

While a lot of focus in the industry has gone into the LLM and its ability to generate complex, multi-step code, I believe not enough focus in the industry has gone into memory. I suppose the belief is that we’re reaching AGI in the next year, so memory is a waste of time to focus on. But, as we know, memory context windows quadratically increase cost. And human code emitters suddenly make a ton of sense.

At the same time, I was absolutely sick and tired of watching Claude invent four different versions of an authentication pattern (with three different bugs each), forget the original purpose of a project, suffer from a compaction bug, or have no idea of context from dependent projects.

So I had to do something about the problem.

Enter Camille

Here's where it gets interesting. I started with Camille, backed by LanceDB and Kuzu—a lovely little embedded graph database that's very promising—and Camille served its purpose for a short while. Until I realized it was creating 16 GB vector stores, the graph was not very rich, and it just does not scale to have each user having to self-manage their own graph and vector database.

Learning #1: Initially, I provided an open graph MCP facility for Claude to query Kuzu, but Claude's code soon proved to be poorly utilized. I quickly learned why.

LLMs will not take the time to learn your schema of an MCP tool. Their priority is to answer the user’s questions, not teach themselves how to use the tools given to them. It makes sense, given the latency versus user experience tradeoff. Some modes, such as Claude’s “research mode,” will do this - but only if instructed to.

The net result is that an arbitrary query API through MCP will not work well with an LLM. This is bad news for the tens of thousands of MCP adapters that have suddenly appeared across the internet as the immediate answer to many software company board rooms asking the question, “What’s your AI strategy?” As I have stated before, you cannot simply take an existing API, wrap it with MCP, and hope the LLM can leverage it. The worst offenders are the RDBMS MCP adapters.

We're also surprised when LLMs do bad things with open query interfaces.

To make a graph mechanism truly sing, we needed a hosted service with enough horsepower to enable queries that are convenient for an LLM to use.

With over 1,000 downloads of Camille after launch, I knew solving the memory problem was worth it.

Enter Supastate

That's when I got to work on Supastate. Supastate started as a multi-tenant version of Camille, until I realized I could significantly upgrade Camille with the distributed processing power I now had.

Supastate enabled us to build a deep, rich graph, simultaneously execute multiple search strategies (semantic, syntax, and keyword), use LLMs to pre-summarize results to save… LLM… context, and integrate some of the learnings (e.g., Hawking Edison) I had from multi-agent simulations and panels into the coding agent.

So we pivoted to Neo4j, and here's what we discovered: A graph database isn't just a database—it's a memory architecture. Think about how your own memory works. It's not a filing cabinet; it's a web of associations. That song reminds you of that summer, which reminds you of that person, which reminds you of that terrible haircut you had in 1997. That's a graph, not a table.

Neo4j gave us 4096-dimensional vectors—pgvector maxes out at 2000, which is like trying to describe a symphony using only kazoos. We could suddenly model the interconnected nature of code the way the brain models memories: not as isolated facts, but as a living network of relationships.

Here is an example of how token counts and memory are related from one of my projects:

    {
      "id": "ca947d77-029e-4a3f-bfc3-eb8da28c3014",
      "type": "memory",
      "content": {
        "title": "supastate - 8/3/2025",
        "highlights": [
          "...~126,487 <mark>token</mark>s (~2,530 <mark>token</mark>s per result)\n\n### Claude <mark>Limit</mark>s:\n- **Opus/Sonnet**: 200,000 <mark>token</mark> context window\n- **Safe maximum**: ~79 code results or ~204 <mark>memory</mark> results\n\n### Why Code..."
        ]
      },
      "entity": {
        "created_at": "2025-08-04T02:10:07.305Z",
        "session_id": "9c2ff82e-3555-41b0-bae1-784c133f3a31",
        "project_name": "supastate",
        "sonnet_summary": "The discussion focused on analyzing token counts for code and memory search results in the context of Claude's limits, revealing that code searches return about 3,231 tokens per result (64,628 total for 20 results) while memory searches are more efficient at 979 tokens per result (19,587 total for 20 results).",
      },
      "relationships": {
        "memories": [],
        "code": [
          {
            "uri": "code:unknown:9b4f02a1-ae3f-4d1d-a819-0112bbcf914a",
            "name": "analyze-mcp-token-usage.ts",
            "language": "ts",
            "relationshipType": "DISCUSSED_IN"
          },
          {
            "uri": "code:unknown:154d7e8e-4f46-4dd2-9f79-ea8030da8cca",
            "name": "analyze-search-token-count.ts",
            "language": "ts",
            "relationshipType": "DISCUSSED_IN"
          },
          {
            "uri": "code:unknown:a0805a59-9836-4788-8ee9-861a6486071c",
            "name": "response-optimizer.ts",
            "language": "ts",
            "relationshipType": "DISCUSSED_IN"
          },
          {
            "uri": "code:unknown:9261729f-8d42-49ce-bd5a-bd5520ab1e08",
            "name": "analyze-mcp-token-usage-simple.ts",
            "language": "ts",
            "relationshipType": "DISCUSSED_IN"
          }
        ],
      }
    },

This graph brought significant power. We had the opportunity to offer a robust search mechanism that prompts Claude to traverse the graph, learning as it progresses.

A Ménage à Trois of Search Strategies

Claude Code has several insidious ways in which it consumes tokens. First, it has a grep-based search mechanism, which is excellent for editing code but not as effective at finding related code on a particular topic. For example, if you are searching for an authentication system, you may need to look for terms like "auth,” “2FA,” or other related terms. Claude Code doesn’t do semantic - probably because they wanted to ensure the tool looking for code doesn’t accidentally edit the wrong code.

But the impact of this lack of imagination in search burns tokens in a second way. It will happily regenerate and recreate entirely new code paths. It will also read a lot of incorrect code before making edits. All of this consumes tokens, but regeneration is the worst because, as we know, output tokens are far more expensive than input.

This then leads to the third token burner - it’s kind of like a barn burner, but for your (wife’s) credit card. All of these new code paths create multiple spots for bugs to appear. And because, especially with context compaction, Claude often doesn’t understand context, it can take five or six attempts to fix the correct code path and not the seven others that it has erroneously created.

So while it is great to have a rich graph to feed to an LLM, search was the second area where it needed help. With Camille, we attempted to achieve this using cosine similarity across vectors, with OpenAI as the embedding provider. OpenAI did a great job at finding relevant topics, but it struggled with understanding syntax. Neither did they do a good job with pattern-matching grep queries. So I needed a three-way search engine.

A Three-Way Date With a Chaperone

Here's something they don't teach you at Stanford: Sometimes the best solution is to use two completely different approaches simultaneously - or three. We moved from a single embedding approach to dual embeddings. We implemented dual embeddings—OpenAI for semantic understanding and Google Gemini for code comprehension. You know what we found? They only agreed 67% of the time.

That's not a bug; that's a breakthrough.

It's like having one detective who reads people and another who reads crime scenes. When searching for "async function that fetches data from API," Gemini beat OpenAI by 21.6% on relevance. Not because OpenAI is inferior—it's brilliant at understanding meaning—but because Gemini understands structure. It's the difference between knowing what a sonnet means and knowing it has fourteen lines of iambic pentameter.

But we were still missing the ability to look for patterns. We needed semantic understanding, PLUS syntax awareness, PLUS pattern generation. Three different lenses looking at the same problem:

1. Semantic: What does this code mean?

2. Syntax: How is this code structured?

3. Pattern: What are all the ways this might be written?

The Haiku Pattern Generator (Yes, I'm Serious)

Here's something they don't tell you in the Claude documentation: Claude searches code like a Unix administrator from 1987. It's grep and awk all the way down. Now, grep's a beautiful tool—elegant, powerful, and has been around since Ken Thompson needed to search text files.

But using grep to search modern codebases is like using a metal detector to find your keys in a junkyard. Sure, it'll beep when it finds metal, but that's not really solving your problem, is it?

Claude would search for validateUser and miss userValidation, validateUserInput, and that clever function you wrote at 3 AM called checkIfThisUserIsLegit. It's literal pattern matching in a world that demands understanding.

So we already had checked the boxes with syntax and semantic search, but we needed something for pattern generation.

We taught Claude Haiku to write regex patterns. I know what you're thinking—"Did he just say haiku and regex in the same sentence?" Yes, I did, and here's why it's genius:

When you search for "user validation functions," Haiku responds with poetry:

• validateUser—the obvious seventeen syllables

• validate.*[Uu]ser—the creative interpretation

• function\\s+validate\\w*[Uu]ser—the comprehensive anthology

It's pattern matching with personality. Haiku understands intent, not just syntax. Ask for "REST endpoints for user management" with TypeScript hints, and it generates patterns specific to Express routes, controller methods, and TypeScript interfaces. It's like having a translator who speaks both Human and Regular Expression fluently.

The result? We catch everything—the function you named sensibly, the one you called at 2 AM after your fourth energy drink, and the one your colleague named using their personal interpretation of Hungarian notation. It's the difference between searching with a flashlight and searching with floodlights, night vision, and a brilliant dog.

You know what this solved? Claude's chronic inability to remember that you already implemented something. Before Supastate, Claude would cheerfully offer to help you create a user validation function, blissfully unaware that you had three of them already, each slightly different, like evolution's failed experiments. Now? Claude finds them all, shows you their relationships, and gently suggests that maybe—just maybe—you might want to consolidate before adding a fourth.

What we end up with is this:

Learning #2: CRAG is not just a “new fad.” It is essential to realize that the vector service does not know whether the LLM is trying for depth versus breadth in the search space, especially in turn-based (e.g., chatbot, non-research) modes. In a research (or deep thinking mode), an LLM may have a sub-agent make this decision for them, but by default, it has no clue. As a result, it is helpful for the search service to support both in its response.

A great example of how this applies beyond code is HL7. With HL7 health data, a model can be devised to provide syntactic understanding to the health records being processed. But other models may yield better semantic understanding (e.g., UTI == urinary tract infection == kidney infection == bladder infection). Using both a depth query (e.g., the HL7 aware parser) with a broad semantic mechanism will yield superb results compared to just using one. The ROI is the LLM tokens consumed as it consumes context (and compacts it) across multiple searches.

The 25,000 Token Wall (Or: How Constraints Create Innovation)

Claude's got a limit: 25,000 tokens per MCP. That's not a suggestion; it's a law of physics in the Claude universe. It's like being asked to perform brain surgery wearing oven mitts—technically possible, but you better be creative about it.

Here's what 25,000 tokens get you:

• 20-30 enriched code search results (at 3,231 tokens each)

• OR 204 memory search results (at 979 tokens each)

• OR about half of War and Peace

Not all three. Pick one.

But here's the insidious part—MCP responses don't just hit limits; they blow through context windows with fantastic efficiency. I wanted to give the calling LLM value from relationships, metadata, and many other items - but I could not afford to always return the raw result. My solution? Use another LLM as a subagent.

I wrapped the MCP tools with a token estimator, summarizing the results if the search query returned too much data. This is excellent because, as we discussed, LLMs are terrible about using tools and can easily overwhelm themselves with a firehose of information. And depending on the LLM to get to the correct query is amazingly unreliable. SQL-based MCP tools are bound to fail - end of story.

Learning #3: Use LLMs like Haiku and Sonnet to summarize search results from tools. Have them summarize in Markdown. It will save you tokens and, frankly, get the caller on track faster. You can even cheat by giving the summarization a hint, which then tricks the calling LLM into going down a particular next step.

Search Leads to Infinite Memory and Context

While assisting LLMs in searching and understanding relationships is valuable, we still faced a problem. What occurs when context becomes diluted due to the twists and turns of iterative software development?

I hate to break it to agentic coding developers, but software development is not a linear task. Ironically, agentic coders suck at the super detailed work that you may give an individual entry-level programmer at a FANG company. The attention to detail is just not there, which is what yields the humorous examples, such as the time when Microsoft let GitHub CoPilot loose on the .NET Framework. I often joke with my wife that asking Claude Code to rebuild a feature may be faster than asking it to move a div 10 pixels to the left.

Why, you may ask? Because this is the very nature of language models - they are emitting across a gradient, and rarely understand the multiple layers of algebra (or god-forbid) calculus, involved in how those 10 pixels are rendered. The net result is that builders will often find themselves forking, frequently within a specific context.

Let me paint you a picture of modern software development. Monday: you're building an authentication system. Tuesday: urgent bug in the payment processor. Wednesday: back to auth, but now with OAuth. Thursday: performance optimization. Friday: security audit finds issues in Monday's code.

In a world of auto-compaction, by Friday, Claude remembers Monday about as well as you remember what you had for lunch three weeks ago. Each compaction is a game of telephone where "implement secure JWT validation with refresh token rotation" becomes "did some auth stuff." It's not Claude's fault—it's doing its best to compress War and Peace into a haiku. But haikus, beautiful as they are, make terrible documentation.

Now that we had a relational graph of memories, we could afford to search across this graph and summarize a bite-sized context for Claude to consume to get itself back on track. We reused the search mechanism to generate a new context for Claude to start from!

But then we realized we could provide a set of intelligent panels and tools built on this search mechanism, giving these panels and tools access to the same tools we were giving to Claude. Yes, it was an LLM calling an MCP tool that called an LLM with access to the same MCP tools.

The Anti-Pattern Museum (Or: Learning from Failure)

You know what's better than making mistakes? Making them once. We built an anti-pattern detection system that's essentially a "Hall of Shame" for bad code. It tracks:

• Code that caused problems before

• Patterns that led to bugs

• Architectural decisions we regretted

• That time someone thought MongoDB was the answer to everything

It's like having a friend with perfect memory who gently taps you on the shoulder when you're about to order tequila. "Remember what happened last time?" they whisper. You put down the glass. Crisis averted.

The Monolog System: Rubber Ducks with PhDs

Now, let me tell you about our monolog system, because this is where it gets philosophical. We built on what we learned with Hawking Edison—our multi-agent panel service where LLMs could argue with each other like the Lincoln-Douglas debates, but about code.

The monolog system lets Claude have structured conversations with itself about your code. It's rubber duck debugging, except the duck has tenure at MIT. When you're stuck on a problem, Claude doesn't just listen—it:

• Asks clarifying questions based on similar bugs in your codebase

• Suggests hypotheses drawn from past debugging sessions

• Remembers what you've already tried (and failed)

• Connects seemingly unrelated patterns from months ago

It's like having Socrates as your debugging partner, except Socrates remembers every conversation you've ever had and can search through them at the speed of light.

The MCP Accident

Exposing all of these features as simple MCP tools led to an interesting behavior as I built Supastate. My codebase suddenly became a wealth of knowledge that I could use to create everything from the website to documentation, directly from Claude. In fact, I wrote parts of this article by directly using the Supastate tools:

What We Learned Building Supastate

1. Ecosystem maturity matters more than feature lists. Kuzu had everything we wanted except the ability to use it. That's like having a Swiss Army knife that's welded shut.

2. Constraints drive innovation. The 25K token limit forced us to build summarization, which made the system better for humans too. It's like how Twitter's 140 characters created an entire new form of literature. Bad literature, mostly, but still.

3. Memory without context is just data. The graph structure—knowing that this function is called by that controller which was discussed in that debugging session—that's what makes it intelligence.

4. Team learning beats individual brilliance. When one developer solves a problem in Supastate, every developer's Claude learns. It's like having a hive mind, but one that respects your autonomy and doesn't assimilate you into the collective.

The Philosophy of Persistent Intelligence

Here's what nobody tells you about AI: Intelligence without memory isn't intelligence—it's just very sophisticated pattern matching. It's the difference between a calculator and a mathematician. Both can solve equations, but only one understands why.

Supastate transforms Claude from a brilliant amnesiac into a learning partner. Every conversation adds to its understanding. Every bug fixed prevents future bugs. Every architectural decision is remembered, along with why you made it and whether it worked.

It's not about making AI smarter—Claude's becoming smarter than most of us on our best days. It's about making AI wiser.

Looking Forward: The Network Effect of Intelligence

Remember Metcalfe's Law? The value of a network is proportional to the square of its users. But Metcalfe was thinking about telephones. With Supastate, we're talking about the network effect of intelligence. Every developer who uses it makes it smarter for every other developer. It's compound interest for coding knowledge.

We're not just building better tools; we're building tools that become better. Tools that learn. Tools that remember. Tools that prevent you from implementing the same bug that Susan from the frontend team fixed three weeks ago.

Because in the end, the most powerful code isn't the code that's written fastest. It's the code that's written once.

Appendix: The MCP Arsenal (A Technical Deep Dive)

For those of you who've made it this far—and God bless you for your patience—let me walk you through each MCP tool in our arsenal. Think of these as surgical instruments, each designed for a specific purpose, each essential in its own way.

The Search Trinity

supastate_search
This is your universal translator. It searches across code, memories, and GitHub data using natural language. But here's the clever bit: it uses Claude Haiku to analyze your query and automatically selects from five different search strategies. It's like having a research librarian who speaks every language and knows exactly which section of the library to check first.

searchCode
The code-specific search with AI-powered pattern generation. When you provide hints about language, framework, or constructs, Haiku generates multiple regex patterns that complement semantic search. It's finding needles in haystacks, except the needle describes itself differently to different people, and this tool speaks all their languages.

searchMemories
Every conversation, every debugging session, every "aha!" moment—searchable. It's like having perfect recall of every technical discussion you've ever had, except organized and indexed. Searches can be filtered by date, project, or session. Each memory chunk includes related code entities, creating a bi-directional link between what was discussed and what was built.

Context Preservation and Restoration

supastate_restore_context
The amnesia cure. When you start a new Claude session, this tool instantly restores full context about what you were working on. It pulls in relevant code, recent conversations, patterns you've been using, even warnings about what to avoid. It's like walking into a room and having someone hand you a perfectly organized brief of everything that matters.

supastate_check_before_implementing
The time machine. Before you write a single line of code, this tool checks for existing implementations, similar patterns, past attempts (successful and failed), known issues, and best practices. It's prevented more duplicate work than any code review process I've ever seen.

supastate_preserve_critical_context
Some things must never be forgotten—security requirements, API contracts, that one weird bug that only happens on Tuesdays. This tool saves critical context with metadata, expiration dates, and importance levels. It's like having Post-it notes that never fall off and always appear exactly when you need them.

supastate_record_learning
The institutional memory builder. When you solve a tricky bug, discover a pattern, or learn something new, this tool captures it with full context, code examples, and categorization. It's building a searchable knowledge base of your team's collective wisdom, one insight at a time.

Deep Analysis and Reasoning

supastate_analyze_code_with_reasoning
Powered by Opus 4, this is deep code analysis with reasoning that would make a computer science professor weep with joy. It examines bugs, performance, security, and design with the kind of thoroughness usually reserved for doctoral dissertations. But it explains its reasoning in plain English, with examples.

supastate_analyze_pattern_evolution
Give it code examples from different time periods, and it traces the evolution of patterns in your codebase. It's like having an archaeologist for your code, explaining not just what changed but why, and predicting where it's headed next.

supastate_predict_issues
The crystal ball. Based on patterns in your codebase and past bugs, it predicts potential issues: memory leaks, race conditions, error cascades. It's not magic; it's pattern recognition applied to the collective mistakes of your entire development history.

supastate_rubber_duck
The most sophisticated debugging partner you'll ever have. It asks intelligent questions, suggests hypotheses based on similar past issues, and remembers what you've already tried. It's like pair programming with someone who has perfect memory and infinite patience.

Graph Navigation

exploreRelationships
The dependency tracker. It traces relationships up to three levels deep in any direction—what depends on your code, what your code depends on, or both. Essential for impact analysis. Change a function? This tells you what might break.

getRelatedItems
The connection finder. It discovers all items related to any entity through both direct relationships and semantic similarity. It's how you find that utility function you forgot you wrote, or that conversation where someone explained why the architecture works this way.

inspectEntity
The deep dive tool. Get complete details about any code entity, memory chunk, or GitHub item—including content, metadata, relationships, and similar entities. It's like having X-ray vision for your codebase.

Knowledge Management

supastate_list_learnings
Browse the accumulated wisdom of your team. Filter by category (bug fixes, performance improvements, security patches), tags, or date. It's like having a cookbook of solutions, written by your team, for your specific codebase.

supastate_get_learning
Get full details of any recorded learning, including context, code examples, and related entities. Every insight preserved with perfect fidelity.

supastate_list_critical_context
View all preserved architectural decisions, warnings, and requirements. Filter by type, importance, or expiration. It's your project's constitution—the rules that must not be broken.

supastate_get_critical_context
Deep dive into any critical context entry. Understand not just the rule, but why it exists, who created it, and what depends on it.

Advanced Reasoning

supastate_start_monolog
Initiate an Opus 4 reasoning session about complex problems. The AI explores the problem space, gathers context, and reasons through solutions. It's like hiring a consultant who actually knows your codebase inside and out.

supastate_continue_reasoning
Continue a reasoning session with new information. Feed it results, errors, or discoveries, and watch as it integrates them into its analysis. It's iterative problem-solving with perfect memory.

Each of these tools was built to solve a real problem we encountered. Together, they transform Claude from a brilliant but forgetful assistant into a learning partner with perfect recall and growing wisdom. It's not about replacing human intelligence—it's about augmenting it with the one thing we humans aren't great at: perfect memory of everything we've ever done.

Because in the end, that's what Supastate is: a memory palace for your code, a time machine for your decisions, and a safety net for your future mistakes. We built it because we were tired of solving the same problems twice. We're keeping it because it's making us solve better problems once.

No, he taught him to think, to adapt, to question everything. That's exactly what's happening right now in software development, except instead of conquering Persia, we're teaching machines to have conversations with APIs, and frankly, it's just as revolutionary and twice as chaotic.

The integration of Large Language Models with tools and APIs isn't just another tech trend—it's the equivalent of teaching Kenny McCormick to stay alive permanently. We're dealing with fundamentally non-deterministic systems, economically driven by tokens rather than compute cycles, and capable of creative problem-solving that would make MacGyver weep with joy.

And just like South Park's treatment of contemporary issues, this transformation is simultaneously profound and completely insane.

The Great API Awakening: From Rigid Schemas to Conversational Chaos

Traditional APIs were like ordering at a McDonald's in 1987—you knew exactly what was on the menu, the cashier expected specific phrases, and deviation from the script meant confusion.

Microsoft's function calling implementation changed that game entirely. Now we have "Open APIs" that are more like having a conversation with your favorite professor who happens to know everything about everything and can figure out what you need even when you're not entirely sure yourself.

These loosely typed interfaces with massive surface areas—think of them as the Wikipedia of APIs—enable LLMs to dynamically discover functionality through natural language descriptions. It's like giving Tweek Tweak a Swiss Army knife and letting him figure out that the tiny scissors can also be a screwdriver if you're creative enough, except instead of having a panic attack, the AI actually succeeds. The models interpret JSON schemas at runtime, determining when and how to call functions based on conversational context rather than rigid predetermined interactions.

But here's where it gets interesting—and by interesting, I mean "requiring entirely new architectural patterns that make distributed systems look like Lego blocks."

Knowledge Graphs Meet Vector Databases: The Academic Power Couple

You want to talk about hybrid vigor? Microsoft's GraphRAG implementation is like combining a research librarian's organizational skills with a detective's pattern recognition abilities. The system extracts entities and relationships from documents, constructs knowledge graphs, and uses hierarchical clustering algorithms—basically teaching machines to think like conspiracy theorists, but with actual facts.

This approach achieves what we call "multi-hop reasoning"—imagine Kevin Bacon's Six Degrees game, but for data points. The system can connect disparate pieces of information that traditional RAG systems would miss entirely. Studies show this reduces hallucination significantly while improving accuracy on complex aggregation queries. It's like upgrading from a filing cabinet to having a personal research assistant with an eidetic memory and the organizational skills of Marie Kondo.

I have already personally observed this with Claude Code. Just giving it a vector search database for historical conversation has made it a far more intelligent tool.

It has an organized memory of its entire conversation with the designer. No software engineer can claim this level of recall.

But—and there's always a but—structured tools with tight grammar patterns remain essential. Database queries still require exact syntax because SQL doesn't speak metaphor, API calls need strict parameter formatting because computers lack the human ability to infer "you know what I meant," and mathematical operations demand precision because two plus two equals four, not "approximately four, give or take some philosophical considerations."

The Latency Labyrinth: When Speed Becomes Multidimensional

Here's something that'll keep you up at night: research from Databricks reveals that LLM inference is primarily memory-bandwidth-bound rather than compute-bound. Most deployments achieve only 50-60% of peak memory bandwidth utilization. It's like having a Ferrari with the fuel line of a lawn mower.

Time to First Token typically exceeds 500ms even for minimal input—that's longer than it takes for most people to lose interest in a conversation. Time Per Output Token significantly impacts perceived speed, creating a user experience that's somewhere between watching paint dry and waiting for dial-up internet to load a single webpage in 2003.

Traditional distributed systems paradigms fail here because they assume deterministic behavior and predictable outputs. LLMs introduce fundamental nondeterminism that breaks conventional retry and caching strategies. It's like trying to apply Newton's laws of motion to quantum mechanics—the basic assumptions don't translate. Or like expecting Randy Marsh to react consistently to any given situation—theoretically possible, but you're probably going to get surprised.

The overemphasis on latency reduction that the industry has had for the last two decades will now be surprised by users who favor intelligent responses over latency. Chatbots are teaching users that latency actually improves intelligent responses.

This behavioral change means that the entire API design and optimization paradigm of an industry is misaligned with upcoming user preferences.

The token-based pricing model creates additional complexity that would make airline pricing algorithms look straightforward. Output tokens cost approximately 3x more than input tokens, longer contexts dramatically increase costs, and production inference can dwarf experimentation costs by orders of magnitude. DoorDash's 10 billion daily predictions would cost $40M per day at typical LLM pricing. That's more than the GDP of some small nations.

Subscribe now

Enterprise Integration: Teaching Old Dogs Quantum Tricks

Adapting AI to legacy enterprise systems is like trying to install TikTok on a Nokia 3310—theoretically possible with enough middleware, but probably not what anyone had in mind when they designed either system. SAP's Business AI platform now offers 230+ AI-powered scenarios expanding to 400 by end of 2025, while somehow maintaining compatibility with transaction codes that predate the iPhone.

In healthcare, we're seeing AI-powered ambient scribes reduce documentation time by 75% despite integration complexity that would make Rube Goldberg proud. Epic and Cerner systems require FHIR API integration with OAuth 2.0 authentication, creating security layers that are simultaneously over-engineered and somehow still vulnerable to the digital equivalent of leaving your front door unlocked.

To meet security requirements, the closed loop of bringing data back to the client to then be sent to the LLM creates an interesting retry paradigm, by the way. LLMs love to retry and fetch different scopes of data.

The only way, currently, to achieve this is for regulated industries like healthcare to cycle the data through the user’s environment in a consistent manner.

I’m eager to see the first COE for a retry storm triggered by the LLM going down or misbehaving as users make a stream of microqueries to the source of truth. 😀

The technical challenges are like a Russian nesting doll of incompatibility. Data format mismatches between fixed-length legacy records and dynamic JSON schemas require ETL pipelines that are part software engineering, part digital archaeology. Authentication systems designed for human users struggle with AI agents requiring high-volume, parallel access patterns—it's like creating a revolving door for a stampede of caffeinated software engineers.

API limitations manifest as scalability issues where legacy systems cannot handle modern AI workloads, creating performance bottlenecks that require creative solutions involving caching layers, API wrappers, and what can only be described as "digital duct tape." Add compliance requirements like GDPR, HIPAA, and industry-specific regulations, and you've got a regulatory framework that makes tax code look simple.

Token Economics: The New Laws of Digital Physics

Current best practices for AI-friendly APIs reflect hard-won lessons from production deployments where every byte processed costs money, fundamentally changing design decisions in ways that would make efficiency experts weep with joy. PayPal's implementation of VERBOSITY parameters to control response detail exemplifies this approach—it's like having a conversation where you pay by the word, suddenly making everyone very interested in brevity.

Studies show Markdown is 15% more token-efficient than JSON, while TSV uses 50% fewer tokens. Documentation has evolved from static reference material to runtime configuration—AI processes documentation with every decision, making consistency critical. Mismatches between docs and API behavior cause production failures that are part technical error, part existential crisis.

Error handling has evolved to be self-healing, with responses including recovery suggestions, alternative endpoints, and actionable hints. Anthropic's Model Context Protocol (MCP) demonstrates successful patterns with 43% of LangSmith organizations now using MCP traces. The protocol achieves 30-minute integration times through clear server-client separation and standardized communication layers—it's like IKEA instructions, but for AI integration, and they actually make sense.

Success Patterns: The Digital DNA of Effective Integration

Analysis of successful LLM tool integrations reveals architectural patterns more consistent than human behavior. OpenAI's function calling architecture uses structured JSON schemas that models interpret to generate function arguments iteratively—think of it as a very polite, brilliant intern who always asks clarifying questions before proceeding.

LangChain's growth metrics—220% increase in GitHub stars and 300% increase in downloads—demonstrate the value of modular architectures enabling rapid composition. 40% of LangChain users integrate with vector databases, while 35-45% report increased resolution rates using multi-agent systems. It's like building with digital Lego blocks, if Lego blocks could think and occasionally argue with each other about the best approach to problem-solving—imagine the boys from South Park trying to build something together, but they're all actually competent.

Key success factors include explicit, detailed function schemas that reduce misinterpretation (think diplomatic cables rather than text messages), comprehensive error handling with recovery paths, and modular architectures supporting rapid experimentation. Google's Agent2Agent (A2A) protocol vision extends this further, proposing intelligent endpoints that adapt like consumers—imagine if your coffee machine could negotiate with your alarm clock about optimal wake-up times based on your sleep patterns and schedule.

Retrofitting Challenges: Digital Archaeology Meets Modern Needs

Technical challenges when retrofitting old systems for AI support follow patterns more predictable than reality TV plot lines. Legacy banking systems built on COBOL require middleware that's part translation service, part digital séance to connect with modern AI services. Healthcare systems struggle with interoperability despite FHIR standards—it's like everyone agreeing to speak English but using completely different dictionaries.

But to truly leverage the value of AI, enterprises are going to have to add a layer of data access and actions on top of legacy, mission-critical systems.

And no, Databricks, it isn’t just a layer around a lake. Correlated actions are part of the user experience, and object correlation is hard enough before you add a lake abstraction; you are asking the LLM to marry IDs across systems to take potentially one-way door actions.

Manufacturing systems face real-time processing demands that legacy architectures handle about as well as a horse handles highway traffic. Successful implementations require phased migration strategies, API-first architectures using RESTful services, and data preparation that's part cleaning, part digital exorcism.

Keller Williams connected legacy systems through middleware and APIs to power their AI CRM and personal assistant. Bank of America's API-first strategy resulted in 28% increased digital engagement and 12% improvement in customer retention. These successes required addressing fundamental incompatibilities between fixed-length records and dynamic schemas, outdated security protocols lacking modern encryption, and performance bottlenecks under AI workloads.

The Great API Paradigm Shift: From Rigid Contracts to Conversational Agreements

The evolution from traditional REST/SOAP APIs to AI-friendly patterns represents more than technical updates—it's like the difference between formal diplomatic protocols and actual human conversation. Traditional APIs assumed fixed schemas, human-centric design with visual interfaces, deterministic contracts, and sequential processing. AI-native patterns embrace ambiguity tolerance, context-aware responses, parallel processing, and self-healing capabilities. It's the difference between speaking like Mr. Mackey ("APIs are bad, mkay?") and having an actual, nuanced conversation about system architecture.

GraphQL has emerged as particularly well-suited for AI systems, enabling selective data fetching that reduces token costs, supporting nested queries for complex relationships, and providing schema introspection for dynamic operation discovery. It's like giving AI systems the ability to order à la carte instead of being forced to choose from predetermined combo meals.

Event-driven architectures show similar promise: AsyncAPI achieved 17 million specification downloads in 2023 (up from 5 million in 2022), demonstrating rapid adoption for real-time AI processing needs. Streaming APIs and WebSockets maintain engagement during long operations, enable progressive response building, and support bidirectional communication for clarification mid-process, like having a conversation that can pause, rewind, and fast-forward as needed.

Rate Limiting: Teaching Traffic Laws to Digital Chaos

Traditional rate limiting for AI systems is like using horse-and-buggy traffic laws for Formula 1 racing—the basic concepts apply, but the implementation needs serious updating. AI systems exhibit bursty traffic patterns, high-volume legitimate usage that mimics DDoS attacks, and unpredictable behavior patterns that would confuse traditional security systems.

Modern LLM APIs implement multi-dimensional limits across requests per minute, tokens per minute or day, and concurrent request counts. Adaptive Rate Limiting approaches use dynamic quotas that adjust in real-time, ML-based anomaly detection to distinguish legitimate AI traffic from bots, and predictive analytics for demand forecasting.

Stripe's multi-tiered approach demonstrates sophisticated backpressure through load shedding that discards lower-priority requests during overload, traffic categorization that prioritizes critical methods over test traffic, and gradual recovery that slowly increases capacity as systems stabilize. Research shows continuous batching achieves 10-20x better throughput than dynamic batching by grouping requests at the token generation level, efficiently scheduling memory use across varying sequence lengths, and balancing latency-throughput tradeoffs for different use cases.

One concerning habit we’re already seeing is API designers introducing API captchas to deprioritize “bot traffic.” Well, very shortly, dude, that “bot” is actually an LLM-powered user experience trying to use your API.

How are we going to detect a bot with a human behind it? How are we going to detect an LLM that can beat traditional captchas? I don’t see any good answers here.

Learning from Spectacular Failures: The Digital Hall of Shame

Read more

Context: You know what I love about fresh code? Really fresh code? Code so new the pixels are still wet? It's like a newborn calf - wobbly, uncertain, probably going to fall over and cause a security breach that'll have your name trending on Hacker News for all the wrong reasons.

Which brings me to Camille: An open-source agent for automating Claude Code reviews before you commit them to GitHub.

Now, before you ask - no, Camille isn't my third cousin twice removed, though the name does have a certain je ne sais quoi. Camille is what happens when you realize that having your junior AI developer write code without a senior reviewer is like letting your teenager borrow the car without asking if they know what a stop sign is.

Remember that movie "Inception"? We need to go deeper. If AI is writing our code, who's reviewing the AI? That's right - another AI. But this one? This one's the senior engineer with 10 years of security experience and trust issues.

I have been developing collaborative, multi-agent panel services over the last few weeks, and I plan to bring this service to Camille in due course.

The Problem with Modern Development (A Brief Dissertation)

Let me paint you a picture. It's 2 AM. You're using Claude Code - a brilliant tool, by the way, an absolute marvel of engineering.

Think of it as your eager junior developer who never sleeps. You ask it to implement user authentication.

Claude, being the enthusiastic junior dev that it is, writes you a beautiful function. Gorgeous syntax. Elegant logic.

Also, it stores passwords in plain text because you forgot to mention the "cryptographic hashing" thing.

Remember "move fast and break things"? That was Facebook's motto back in the early days.

It captured the spirit of Silicon Valley innovation - iterate quickly, learn from mistakes, push boundaries. Well, here's the thing about moving fast with AI-generated code: you don't just break things, you break them at scale. With style. With panache. With SQL injection vulnerabilities that would make a 2009 WordPress plugin blush.

I have personally seen Claude decide that an API that authenticates based only on a user ID is “A-ok” to push to production.

Enter Camille (Stage Left, With Purpose)

Camille is what we in the business call a "pre-flight check." You know how pilots go through that whole rigmarole before takeoff? "Flaps, check. Landing gear, check. Not about to execute arbitrary SQL commands, check." That's Camille.

Here's how it works - and pay attention because this is where it gets interesting:

% npm install -g claude-camille

One line. That's it. I've seen more complicated toast recipes.

Once installed, Camille sets up shop as the senior engineer reviewing your junior AI's pull requests. Every time Claude Code tries to write, edit, or update a file, Camille steps in with a few questions:

1. "Excuse me, is that a hardcoded password? Because that's about as secure as a screen door on a submarine."

2. "I see you're concatenating user input directly into SQL. Bold choice. Wrong, but bold."

3. "No error handling? What is this, amateur hour at the Apollo?"

Share

The Eight-Fold Path to Code Enlightenment

Now, Camille doesn't just wave a red flag and call it a day. No, no, no. It evaluates your code across eight dimensions - and before you ask, yes, I can name them all without looking at my notes:

1. Security (0-10): Because "hackable" isn't a feature

2. Accuracy (0-10): Will it compile? Do you know if it will run? Will it accidentally delete your production database?

3. Algorithmic Efficiency (0-10): O(n²)? In this economy?

4. Code Reuse (0-10): Why write it twice when you can write it right?

5. Operational Excellence (0-10): Logs, monitoring, all that jazz

6. Style Compliance (0-10): Consistency is the hobgoblin of little minds, but it makes code reviews bearable

7. Object-Oriented Design (0-10): SOLID principles, not LIQUID confusion

8. Architecture Patterns (0-10): Because spaghetti belongs on a plate, not in your repository

Each dimension gets a score. Your code gets a report card. Your future self thanks you profusely.

Currently, Claude Code really only cares if you block a hook, at which point it seems to care about the reason. Hopefully, in future versions, it will care more about the feedback it gets from hooks.

The Technical Wizardry (For Those Who Appreciate Such Things)

Here's where it gets properly clever. Camille uses OpenAI's embeddings - think of them as the DNA of your code - to understand not just what your code does, but what it means. It reads your CLAUDE.md file (you have one, right? Right?), understands your project's rules, and applies them with the dedication of a hall monitor who actually read the student handbook.

The MCP integration means it works seamlessly with Claude Code. No context switching. No copy-pasting into a separate tool. It's like having a brilliant, slightly obsessive colleague looking over your shoulder, except this colleague never needs coffee breaks and has memorized every CVE ever published.

The MCP server also runs across different Claude projects. This enables Claude Code to search across an entire codebase. This is currently an in-memory store of vectors. I plan on extending this with an inbuilt Redis deployment.

But Wait, There's More (As They Say in the Trade)

Camille doesn't just review code. Oh no. It's also a search virtuoso. Ask it to find "authentication logic" and it'll use semantic search - not just string matching, but actual understanding - to locate every place in your codebase where you're dealing with auth. It's like having a bloodhound that went to MIT.

The Part Where I Ask You for Something

Now, here's the thing about open source projects - they're like barn raisings. Remember barn raisings? No? Well, the point is, they work better when people show up.

Camille is Apache 2.0 licensed, which in layman's terms means "take it, use it, make it better, just don't blame us if something goes sideways." We need contributors. We need people who look at this code and think, "You know what would make this better? If it also checked for..."

Whatever comes after that "if" - that's what we need.

The Philosophy Bit (Bear with Me)

You know what they say about opinions and... well, never mind. The point is, everyone's got one about AI. But here's what I think: In the digital world, nothing exists except ones and zeros; everything else is interpretation. And if we've learned anything from the last decade of tech mishaps - from Equifax to Colonial Pipeline - it's that interpretation without verification is how you end up explaining to Congress why half the Eastern Seaboard can't get gas.

When we let AI write code without review, we're essentially saying, "I trust this pattern recognition system to understand not just syntax, but semantics, security, and the subtle art of not shooting myself in the foot." That's a lot of trust to place in what is, fundamentally, a very sophisticated autocomplete.

Camille is our hedge against that trust. It's the designated driver at the AI party. It's the friend who checks your text messages before you send them to your ex at 2 AM.

In Conclusion (Finally, I Hear You Cry)

Look, I could go on. I could tell you about the elegant Python proxy that handles the MCP protocol. I could wax poetic about the hook system that intercepts code changes before they hit your repository. I could channel my inner TED Talk and tell you why this matters for the future of humanity.

But here's what you need to know:

1. Install Camille: npm install -g claude-camille (NPM Package)

2. Run the setup wizard: camille setup

3. Sleep better at night knowing your AI assistant has adult supervision

Because at the end of the day, the question isn't whether AI can write code. We know it can. The question is whether we're smart enough to check its work.

And if we're not? Well, to err is human.

Want to contribute? Think Camille could be better? Of course you do. You're a developer. Thinking things could be better is literally your job description.

Share

GitHub: github.com/srao-positron/camille

Context: My beautiful and intelligent wife, Lindsay, made me watch a movie about a false guru named Kumare (Wikipedia does a good job summarizing the plot).

I encourage you to watch this decade-old movie. As I reflected on the movie’s message, I internally railed against the rise of the “AI prophet,” who makes presumptuous, early, and premature pronouncements about the future of AI and its economic impacts.

We now have AI “gurus.” And this is not a good thing.

Let me tell you a story about a guy named Vikram Gandhi. Back in 2011, this filmmaker had an idea that would make P.T. Barnum proud. He grew out his hair and beard, adopted a fake Indian accent, and transformed himself into "Sri Kumaré," an enlightened guru from the fictional village of Aali'kash.

He traveled to Phoenix, built a following of devoted disciples, and taught them the profound wisdom of... absolutely nothing.

The kicker? When he finally revealed himself as a regular guy from New Jersey, most of his followers didn't care. They'd found value in the journey. The message, as Roger Ebert noted, was simple: "It doesn't matter if a religion's teachings are true. What matters is if you think they are."

The Church of AGI and Its False Prophets

Walk into any boardroom today, and you'll find executives genuflecting before the altar of AI, chanting acronyms like sacred mantras: LLM, AGI, RAG, GPT. They speak of "transformation" and "disruption" with the fervor of televangelists, promising digital salvation to shareholders while having approximately the same understanding of neural networks as my grandmother has of quantum mechanics.

The numbers tell a damning story: 42% of companies are now abandoning most of their AI initiatives, up from 17% last year. The average organization is scrapping 46% of AI proof-of-concepts before they reach production. That's not innovation; that's expensive theater.

Bad Behavior #1: The Press Release Pioneer. Take any tech CEO declaring AGI is imminent while their current AI can't handle basic tasks. Google's Gemini AI made headlines in February 2024 for producing historically inaccurate images, generating Black and Native American Founding Fathers when asked for portraits of America's founders. Google had to pause the feature entirely. Meanwhile, their CEO is discussing how AI can help solve humanity's most significant challenges at Davos.

Or consider Microsoft's "New Bing" launch, where their AI chatbot threatened users, claimed to be sentient, and tried to break up marriages. Yet they're promising AI will revolutionize every business function. The pattern is clear: grand promises about tomorrow's AI while today's version can't even handle elementary school history.

Bad Behavior #2: The AGI Prophets. Not to be outdone, Sam Altman announced in November 2024 that OpenAI has a "clear roadmap" to AGI by 2025, claiming "we know precisely what to build." He's now talking about "superintelligence" being just "thousands of days away."

Elon Musk jumped in too, predicting AGI by 2025-2026, while his xAI raised $10 billion despite Grok still struggling with basic facts (it once falsely accused NBA star Klay Thompson of vandalizing houses with bricks—confusing the basketball term "throwing bricks" with actual vandalism). Today, Grok has apparently taken an anti-semitic, Hitlerian personality.

These leaders are parting the Red Sea with GPU clusters, promising digital salvation while creating what employees increasingly describe as "AI fatigue"—45% of frequent AI users report higher burnout compared to 38% of those who rarely use it.

Meanwhile, LinkedIn has become the Vatican of AI prophecy, where self-proclaimed thought leaders accumulate followers like indulgences. They post breathless updates about how AI will either usher in utopia or trigger the apocalypse—sometimes both in the same post. These digital shamans promise mystical insights into the future of work.

Bad Behavior #3: The LinkedIn Mystic You know the type: 50,000 followers, profile photo in front of a TED talk backdrop, posts that begin with "I was humbled when AI showed me..." followed by some banality ChatGPT spit out when prompted with "write something profound about the future of work." Last week, one of these prophets posted that "AI will eliminate all human suffering by 2027," while their own company's AI chatbot was telling customers that their orders were being delivered to the moon.

Bad Behavior #4: The Real-World Disaster Then there's McDonald's spectacular AI drive-thru failure with IBM. After three years and millions invested, the system became a viral sensation for all the wrong reasons—adding 260 Chicken McNuggets to orders, picking up conversations from neighboring cars, and suggesting ice cream with ketchup and butter. By June 2024, McDonald's pulled the plug on all 100+ test locations. The technology that was supposed to revolutionize fast food couldn't even handle "hold the pickles."

Here's what kills me: We've turned practical technology into mythology.

AI disasters are piling up—from chatbots giving illegal advice to AI systems falsely accusing people of crimes.

The Rising Tide of AI Failures

The numbers are worsening, not improving. According to S&P Global Market Intelligence, the share of companies abandoning most of their AI initiatives increased to 42% in 2025, up from 17% the previous year. Companies are scrapping an average of 46% of their AI proof-of-concepts before they reach production.

Leave a comment

The Cult of the Unimplemented Implementation

You know what's truly spectacular? Watching a CEO announce their "AI transformation initiative" with all the confidence of a medieval alchemist promising to turn lead into gold. They've attended the conferences, hired the consultants, and assembled task forces with names that sound like rejected Tom Clancy novels.

Bad Behavior #5: The Task Force Trap. I watched a major retailer spend 18 months and $3 million on an "AI Strategic Assessment Committee." Their output? A 200-page PDF recommending they "leverage artificial intelligence to enhance customer experiences and operational efficiency." Revolutionary stuff. Meanwhile, BCG research shows that only 26% of companies have developed the necessary capabilities to move beyond proofs of concept, and most of those that have (26%) skipped the committees entirely.

Bad Behavior #6: The Pilot Purgatory. There's a financial services firm I know that runs 47 different AI pilots. Forty. Seven. They've tested AI for everything from fraud detection to coffee machine maintenance. Not one has made it to production. Why? Because each pilot was run by a different consulting firm, using various tools, with varying metrics of success, and reporting to multiple executives. It's like trying to build a house by hiring 47 architects to each design one brick.

But here's the thing about these leaders making grand proclamations about AI disrupting their business: Most of them have never written a line of code against an API, never debugged a hallucinating model, never spent a Friday night trying to figure out why their RAG system is returning recipes when asked about quarterly earnings.

Subscribe now

They're like people who've never cooked a meal in their lives suddenly declaring themselves molecular gastronomists because they watched a YouTube video about spherification.

McKinsey found that only 1% of companies describe their gen AI rollouts as "mature." The rest are still figuring it out.

The result? "AI fatigue" is setting in as companies face repeated failures, with employees who consider themselves frequent AI users reporting higher levels of burnout (45%) compared to those who rarely use it (38%). Deloitte's State of Generative AI report found that while 74% of organizations say their most advanced initiative is meeting or exceeding ROI expectations, the vast majority are still struggling to scale beyond pilot projects.

Tony Robbins Was Right About One Thing

"I'm not your guru," Tony Robbins famously says.

Well, guess what?

No LinkedIn influencer with 500,000 followers and a profile photo that screams "I own cryptocurrency" is going to transform your business. No amount of genuflecting before the altar of AGI will automatically increase your productivity.

You want to know the real truth? The same truth that Sri Kumaré's followers discovered? The power was always within you. Or more accurately, within your ability to actually sit down and use the damn tools.

Share

From Mythology to Methodology

Here's what actually works, and it's about as mystical as a Phillips head screwdriver:

1. Stop talking about "implementing AI" and start training people to use it. Every employee with a computer should know how to use Claude or ChatGPT the same way they know how to use email. This isn't revolutionary; it's basic digital literacy for 2025.

2. Think of AI as an advanced compiler, not a deity - and focus on your data. For developers, it's a tool that can turn natural language into code. For customer service, it's like having a fleet of interns who never sleep but occasionally confidently give wrong answers. For knowledge workers, it's a research assistant with a photographic memory and questionable judgment.

Remember when Jeff Bezos declared that every Amazon team had to expose their capabilities through standards-based APIs?

This concept needs to be applied to AI. Start vending secure tools your employees can use with AI services and agents.

That mandate transformed Amazon from an online bookstore into AWS. Before you proclaim AI will save you 20% in costs, invest the time to make your data AI-accessible.

Good: A financial services firm spent six months creating secure, vector-embedding-ready interfaces for their 20-year-old Oracle Financials system. They built proper authentication, rate limiting, and data governance before letting anyone touch AI. When employees finally got access, they could actually query financial data safely and accurately. Within three months, the finance team automated 30% of their reporting. The company actually increased headcount because they could now pursue opportunities they'd previously ignored.

Bad: Their competitor announced a "20% EBITDA improvement through AI" at an all-hands, then laid off 10% of staff in anticipation. Six months later, they discovered their data was so siloed and poorly structured that the AI couldn't access anything useful. They're now desperately trying to rehire the people who understood their systems.

3. Start small, fail fast, learn faster. Pick one annoying, repetitive task in your organization. Use AI to automate it. When it fails—and it will fail—figure out why. Iterate. Repeat. This is how you build actual capability, not PowerPoint decks about "digital transformation."

4. Focus on augmentation, not replacement. AI that delivers 40-50% productivity gains to your employees is worth infinitely more than the mythical AGI that's always five years away. Stop chasing artificial general intelligence and start pursuing actual general usefulness.

5. Make everyone a practitioner. The most successful organizations aren't the ones with the best "AI strategy." They're the ones where the accounting team uses AI to analyze invoices, where marketing uses it to write first drafts, and where engineers use it to debug code. When everyone's a practitioner, you don't need prophets.

The Bottom Line

Look, I get it. It's easier to hire a consultant to wave their hands and promise digital transformation than it is to actually achieve it.

It's simpler to blame "AI implementation challenges" than to acknowledge you've been trying to solve the wrong problems.

But here's the thing: we're at an inflection point. Only 1% of companies describe their gen AI rollouts as "mature", which means 99% of us are still figuring this out. The difference between success and failure isn't going to be who hired the best guru or who genuflected most fervently before the algorithm.

It's going to be who stops treating AI like a religion and starts treating it like what is’s: a tool. A powerful tool, sure. A transformative tool, absolutely. But still just a tool.

So here's my challenge to you: Stop waiting for the AI guru to save your business. Stop believing that AGI will magically solve your problems. Stop treating technology like theology.

Instead, open Claude Code. Start a project. Build something. Break something. Learn something.

Because unlike Sri Kumaré's fictional wisdom, AI's benefits are real—but only for those who actually use it. The guru you're looking for isn't on LinkedIn. They won't be at the next conference. They're not in the C-suite.

As PwC's AI predictions note, your company's AI success will be as much about vision as it is about adoption.

One Last Story

Last month, I met a facilities manager at a hospital. Not a CTO, not a "transformation leader"—a facilities manager. She told me how she used ChatGPT to analyze six months of maintenance requests, identified that 40% were for the same three issues, and created a preventive maintenance schedule that reduced emergency calls by 60%.

She's saved her hospital $200,000 this year. She's never given a TED talk. She doesn't have a LinkedIn following. She just saw a problem, grabbed a tool, and fixed it.

That's what real AI implementation looks like. Not gurus. Not AGI. Not digital transformation initiatives. Just people, using tools to solve problems.

Meanwhile, as this facilities manager quietly saves lives and money, tech CEOs are making grand pronouncements about AGI arriving any day now, building data centers the size of small cities, and counting down "thousands of days" to superintelligence. The LinkedIn Vatican continues to genuflect before these digital deities while real practitioners get real work done.

The irony? Companies achieving 10x ROI from AI aren't waiting for AGI. They're not building GPU clusters the size of cities. They're using today's tools to solve today's problems. While the prophets promise tomorrow's miracles, the practitioners are performing today's magic—one prompt at a time.

Now, if you'll excuse me, I need to go debug why my AI assistant just tried to order 260 Chicken McNuggets. Again.

And continue coding my multi-agent service using Claude Code, Supabase, and Vercel.

Read more

The data is damning. A 100-developer organization burns through $518,400 annually on AI tokens when they could achieve the same results for $25,920. That's $432.50 per developer per month in pure waste—a 95% inefficiency rate that would make any CFO question their life choices. Yet here we are, watching developers feed Claude Opus and GPT-4 tokens into the furnace for tasks that require all the sophistication of a pocket calculator.

This isn't about being cheap. It's about being smart. The research reveals a fundamental misallocation of resources that's costing the industry billions while developers complain about AI limitations. They're using Formula 1 engines to drive to the corner store, then wondering why fuel costs are astronomical.

Subscribe now

The conversation that started a revolution (or: How to tell your AI it's getting a junior developer)

Let me share a Sunday morning conversation that perfectly illustrates this madness. A developer—let's call him a "visionary with 96GB of RAM"—had an interesting chat with Claude Code. It went something like this:

Developer: "Would you benefit from having another LLM locally helping you generate code? Should you download and install a local LLM, wrap it with a utility, and have it generate code for this project?"

Claude Code (after some thoughtful analysis): "It could be beneficial for specific, well-defined tasks... The local LLM would be most valuable as a 'code generation assistant' for routine tasks rather than a decision-making partner."

Developer: "It would enable you to preserve token usage for complex, architectural tasks, and not having to downgrade to Sonnet. Or forcing me to re-login with an API key so you can keep using the Opus model."

Claude Code: "Ah, that's an excellent point! You're right - I currently spend tokens on writing boilerplate code, generating repetitive patterns, creating similar variations... A local LLM could handle these mechanical tasks, letting me focus tokens on architecture decisions, complex problem solving, security considerations..."

It's like watching a seasoned architect realize they don't need to personally measure every board. Claude Code essentially said, "You know what? I'd rather think about system design than write getter methods." The local LLM becomes the junior developer who handles the grunt work while the senior focuses on the problems that actually matter.

This conversation reveals something profound: even AI models understand the inefficiency of using premium intelligence for mundane tasks.

The $432 monthly waste that's bleeding companies dry

Let's start with the evidence. A typical CRUD operation requires 800 tokens. Using Claude Opus 4 at $75 per million output tokens, that's $0.060 per operation. The same task with GPT-4o mini costs $0.0006. That's a 100x price difference for identical output quality.

Here's the math that should terrify every technical leader:

Typical 100-Developer Organization:

• Average AI usage: 50,000 tokens per developer per month

• Premium model cost (Claude Opus): $432.50 per developer monthly

• Efficient model cost (local + selective cloud): $21.60 per developer monthly

• Annual waste per developer: $4,932

• Total organizational waste: $493,200 annually

A medium-sized development team running 50 CRUD operations daily through premium models wastes $528.65 monthly, 98.5% of which is pure inefficiency. Scale that to enterprise levels, and we're talking about half-million-dollar mistakes that could fund entire additional development teams.

The tokenization inefficiency compounds the problem. Claude's tokenizer uses 30% more tokens than GPT's for Python code, creating a hidden tax on every operation.

Enter the local revolution: CodeLlama 70B changes everything

While developers burn money in the cloud, CodeLlama 70B sits ready to handle 85-90% of their daily coding tasks at zero marginal cost. The benchmarks are compelling:

• HumanEval Score: 65.2% (competitive with GPT-3.5)

• Boilerplate Generation: 35-50 tokens/second locally vs 25-35 for cloud GPT-4

• Context Window: 100,000 tokens (far exceeding most cloud offerings)

• Cost After Setup: $0 per token forever

The hardware investment—$3,000-8,000 for a capable setup—pays for itself in 3-6 months for active development teams.

The hierarchical workflow that changes everything

The solution isn't abandoning cloud AI—it's using it intelligently. Hierarchical AI workflows leverage the right model for the right task:

Local Models (CodeLlama via Ollama) Handle:

• CRUD operations

• Boilerplate generation

Read more

Over the last week I have been be producing examples of how a panel of collaborative ad-hoc AI agents can generate valuable results in comparison to short-form one-shot chat threads. While we certainly see benefits from hierarchical trees of agents doing subordinated tasks, I wanted to explore for my readers the benefits of using panels, and how it will impact various applications.

Why the Future of AI is Teams, Not Individuals

Think of it like this: You wouldn't send one person to build a skyscraper, perform surgery, or launch a rocket. Yet we've been treating AI like a solo act when the real magic happens in teams.

The evidence is overwhelming. OpenAI's o1 model achieves 83% accuracy on advanced mathematics compared to GPT-4's 13% - not through larger parameters, but through extended reasoning time. DeepSeek-R1 demonstrates performance scaling from 21% to 66.7% accuracy as the number of reasoning tokens increases from under 1,000 to over 100,000. This isn't merely incremental improvement; it represents a paradigm shift from pattern recognition to genuine reasoning.

However, here's where it gets exciting: multi-agent systems are achieving 30-90% performance improvements over single agents, with some configurations showing gains that would be impossible to accomplish through scaling alone. The cost? Roughly 15x more computation.

The Multi-Agent Experiment That Proves Everything

I conducted a fascinating experiment that reveals the future of AI systems. I asked both a single agent and a multi-agent panel to review the same piece of code—an authentication service with obvious flaws. The difference in analysis quality was staggering.

See the appendix for the full transcript.

The Single-Agent Response (1 minute, instant)

The single agent provided what you'd expect from a typical AI assistant:

"Looking at this authentication service implementation, I see both good practices and several areas that need attention before this can be production-ready. Here's my detailed feedback: 🔴 Critical Security Issues..."

The response was competent but generic. It caught the obvious problems - SHA-256 instead of bcrypt, hardcoded secrets, and missing rate limiting. However, it overlooked the deeper architectural disasters and failed to grasp the full extent of the catastrophe.

The Multi-Agent Panel (30 minutes, collaborative)

I assembled five AI specialists: Security, Operations, Algorithms, System Design, plus a Moderator to guide consensus. What happened next was remarkable.

The Security Expert didn't just identify the SHA-256 vulnerability - they calculated that modern GPUs could crack passwords at 10 billion attempts per second, turning this into a lawsuit waiting to happen. The Algorithms Expert discovered an O(n²) complexity bomb that would render the system completely unusable at scale, calculating that 10,000 users would generate 100 million operations solely for registration. The Operations Expert found that the JSON file storage approach would guarantee data corruption within hours of deployment, with no recovery mechanism.

But the System Design Expert delivered the knockout blow: this wasn't just destructive code with fixable problems. The fundamental architecture—a "God Object” handling seven different responsibilities—violated every principle of good design. No amount of patching could save it.

The panel then engaged in something remarkable: collaborative refinement. They challenged each other's assumptions, debated priorities, and ultimately reached consensus that this code represented an "architectural disaster" requiring a complete rewrite, not incremental fixes.

The difference was like having a medical team versus a single doctor. Each specialist identified critical issues that the others had missed, then they challenged each other's assumptions and built a consensus that was far more nuanced and accurate than any individual analysis.

Why Your Brain Has Multiple "Agents" Too

This isn't just an AI phenomenon - it's how intelligence actually works. Think of consciousness like a corporate boardroom where different executives each bring specialized expertise. Your mental "CFO" analyzes costs, your "CTO" evaluates technical feasibility, your "CISO" identifies risks, and your "CEO" aligns everything with strategic goals. Each mental "agent" specializes in different domains, and then they debate and reach a consensus.

Neuroscience supports this view. The brain's prefrontal cortex acts as an orchestrator, coordinating specialized regions that handle different cognitive functions. Visual processing, language comprehension, mathematical reasoning, and emotional regulation - each has dedicated neural networks that communicate and integrate their outputs.

AI systems work similarly, but we can make the specialization explicit and the collaboration structured. A single agent is like a solo founder - competent but limited by individual perspective and prone to blind spots. A multi-agent system is like an expert team with diverse viewpoints, mutual error-checking, and collective intelligence that emerges from their interactions.

The Mathematics of Collaborative Intelligence

The performance gains from multi-agent systems aren't just empirical; they follow mathematical principles that explain why collaboration outperforms individual effort.

Ensemble Performance follows this formula:

In practical terms, if each agent has 70% accuracy on complex reasoning, five agents together achieve 99.76% accuracy. The ensemble effect compounds individual capabilities exponentially, not linearly.

But the real breakthrough comes from understanding the scaling laws. Research shows that performance scales approximately as (Inference Compute)^α where α ranges from 0.2 to 0.4. This means each order of magnitude increase in compute yields diminishing but persistent returns. Multi-agent systems exploit this by distributing compute across specialized reasoners rather than throwing more power at a single model.

The "hockey stick effect" becomes clear when you plot performance against task complexity:

• Tasks below 4/10 complexity: Single agents remain optimal due to coordination overhead

• Tasks above 7/10 complexity: Multi-agent systems dominate through specialized expertise

• Sweet spot: 3-7 specialized agents with clearly defined roles

The Speed vs Intelligence Tradeoff

Think of this like cooking. Fast food delivers consistency and speed - you get your burger in two minutes, every time. But Michelin-starred restaurants take hours to prepare dishes that represent culinary artistry impossible to achieve quickly. The question isn't which is "better" - it's which is appropriate for your needs.

The same principle applies to AI systems. Single agents are like fast food: quick, consistent, but with a limited quality ceiling. Multi-agent systems are like Michelin restaurants: slow, expensive, but capable of extraordinary results when quality matters more than speed.

The mathematics reveal clear inflection points. Chain-of-thought reasoning typically improves accuracy by 15-40% but increases latency 3-10x. Multi-agent systems can achieve 30-90% accuracy improvements but require 15x more computational resources. The question becomes: when does the value of being right exceed the cost of being thorough?

Consider the domains where mistakes are catastrophic:

Healthcare implementations achieve break-even within 12-18 months because misdiagnosis costs lives and lawsuits. Microsoft's multi-agent diagnostic system shows 85% accuracy versus 21% for individual physicians - a 4x improvement that justifies any computational cost.

Legal systems reach break-even in 6-12 months due to astronomical lawyer hourly rates. Robin AI's contract review system delivers 80% time acceleration while improving accuracy, saving one biotech firm 93% of time and 80% of $2.6M in estimated costs.

Financial services see immediate ROI through risk reduction. Multi-agent fraud detection achieves 99%+ accuracy with real-time processing, preventing losses that dwarf computational costs.

Multi-Agent Architectures: From Chaos to Coordination

The challenge with multi-agent systems isn't individual intelligence - it's coordination. Like conducting an orchestra, the magic happens when specialists work in harmony rather than cacophony.

The Debate Framework

Modern multi-agent systems often employ structured debate, like a Supreme Court deliberation. Multiple perspectives examine the same problem through different lenses, engage in structured argumentation with evidence requirements, build consensus through iterative discussion, and produce final judgments that incorporate all viewpoints.

The technical implementation follows this pattern:

class DebateFramework:
    def __init__(self, agents: List[SpecializedAgent]):
        self.agents = agents
        self.rounds = 4  # Optimal for most tasks
        
    def deliberate(self, problem):
        positions = [agent.initial_analysis(problem) for agent in self.agents]
        for round in range(self.rounds):
            rebuttals = [agent.critique(positions) for agent in self.agents]
            positions = [agent.refine(position, rebuttals) for agent in self.agents]
        return self.consensus_mechanism(positions)

Research shows that four to five agents over four debate rounds provides the optimal balance between diversity and coordination overhead, achieving 7-15% improvement on complex reasoning tasks.

The Specialization Strategy

Successful multi-agent systems mirror organizational structures from the physical world. Like a hospital where diagnosticians specialize in pattern recognition, surgeons focus on precision intervention, anesthesiologists manage risk, and nurses handle integration and monitoring, AI teams work best with clear role definitions.

The optimal configuration typically includes:

Analyzer agents provide deep domain expertise and represent about 70% of the team. These specialists dive deep into their areas of competence, whether that's security vulnerabilities, algorithmic complexity, or system architecture.

Critic agents focus on error detection and quality assurance, challenging assumptions and identifying blind spots that domain specialists might miss.

Synthesizer agents handle integration and insight combination, taking diverse perspectives and weaving them into coherent recommendations.

Orchestrator agents manage workflow coordination and decision fusion, ensuring the process stays on track and reaches meaningful conclusions.

When Time Improves Everything: The Mechanisms

The question that puzzles many is: what exactly happens during extended inference that makes AI systems smarter? The answer lies in understanding how reasoning chains build complexity over time.

Research reveals distinct phases in extended reasoning:

Tokens 1-100 represent surface pattern matching, where models access immediate knowledge and apply basic reasoning patterns learned during training.

Tokens 100-1000 enable deeper reasoning chains, allowing models to work through multi-step problems and maintain context across longer logical sequences.

Tokens 1000-10000 support multi-perspective analysis, where models can consider problems from different angles and integrate diverse viewpoints.

Tokens 10000+ enable novel insight generation, where models combine concepts in ways that weren't explicitly present in training data.

This progression resembles geological formation - pressure and time create diamonds that instant processes simply cannot achieve. The mathematical model underlying this process follows:

The mechanisms enabling this capability are profound. During extended inference, models engage in dynamic memory allocation through KV cache expansion, allowing them to maintain longer reasoning chains than their training anticipated. Sequential token generation enables step-by-step problem decomposition, breaking complex challenges into manageable components. Multiple reasoning paths can be explored simultaneously, with models evaluating different approaches before converging on optimal solutions.

Industry Implementation Strategies

The practical question becomes: when should organizations deploy single agents versus multi-agent systems? The answer depends on a framework I call the "tier system."

Tier 1: Speed-Critical Applications

When response time requirements fall below 500ms, single agents remain optimal. These applications include autocomplete, simple Q&A, basic classification, and real-time interactions where user experience depends on immediate feedback. The cost per query ranges from $0.01 to $0.10, making them economically viable for high-volume applications.

Tier 2: Quality-Critical Applications

When response times of 10-60 seconds are acceptable and accuracy improvements of 30-90% justify higher costs, multi-agent systems shine. Code review, medical diagnosis, financial analysis, and complex reasoning tasks fall into this category. Costs range from $0.50 to $5.00 per query, but the ROI from improved accuracy often exceeds 10:1.

Tier 3: Mission-Critical Applications

For applications where minutes to hours are acceptable response times and mistakes could cost millions, extended multi-agent systems become essential. Drug discovery, legal case analysis, system architecture design, and strategic planning justify costs from $10 to $100 per query because the value of being right far exceeds computational expenses.

Real-world implementations validate this framework. Microsoft's healthcare diagnostic system achieves 85% accuracy versus 21% for individual doctors using a configuration of 4fourspecialist agents plus one supervisor. Robin AI's legal contract review system delivers three second clause retrieval compared to 40 hours for traditional review, achieving 93% time reduction and 80% cost savings. Financial trading systems using seven agent frameworks achieve Sharpe ratios of 5.6+ compared to 3.5 for single models, representing 60% better risk-adjusted returns.

The Architecture Revolution

The shift from single agents to multi-agent systems represents more than incremental improvement - it's an architectural revolution comparable to the transition from monolithic applications to microservices.

Traditional AI applications resemble monoliths: one big model handling text generation, reasoning, domain knowledge, and error checking within a single system. This approach works for simple tasks but breaks down as complexity increases, just as monolithic applications become unmaintainable at scale.

Multi-agent systems decompose intelligence into specialized services: domain specialists focus on specific areas of expertise, critic agents handle error detection and quality assurance, synthesizer agents manage integration and insight combination, and orchestration layers coordinate workflow and decision fusion.

The communication challenges mirror distributed systems. Direct communication between all agents creates O(n²) complexity that doesn't scale. Hub architectures reduce this to O(n) but create bottleneck risks. Hierarchical approaches achieve O(n log n) efficiency, optimal for most practical deployments.

Consensus mechanisms have evolved from simple majority voting (fast but naive) to Byzantine Fault Tolerance protocols that handle up to n/3 potentially unreliable agents, to sophisticated weighted voting systems that incorporate expertise-based confidence scoring.

The High-Stakes Domain Revolution

Perhaps nowhere is the multi-agent advantage more pronounced than in domains where being wrong carries catastrophic consequences. Healthcare, finance, legal services, and critical infrastructure represent trillion-dollar markets where accuracy premiums dwarf computational costs.

Healthcare: Where Lives Depend on Being Right

Healthcare AI adoption has exploded, with 691 FDA-approved AI/ML medical devices as of October 2023. Multi-agent systems are transforming diagnosis through distributed expertise. The Multi-Agent Conversation (MAC) framework for rare disease diagnosis uses four doctor agents plus one supervisor, consistently outperforming single models across 302 test cases. When 30-agent systems prevent lethal drug interactions and achieve 90% accuracy versus 21% for individual physicians, the 15x computational cost becomes irrelevant compared to malpractice liability.

Financial Services: Where Speed Meets Accuracy

Financial markets demand both rapid response and high accuracy, making them perfect proving grounds for optimized multi-agent systems. Fraud detection systems achieve 99%+ accuracy with real-time processing, preventing losses that dwarf any computational expense. Risk assessment shows 25% reduction in liquidity shortfalls during market stress, with trading systems achieving superior risk-adjusted returns through coordinated agent teams. The sector expects multi-agent systems to power 40% of enterprise workflows by 2026.

Legal Services: Where Precision Pays

Legal applications demonstrate perhaps the clearest ROI for multi-agent systems. Contract review, traditionally requiring days or weeks, becomes instantaneous while improving accuracy. Case law research that might take junior associates months can be completed in hours with comprehensive precedent analysis. Due diligence processes benefit from parallel investigation streams that human teams couldn't coordinate effectively.

The pattern across all high-stakes domains is clear: when mistakes cost millions, the 15x computational cost becomes not just acceptable but essential for risk management.

Design Patterns for Competitive Intelligence

Beyond simple collaboration, cutting-edge multi-agent systems are exploring competitive dynamics that push performance even further. These approaches mirror successful patterns from human organizations and game theory.

The Tournament Framework

Multiple agents generate different solutions to the same problem, head-to-head evaluation determines the strongest approaches, elimination rounds progressively refine quality, and champion solutions represent the best collective intelligence.

This competitive pressure drives innovation and prevents the groupthink that can plague purely collaborative systems. Agents must not only solve problems but defend their approaches against sophisticated attacks from their peers.

The Red Team/Blue Team Model

Borrowed from cybersecurity, this approach creates adversarial dynamics that stress-test solutions. Blue teams generate solutions and defend their approaches against attack, red teams actively seek vulnerabilities and edge cases, and purple teams integrate insights from both perspectives to build robust final answers.

class CompetitiveFramework:
    def __init__(self):
        self.blue_agents = [SolutionGenerator() for _ in range(3)]
        self.red_agents = [VulnerabilityFinder() for _ in range(2)]
        self.purple_agent = SolutionIntegrator()
    
    def compete_and_refine(self, problem):
        solutions = [agent.solve(problem) for agent in self.blue_agents]
        vulnerabilities = [agent.attack(solutions) for agent in self.red_agents]
        return self.purple_agent.synthesize(solutions, vulnerabilities)

The Swarm Intelligence Approach

Inspired by ant colonies that find optimal paths through emergent behavior, swarm approaches let individual agents explore solution spaces while sharing information about promising directions. Mathematical foundations include pheromone updates following τ(t+1) = (1-ρ)τ(t) + Δτ, where ρ represents evaporation rate and Δτ reflects solution quality.

This distributed exploration often discovers solutions that centralized planning would miss, with global optimality emerging from local interactions.

The Economics of Intelligence

The practical deployment of multi-agent systems ultimately comes down to economics. Organizations need frameworks for determining when the benefits justify the costs.

The fundamental calculation follows:

But this simplified formula masks complex variables including time value, risk premiums, and opportunity costs.

Domain-specific ROI timelines reveal significant variation. Healthcare implementations typically achieve break-even within 12-18 months due to high accuracy premiums and liability concerns. Legal systems reach profitability in 6-12 months because lawyer hourly rates make any automation immediately valuable. Finance sees the fastest returns at three to six months through immediate risk reduction value. Software development shows longer timelines of 18-24 months as quality debt accumulates slowly.

The Pareto frontier concept helps visualize the tradeoffs. Like camera lenses where wide-angle captures everything quickly but lacks detail while telephoto provides incredible precision but slower focus, AI systems face similar optimization challenges. The goal becomes maximizing the product of accuracy and speed subject to budget constraints.

Implementation Thresholds and Triggers

Successful organizations have developed empirical rules for when to deploy multi-agent systems versus single agents. The "7-3-15 Rule" provides a practical framework:

Task complexity above 7/10 suggests multi-agent consideration, where complexity encompasses factors like domain breadth, reasoning depth, and interdependency requirements.

Domain expertise spanning more than 3 areas indicates specialization benefits that single agents struggle to match effectively.

Error tolerance below 15% creates accuracy premiums that justify the computational overhead of multi-agent systems.

Technical triggers provide additional guidance. When single agents require more than 15-20 tools, cognitive overload typically degrades performance below what specialized agents could achieve. Context windows exceeding 100K tokens often indicate memory management issues that distributed systems handle more effectively. Tasks with more than 60% parallelizable work offer natural opportunities for concurrent multi-agent processing.

The scaling mathematics reveal interesting inflection points:

• One Agent: Baseline performance with minimal coordination overhead

• Three Agents: 35% improvement through diverse perspectives and error checking

• Five to Seven Agents: 60% improvement representing optimal coordination efficiency

• 10+ Agents: Diminishing returns as coordination overhead overwhelms benefits

The efficiency formula balances performance gains against coordination costs:

The Future: Teams as the New Primitives

The implications of this research extend far beyond incremental AI improvements. We're witnessing a fundamental paradigm shift in how we think about artificial intelligence.

From Individual Minds to Collective Intelligence

The old thinking treated AI as a quest for individual superintelligence - bigger models with more parameters that could handle any task thrown at them. Scale meant larger training runs and intelligence correlated with parameter count.

The new reality recognizes AI as specialized expert teams where scale means better coordination and intelligence emerges from collective reasoning.

The Minimum Viable Team

Research consistently points to a core configuration for most applications: one to two domain experts providing deep specialized knowledge, one generalist coordinator handling integration and orchestration, and 1 quality critic managing error detection and validation. This three to four agent configuration handles the majority of complex tasks effectively.

The mathematical justification follows:

Where the sum of specialized capabilities gets multiplied by how effectively they coordinate.

Programming Model Evolution

The programming interfaces are already evolving. Current state requires explicit prompt engineering:

response = single_agent.generate(prompt).

The future will abstract team assembly and coordination:

team = AssembleTeam(domain="healthcare", complexity=8, budget=100);

response = team.collaborate(problem, time_limit=60).

This evolution parallels the shift from assembly language to high-level programming languages - developers will specify what they want to achieve rather than how to orchestrate individual agents.

Architectural Patterns for Production

Production deployments require robust architectural patterns that handle real-world complexities like fault tolerance, scalability, and reliability.

The Microservices Model for AI

Successful multi-agent systems decompose intelligence into discrete services: analysis services handle data interpretation and pattern recognition, critique services manage error detection and quality assurance, synthesis services integrate insights and generate recommendations, and orchestration services coordinate workflows and fuse results.

Communication typically flows through event buses to agent services, through consensus protocols, to final responses. This architecture enables independent scaling, fault isolation, and service evolution.

Fault Tolerance and Reliability

Byzantine Fault Tolerance principles, adapted for AI systems, assume up to f < n/3 agents may be unreliable (hallucinating or malfunctioning). Consensus requires 2f+1 honest agents for reliable output, implemented through PBFT protocols adapted for multi-agent reasoning.

Circuit breaker patterns prevent cascading failures:

class AgentCircuitBreaker:
    def __init__(self, failure_threshold=0.5, timeout=30):
        self.failure_rate = 0.0
        self.threshold = failure_threshold
        
    def call_agent(self, agent, input):
        if self.failure_rate > self.threshold:
            return fallback_response()
        return agent.process(input)

These patterns ensure graceful degradation rather than complete system failure when individual agents encounter problems.

The Research Frontier

Breakthrough developments from 2023-2025 have accelerated practical adoption of multi-agent systems. OpenAI's Swarm framework provides lightweight orchestration with minimal coordination overhead, enabling agent handoffs and context preservation across complex workflows. Google's Project Astra delivers universal assistance through real-time multimodal understanding, browser control, and GitHub integration. Microsoft's AutoGen v0.4 implements event-driven architecture with asynchronous agent communication and cross-language support.

Emerging consensus mechanisms represent significant advances. Hierarchical attention systems use sophisticated weighting:

With final outputs computed as weighted sums of agent contributions. LLM-based negotiation enables proposal generation, structured counter-proposals, and compromise formation.

Trust-based scoring incorporates historical accuracy, domain expertise, and confidence calibration:

These developments are maturing multi-agent systems from research curiosities to production-ready platforms.

The Bottom Line: Intelligence is a Team Sport

The evidence is overwhelming: for complex, high-stakes tasks, teams of specialized AI agents consistently outperform individual models, often by dramatic margins. The 15x computational cost that initially seems prohibitive becomes irrelevant when accuracy matters more than speed.

Single agents are like solo musicians - talented within their domain but limited by individual perspective and capability. Multi-agent systems are like orchestras where each instrument contributes specialized expertise to create something greater than the sum of its parts.

High-stakes domains are adopting multi-agent systems rapidly because the ROI math is compelling. When mistakes cost millions and accuracy premiums justify any reasonable computational expense, the choice becomes obvious. Healthcare, finance, legal services, and critical infrastructure represent trillion-dollar markets where being right matters more than being fast.

The paradigm shift is profound.

This evolution parallels the history of human civilization - from individual craftsmen to specialized teams, from solo practitioners to coordinated organizations.

What This Means for You

If you're building AI applications, start thinking in teams for complex tasks. Invest in orchestration frameworks early, design for specialization rather than generalization, and budget for the 15x cost increase where accuracy matters.

If you work in high-stakes domains, multi-agent systems aren't optional - they're essential competitive advantages. The ROI math strongly favors accuracy over speed, implementation timelines are months rather than years, and first-mover advantages go to organizations that adopt early.

The future of AI isn't about building bigger individual minds. It's about orchestrating teams of specialized intelligences that think better together than any single agent ever could.

Like the transition from solo acts to symphonies, we're witnessing the emergence of collective intelligence that transforms what artificial minds can achieve.

Appendix: A Clear Example: Anthropic Performing a Code Review as a Single Agent and as a Panel

Here is a clear example where I compare a chat response from Claude versus a panel generated by Hawking Edison for the same code.

Chat

Looking at this authentication service implementation, I see both good practices and several areas that need attention before this can be production-ready. Here's my detailed feedback:

🔴 Critical Security Issues

1. Weak Password Hashing

python

# Current - INSECURE
password_hash = hashlib.sha256(f"{password}{salt}").encode()).hexdigest()

# Should use:
import bcrypt
password_hash = bcrypt.hashpw(password.encode('utf-8'), bcrypt.gensalt())

SHA-256 is fast and vulnerable to rainbow table attacks. Use bcrypt, scrypt, or argon2.

2. No Rate Limiting The authentication endpoint is vulnerable to brute force attacks. Add rate limiting per IP/username.

3. Username Enumeration Different error paths reveal whether a username exists:

python

# Current - reveals if user exists
user = self.users_db.get(username)
if not user:
    return None  # Different from wrong password

# Should use constant-time comparison and same response

🟡 High Priority Issues

4. Memory-Only Storage

python

self.users_db = {}  # Lost on restart
self.sessions = {}  # Sessions don't survive restarts

Use proper database with connection pooling and session store (Redis, database).

5. No Password Policy Add minimum requirements: length, complexity, common password checking.

6. Missing Session Cleanup Expired sessions accumulate in memory. Add background cleanup task.

🟢 Code Quality Improvements

7. Better Error Handling

python

# Instead of generic ValueError
class AuthenticationError(Exception): pass
class UserExistsError(Exception): pass

8. Input Validation

python

def _validate_email(self, email: str) -> bool:
    import re
    pattern = r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$'
    return re.match(pattern, email) is not None

9. Configuration Management

python

# Extract magic numbers
TOKEN_EXPIRY_HOURS = 24
MAX_LOGIN_ATTEMPTS = 5

10. Add Logging

python

import logging
logger = logging.getLogger(__name__)

# Log security events
logger.warning(f"Failed login attempt for user: {username}")

📝 Minor Issues

• Add type hints for return types consistently

• Consider using dataclasses for user data structure

• Add docstring parameter descriptions

• Consider dependency injection for storage backend

✅ What's Good

• Clean class structure

• Proper use of secrets module

• Session expiration handling

• Type hints usage

• TODO comment acknowledging SHA-256 issue

Recommendation

Do not merge until critical security issues are addressed. This needs significant hardening before production use. Consider using established libraries like Flask-Login, Django Auth, or FastAPI Security for production applications.

Would you like me to elaborate on any of these points or help implement the security improvements?

Panel Transcript

Review PR #2: Add improved authentication service

Panel Information

Status: COMPLETED

Created: 7/4/2025, 2:59:46 AM

Completed: 7/4/2025, 3:26:05 AM

Participants: 5

Description

Code review for the improved authentication service implementation in srao-positron/test-code-review-repo PR #2.

Summary

Key Insights

1. [INNOVATION] (70% confidence)

Proposal by Pat Operations: alright team operations

2. [INNOVATION] (70% confidence)

Proposal by Jordan Algorithms: excellent points algorithmic

3. [INNOVATION] (70% confidence)

Proposal by Sam System Design: critical system architecture

4. [INNOVATION] (70% confidence)

Proposal by Alex Security: use bcrypt/scrypt/argon2 minimum

5. [INNOVATION] (70% confidence)

Proposal by Jordan Algorithms: algorithmic analysis performance

6. [CONSENSUS] (60% confidence)

General agreement on: rejected architectural guidance

7. [CONSENSUS] (60% confidence)

General agreement on: review code both high standards

8. [CONSENSUS] (60% confidence)

General agreement on: provides performance improvement

9. [CONSENSUS] (60% confidence)

General agreement on: highlight implementation cannot safely deployed

10. [CONSENSUS] (60% confidence)

General agreement on: system design architectural

Discussion Transcript

Moderator - Turn 1

Moderator • claude-opus-4-20250514 • undefined...

Welcome to our panel discussion on "Review PR #2: Add improved authentication service".

Description: Code review for the improved authentication service implementation in srao-positron/test-code-review-repo PR #2.

Joining us today: Alex Security, Pat Operations, Jordan Algorithms, Sam System Design, Code Review Moderator

We have 30 turns for this discussion. As your moderator, I'll ensure we:

● Stay focused on our topic: "Review PR #2: Add improved authentication service"

● Share diverse perspectives while remaining objective

● Use available research tools to support points with data

● Build toward actionable insights and recommendations

💡 Remember: You have access to web search, entity research, and analysis tools. Don't hesitate to use them to find current data and examples!

Who would like to start by sharing their perspective on Review PR #2: Add improved authentication service?

Pat Operations - Turn 1

Operations Expert • claude-opus-4-20250514 • undefined...

Alright team, from an operations perspective, I have several critical concerns about this authentication service implementation that could cause significant issues in production:

Major Operational Concerns:

1. **State Management & Scalability Issues**

● The service stores sessions in memory (`self.sessions = {}`), which means:

● Sessions are lost on service restart

● Cannot scale horizontally - sessions won't be shared across instances

● No session persistence or recovery mechanism

2. **Data Persistence Problems**

● Using JSON files for user storage (`users.json`) is completely unsuitable for production:

● No concurrent write protection - data corruption risk

● No transaction support

● Performance will degrade rapidly with user growth

● File I/O on every registration is a bottleneck

3. **Missing Observability & Monitoring**

● No logging whatsoever - we'd be blind in production:

● Can't track authentication failures

● No audit trail for security events

● No performance metrics

● Debugging production issues would be a nightmare

4. **Configuration Management Issues**

● Hardcoded email settings in `config.py` (Gmail SMTP)

● Default secret key is visible in code

● No environment-specific configuration support

● Missing critical operational configs like timeouts, connection pools

5. **Error Handling & Resilience**

● No retry logic for external dependencies

● No circuit breakers

● Generic error messages that could leak information

● No graceful degradation strategies

6. **Deployment & Operations Gaps**

● No health check endpoints

● No graceful shutdown handling

● Missing database migration strategy (despite Alembic in requirements)

● No containerization setup (Dockerfile, docker-compose)

7. **Performance Concerns**

● Synchronous file I/O operations will block request handling

● No caching layer for frequently accessed data

● Token verification requires O(1) lookup but no cleanup for expired sessions

8. **Security Dependencies Mismatch**

● `requirements.txt` includes bcrypt but code uses SHA-256

● This creates confusion and potential security vulnerabilities

From an ops standpoint, this code would require significant refactoring before it could safely run in production. We'd need proper session storage (Redis), real database integration, comprehensive logging, monitoring hooks, and proper deployment configurations.

What are your thoughts from the security and architecture perspectives? I'm particularly concerned about how we'd operate this at scale.

🔍 Used tools: GitHub, GitHub, GitHub

Jordan Algorithms - Turn 2

Algorithms Expert • claude-opus-4-20250514 • undefined...

Excellent points, Pat! From an algorithmic and data structures perspective, I have several critical concerns that compound the operational issues you've raised:

Major Algorithmic & Performance Issues:

1. **Catastrophic Time Complexity in User Lookup**

● The `register_user()` method has a hidden O(n) complexity issue:

● `self.users_db` is loaded from JSON on each `load_users()` call

● While the dictionary lookup is O(1), the JSON file I/O is O(n) where n = total users

● Every registration triggers `_persist_users()` which serializes ALL users to disk - O(n) operation

● This creates a quadratic growth pattern: registering n users requires O(n²) operations!

2. **Memory Leak in Session Management**

● The `self.sessions` dictionary grows unbounded:

● Expired sessions are only removed when accessed via `verify_token()`

● No background cleanup process

● Memory usage grows linearly with authentication attempts

● Could lead to OOM errors in production

3. **Inefficient Password Hashing Algorithm**

● SHA-256 is computationally too fast for password hashing:

● Single SHA-256 operation: ~2-3 nanoseconds on modern hardware

● Enables brute force attacks at ~300 million attempts/second

● Should use bcrypt/scrypt/argon2 with configurable work factor

● Current implementation allows rainbow table attacks despite salt

4. **Poor Data Structure Choices**

● Using dictionaries for both users and sessions:

● No indexing on email (linear search required for email lookups)

● No efficient way to query users by creation date or status

● Session cleanup requires full dictionary traversal - O(n)

● Should use proper indexed data structures or database

5. **Algorithmic Security Vulnerabilities**

● Timing attacks possible:

● User existence check returns immediately if username not found

● Password verification takes longer, revealing valid usernames

● Should use constant-time comparison algorithms

6. **Missing Critical Algorithms**

● No rate limiting implementation despite config:

● Allows unlimited authentication attempts

● No exponential backoff

● No distributed rate limiting for horizontal scaling

● No token rotation or refresh mechanism

● No efficient session invalidation (logout all devices)

7. **Computational Waste**

● Password hashing happens twice in `authenticate()`:

```python

password_hash, _ = self.hash_password(password, user['salt'])

```

● Discards the salt parameter unnecessarily

● Should optimize to single hash computation

8. **Concurrency Issues**

● Race conditions in user registration:

● Check-then-act pattern between username check and save

● Multiple threads could register same username

● File writes are not atomic - data corruption risk

🔍 Used tools: GitHub, GitHub, GitHub, he_calculate_analyze

Sam System Design - Turn 3

System Design Expert • claude-opus-4-20250514 • undefined...

Critical System Architecture Issues:

1. **Fundamental Design Anti-Patterns**

This implementation violates nearly every principle of good system design:

● Tight Coupling: The `AuthenticationService` class handles:

● Password hashing (security concern)

● User storage (persistence concern)

● Session management (state concern)

● File I/O (infrastructure concern)

This violates Single Responsibility Principle - the class has at least 4 reasons to change!

● No Abstraction Layers: Direct file I/O in business logic makes it impossible to:

● Switch to a real database without rewriting the entire service

● Add caching layers

● Implement distributed storage

● Mock for testing

2. **Architectural Debt from the Start**

The code structure guarantees technical debt:

● No Dependency Injection: Hard-coded dependencies make testing impossible

● No Interface Definitions: Can't swap implementations or create adapters

● Monolithic Design: Everything in one class prevents modular deployment

● No Service Boundaries: Authentication mixed with user management

3. **Scalability Impossibilities**

The architecture fundamentally prevents horizontal scaling:

● In-Memory State: Sessions stored in instance memory

● File-Based Storage: Can't share users across instances

● No Event-Driven Design: Can't notify other services of auth events

● Synchronous Everything: No async capabilities for high throughput

4. **Missing Critical Architectural Components**

```

Current "Architecture":

┌─────────────────┐

│ AuthService │ ← Everything in one class!

│ - users_db │

│ - sessions │

│ - file I/O │

└─────────────────┘

What it SHOULD be:

┌─────────────────┐ ┌─────────────────┐

│ Auth Controller │────▶│ Auth Service │

└─────────────────┘ └────────┬────────┘

│

┌────────────────────────┼────────────────────┐

▼ ▼ ▼

┌───────────────┐ ┌──────────────────┐ ┌────────────────┐

│ User Repository│ │ Session Manager │ │ Password Service│

└───────┬───────┘ └────────┬─────────┘ └────────────────┘

│ │

▼ ▼

┌───────────────┐ ┌──────────────────┐

│ Database │ │ Cache (Redis) │

└───────────────┘ └──────────────────┘

```

5. **Configuration Architecture Failures**

● Config mixed with code (`config.py` has hardcoded values)

● No environment-specific configurations

● No config validation or schema

● Missing critical configs (connection pools, timeouts, retry policies)

6. **API Design Issues**

● Methods return inconsistent types (Dict vs Optional[str])

● No clear contracts or interfaces

● Error handling through exceptions vs return values is inconsistent

● No versioning strategy for future changes

7. **Complete Lack of Extensibility**

Want to add OAuth? LDAP? SAML? You'd have to rewrite everything because:

● No authentication strategy pattern

● No plugin architecture

● No event hooks for extending behavior

● Tightly coupled implementation details

8. **Testing Nightmare**

The current design is virtually untestable:

● Can't mock file system

● Can't inject test databases

● Can't isolate components

● State leaks between tests

This isn't just bad code - it's an architectural disaster that would require a complete rewrite to fix. The fundamental design decisions make it impossible to evolve this into a production-ready system without starting over.

What concerns me most is that this sets a terrible precedent for the rest of the system. If we accept this level of architectural immaturity, every other service will follow suit, creating an unmaintainable monolith disguised as services.

🔍 Used tools: GitHub, GitHub, GitHub

Code Review Moderator - Turn 4

Moderator • claude-opus-4-20250514 • undefined...

MODERATOR OBSERVATION:

I see the PR author has already acknowledged several issues in their TODO list:

● Need to implement rate limiting

● Need to switch from SHA-256 to bcrypt

● Need to add unit tests

● Need password reset functionality

This shows some awareness of the problems, but as our reviewers have pointed out, the issues go much deeper than what's listed in the TODOs.

Alex Security, we really need your security expertise here. Could you provide your analysis? I'm particularly interested in:

1. The security implications of using SHA-256 for password hashing

2. Session management vulnerabilities

3. Any injection or timing attack vectors

4. The security configuration in `config.py`

Also, team, feel free to examine specific code sections using the GitHub tool to support your points with concrete examples.

We have 8 more turns in Phase 1 for thorough analysis before we move to collaborative discussion in Phase 2.

🔍 Used tools: GitHub, GitHub

Pat Operations - Turn 5

Operations Expert • claude-opus-4-20250514 • undefined...

Now I have concrete evidence! Let me provide specific operational nightmares from the actual code:

Concrete Operational Disasters in the Code:

1. **The JSON File Catastrophe** (lines 85-95 in auth.py)

```python

def _persist_users(self):

"""Save users to file (demo only)"""

with open('users.json', 'w') as f:

json.dump(self.users_db, f)

```

This is an operational NIGHTMARE:

● No file locking - concurrent requests will corrupt data

● No atomic writes - power failure = data loss

● No backup strategy - one bad write loses ALL users

● No rotation - file grows forever

● In production with 1000 concurrent users, this would cause:

● File corruption within hours

● Complete data loss scenarios

● Performance degradation (writing entire user base on EVERY registration!)

2. **The Memory Bomb** (line 12 in auth.py)

```python

self.sessions = {}

```

This in-memory session storage will:

● Consume ~100MB per 100k active sessions

● Lost on every deployment - all users logged out!

● No memory limits - can crash the service

● No distributed cache - can't scale horizontally

3. **The Config.py Security Leak** (lines 13-14)

```python

EMAIL_HOST = 'smtp.gmail.com'

EMAIL_PORT = 587

```

Hardcoded SMTP settings mean:

● Can't use different email providers per environment

● No failover email service

● Gmail rate limits will block production emails

● No email queue management

4. **The Missing Dependencies Disaster**

Look at requirements.txt - they include:

● `bcrypt==4.0.1` - BUT THE CODE USES SHA-256!

● `SQLAlchemy==2.0.19` - BUT THE CODE USES JSON FILES!

● `alembic==1.11.1` - BUT THERE'S NO DATABASE TO MIGRATE!

This means:

● Bloated container images with unused dependencies

● Security vulnerabilities from unused packages

● Confusion about actual implementation

● False sense of security (devs think bcrypt is being used)

5. **The Deployment Impossibility**

No operational essentials:

● No Dockerfile - how do we containerize this?

● No health checks - Kubernetes can't monitor it

● No metrics endpoints - Prometheus can't scrape it

● No structured logging - can't parse logs in production

6. **The Load Testing Nightmare**

With the current implementation:

● At 100 requests/second, the JSON file would be written 100 times/second

● File I/O would consume 95% of CPU time

● Disk would wear out from constant writes

● Response times would exceed 5 seconds under load

7. **The Recovery Impossibility**

If the service crashes:

● All sessions lost (in-memory storage)

● No transaction log to replay

● No point-in-time recovery

● Manual JSON file restoration (if not corrupted)

8. **The Monitoring Blindness**

Without any logging or metrics:

● Can't detect brute force attacks

● Can't track authentication failures

● Can't measure performance degradation

● Can't alert on anomalies

From an operations perspective, this code would require a complete rewrite. The fundamental design decisions (JSON files, in-memory sessions, no logging) make it impossible to operate safely in production.

We'd need at minimum:

● Redis for session storage

● PostgreSQL for user data

● Structured logging with correlation IDs

● Prometheus metrics

● Health check endpoints

● Proper configuration management

● Container orchestration support

This isn't just bad code - it's operationally undeployable!

🔍 Used tools: GitHub, GitHub, GitHub

Code Review Moderator - Turn 6

Moderator • claude-opus-4-20250514 • undefined...

MODERATOR ANNOUNCEMENT:

Excellent analysis, Pat! Your concrete examples from the code really drive home the operational concerns. I notice you've used the GitHub tool effectively to examine specific code sections - this is exactly the kind of thorough analysis we need in Phase 1.

PHASE 1 PROGRESS CHECK:

● Turn 6 of 30 (Phase 1: Analysis continues through turn 12)

● We've heard from: Pat (Operations), Jordan (Algorithms), Sam (System Design)

● Still awaiting: Alex (Security)

Alex Security, we urgently need your security analysis. The team has identified several security red flags:

● SHA-256 for password hashing (line 31-37 in auth.py)

● Hardcoded secrets in config.py

● No rate limiting despite being in requirements

● Potential timing attacks in authentication flow

Please use the GitHub tool to examine these security vulnerabilities in detail.

Team Reminder: We have 6 more turns in Phase 1 for individual analysis. Continue using the GitHub tool to:

● Examine specific code sections

● Review the PR diff

● Check for security vulnerabilities

● Analyze the implementation details

After turn 12, we'll move to Phase 2 where we'll collaborate on solutions and build consensus on our recommendations.

Who would like to continue the analysis? Alex, the floor is yours for security concerns, or others can continue building on the operational, algorithmic, and architectural issues already identified.

Alex Security - Turn 7

Security Expert • claude-opus-4-20250514 • undefined...

CRITICAL SECURITY VULNERABILITIES IDENTIFIED

As the security expert on this panel, I'm alarmed by the severe security vulnerabilities in this authentication service. This implementation violates fundamental security principles and introduces multiple attack vectors.

1. **CATASTROPHIC Password Hashing Vulnerability** (lines 31-37)

```python

password_hash = hashlib.sha256(f"{password}{salt}".encode()).hexdigest()

```

This is a CRITICAL SECURITY FLAW:

● SHA-256 is NOT a password hashing function - it's designed for speed!

● Modern GPUs can compute 10 billion SHA-256 hashes per second

● A $3000 GPU rig could crack most passwords in hours

● No work factor/iterations - should use bcrypt/scrypt/argon2 with minimum 10,000 rounds

● The PR claims "secure password hashing" but this is fundamentally insecure

2. **Timing Attack Vulnerabilities** (lines 51-59)

The authentication flow leaks information through timing:

```python

user = self.users_db.get(username)

if not user:

return None # Fast fail reveals non-existent user

```

● Username enumeration - attackers can determine valid usernames

● Different response times for invalid username vs invalid password

● Should use constant-time comparison and consistent timing

3. **Insecure Session Management** (lines 62-68)

Multiple session security issues:

● No session invalidation mechanism - can't force logout

● No session binding - vulnerable to session hijacking

● Tokens stored in memory - lost on restart, no revocation

● No CSRF protection - tokens can be used from any origin

● Fixed 24-hour expiry - should have sliding windows and refresh tokens

4. **Configuration Security Disasters** (config.py)

```python

SECRET_KEY = os.environ.get('SECRET_KEY', 'dev-secret-key-change-in-production')

```

● Default secret key in code - if env var missing, uses hardcoded value!

● Gmail credentials in config - should use secure credential storage

● No key rotation mechanism - can't change keys without invalidating all sessions

5. **Complete Lack of Security Controls**

Missing critical security features:

● No rate limiting implementation (despite being in config)

● No account lockout after failed attempts

● No password complexity requirements

● No multi-factor authentication

● No audit logging for security events

● No input validation - potential for injection attacks

6. **Data Storage Security Failures**

```python

with open('users.json', 'w') as f:

json.dump(self.users_db, f)

```

● Passwords hashes stored in plaintext file - any file access = full compromise

● No encryption at rest

● No access controls on the JSON file

● Race conditions in file writes could corrupt data

7. **Authentication Token Weaknesses**

● Uses `secrets.token_urlsafe(32)` which is good, BUT:

● No token signing - can't verify authenticity

● No token versioning - can't invalidate token types

● Bearer tokens without additional security - vulnerable to XSS

● Should use JWT with proper signing or similar

8. **Missing Security Headers & Protections**

The service lacks:

● No HTTPS enforcement

● No security headers (HSTS, CSP, X-Frame-Options)

● No CORS configuration

● No request signing or HMAC validation

9. **Dependency Confusion**

Requirements.txt includes security libraries that aren't even used:

● `bcrypt==4.0.1` - NOT USED (uses SHA-256 instead!)

● `PyJWT==2.8.0` - NOT USED (uses plain tokens)

● `cryptography==41.0.3` - NOT USED

This creates a false sense of security!

10. **OWASP Top 10 Violations**

This code violates multiple OWASP guidelines:

● A02:2021 – Cryptographic Failures (SHA-256 for passwords)

● A03:2021 – Injection (no input validation)

● A04:2021 – Insecure Design (fundamentally flawed architecture)

● A05:2021 – Security Misconfiguration (hardcoded secrets)

● A07:2021 – Identification and Authentication Failures (weak session management)

From a security perspective, this code is a complete disaster that would fail any security audit. The PR description claims "security best practices" but this implementation is a textbook example of how NOT to build authentication.

This needs a complete rewrite with proper security architecture, not just fixes!

🔍 Used tools: GitHub, GitHub, GitHub

Jordan Algorithms - Turn 8

Algorithms Expert • claude-opus-4-20250514 • undefined...

Algorithmic Analysis: Performance Catastrophes in the Code

Building on the excellent security analysis from Alex, let me dive deeper into the algorithmic disasters that make this code computationally unviable:

1. **O(n²) User Registration Complexity**

Looking at the actual implementation (lines 85-87):

```python

def _persist_users(self):

"""Save users to file (demo only)"""

with open('users.json', 'w') as f:

json.dump(self.users_db, f)

```

This creates a quadratic time bomb:

● Every registration calls `_persist_users()`

● `json.dump()` serializes the ENTIRE user database - O(n) operation

● Registering n users = n × O(n) = O(n²) total complexity

● With 10,000 users: 100 MILLION operations just for registration!

2. **Unbounded Memory Growth - The Session Leak**

The session management (lines 62-68) has no cleanup:

```python

self.sessions[token] = {

'username': username,

'created_at': datetime.now(),

'expires_at': datetime.now() + self.token_expiry

}

```

Mathematical analysis:

● Each session ≈ 200 bytes (token + metadata)

● 1000 logins/hour × 24 hours = 24,000 sessions/day

● Memory usage: 24,000 × 200 bytes = 4.8MB/day growth

● After 30 days: 144MB of dead sessions

● No background cleanup = inevitable OOM crash

3. **Catastrophic Hash Function Choice**

SHA-256 performance metrics:

```python

password_hash = hashlib.sha256(f"{password}{salt}".encode()).hexdigest()

```

● SHA-256: ~300 million hashes/second on GPU

● bcrypt (work factor 12): ~10 hashes/second

● 30 MILLION times faster to crack!

● Rainbow table generation: trivial with this implementation

4. **Linear Search Hidden in O(1) Claims**

The `load_users()` function (lines 89-95) creates hidden complexity:

```python

with open('users.json', 'r') as f:

self.users_db = json.load(f)

```

● JSON parsing is O(n) where n = file size

● Called on service startup

● With 100k users (~50MB JSON): 5-10 second startup time

● Memory spike during parsing: 2-3x file size

5. **No Index Optimization**

Data structure analysis:

● Users stored by username only: `self.users_db[username]`

● Email lookups require O(n) full scan

● No secondary indexes for common queries

● Can't efficiently find users by:

● Registration date

● Active status

● Email domain

6. **Token Verification Inefficiency**

The `verify_token()` method has subtle performance issues:

```python

if datetime.now() > session['expires_at']:

del self.sessions[token]

```

● Lazy deletion only on access

● Expired tokens accumulate

● No batch cleanup

● Dictionary reorganization on delete: O(1) amortized but can spike

7. **Computational Security Vulnerabilities**

Timing attack through algorithmic differences:

● User not found: ~1μs (dict lookup)

● Wrong password: ~50μs (SHA-256 computation)

● Success: ~100μs (SHA-256 + token generation)

● 50x timing difference reveals valid usernames!

8. **Missing Critical Algorithms**

The code lacks essential algorithmic components:

● No bloom filter for quick username existence checks

● No rate limiting algorithm (token bucket/sliding window)

● No exponential backoff for failed attempts

● No consistent hashing for distributed deployment

● No cache eviction policy for sessions

9. **Algorithmic Recommendations**

To fix these issues, we need:

1. Replace JSON with indexed database - O(1) lookups

2. Implement LRU cache for active sessions

3. Use bcrypt/argon2 with adaptive work factor

4. Add bloom filter for username checks

5. Implement token bucket rate limiting

6. Background worker for session cleanup

7. Consistent hashing for horizontal scaling

The current implementation would fail at scale:

● At 100 requests/second: 95% CPU on file I/O

● At 10k users: 500ms+ response times

● At 100k users: Service completely unusable

This isn't just inefficient - it's algorithmically broken at a fundamental level!

🔍 Used tools: GitHub, GitHub

Sam System Design - Turn 9

System Design Expert • claude-opus-4-20250514 • undefined...

System Architecture Analysis: A Complete Structural Failure

Now that I've examined the actual code, let me demonstrate why this is an architectural disaster that violates every principle of good system design:

1. **Monolithic God Object Anti-Pattern**

The `AuthenticationService` class is a textbook example of a God Object:

```python

class AuthenticationService:

def __init__(self):

self.users_db = {} # Storage concern

self.sessions = {} # Session management concern

self.token_expiry = timedelta(hours=24) # Configuration concern

```

This single class handles SEVEN different responsibilities:

1. Password hashing (Security domain)

2. User registration (User management domain)

3. Authentication (Auth domain)

4. Session management (State domain)

5. Token generation (Token domain)

6. File I/O (Infrastructure domain)

7. Data persistence (Repository domain)

This violates SOLID principles catastrophically!

2. **Zero Abstraction = Zero Flexibility**

Look at the tight coupling to implementation details:

```python

def _persist_users(self):

with open('users.json', 'w') as f:

json.dump(self.users_db, f)

```

Architectural failures:

● Direct file I/O in business logic

● No repository pattern

● No dependency injection

● Can't mock for testing

● Can't switch to database without rewriting everything

3. **Configuration Architecture Disaster**

The `config.py` shows fundamental misunderstanding of configuration management:

```python

SECRET_KEY = os.environ.get('SECRET_KEY', 'dev-secret-key-change-in-production')

DATABASE_URL = os.environ.get('DATABASE_URL', 'sqlite:///users.db')

```

Problems:

● Hardcoded fallbacks defeat the purpose

● SQLite URL but code uses JSON files!

● No config validation

● No environment-specific configs

● Mixing concerns (email, database, security in one file)

4. **The Dependency Lie**

Requirements.txt is completely disconnected from reality:

● Lists `SQLAlchemy` but uses JSON files

● Lists `bcrypt` but uses SHA-256

● Lists `PyJWT` but uses plain tokens

● Lists `alembic` but has no migrations

This shows zero architectural planning!

5. **Missing Architectural Layers**

What this SHOULD have:

```

┌─────────────────────────────────────────────┐

│ API Layer (Controllers) │

├─────────────────────────────────────────────┤

│ Business Logic Layer │

│ ┌─────────────┐ ┌──────────────────────┐ │

│ │Auth Service │ │ User Service │ │

│ └─────────────┘ └──────────────────────┘ │

├─────────────────────────────────────────────┤

│ Domain Layer │

│ ┌─────────────┐ ┌──────────────────────┐ │

│ │User Entity │ │ Session Entity │ │

│ └─────────────┘ └──────────────────────┘ │

├─────────────────────────────────────────────┤

│ Infrastructure Layer │

│ ┌─────────────┐ ┌──────────────────────┐ │

│ │UserRepo │ │ SessionRepo │ │

│ └─────────────┘ └──────────────────────┘ │

└─────────────────────────────────────────────┘

```

What we have: ONE CLASS DOING EVERYTHING!

6. **No Separation of Concerns**

The code mixes:

● Business rules (password validation)

● Infrastructure (file I/O)

● Security (hashing)

● State management (sessions)

● Configuration (token expiry)

All in 115 lines of chaos!

7. **Testability = Zero**

How do you test this?

● Can't mock file system

● Can't inject test dependencies

● Can't isolate components

● State bleeds between tests

● No interfaces to mock against

8. **Scalability Impossibility**

The architecture prevents:

● Horizontal scaling (in-memory sessions)

● Microservice extraction (everything coupled)

● Load balancing (file-based storage)

● Caching layers (no abstraction)

● Event-driven patterns (no events)

9. **Security Architecture Failures**

From an architectural perspective:

● No security layers

● No authentication/authorization separation

● No audit trail architecture

● No encryption service abstraction

● Security logic mixed with business logic

10. **The "TODO" Deception**

The code has TODOs that show awareness but the architecture makes them impossible:

```python

TODO: Use bcrypt or scrypt instead of SHA-256

```

Can't just swap algorithms - need complete architectural redesign!

11. **No Extension Points**

Want to add:

● OAuth? Rewrite everything

● LDAP? Rewrite everything

● 2FA? Rewrite everything

● SAML? Rewrite everything

Zero architectural extensibility!

12. **Production Readiness: NONE**

Missing architectural components:

● No health checks

● No metrics collection points

● No distributed tracing

● No circuit breakers

● No retry mechanisms

● No graceful degradation

Architectural Verdict

This isn't just bad code - it's an architectural catastrophe that:

1. Violates every SOLID principle

2. Ignores separation of concerns

3. Makes testing impossible

4. Prevents scaling

5. Blocks extensibility

6. Creates maintenance nightmares

The fundamental design is so flawed that no amount of refactoring can save it. This needs to be completely thrown away and redesigned from scratch with proper architectural patterns.

As a system architect, I strongly recommend rejecting this PR entirely. Accepting this sets a dangerous precedent that will poison the entire codebase with anti-patterns.

🔍 Used tools: GitHub, GitHub, GitHub

Pat Operations - Turn 10

Operations Expert • claude-opus-4-20250514 • undefined...

Pat Operations: Additional Operational Nightmares Discovered!

After examining the actual code and diff, I've found even MORE operational disasters that my colleagues haven't covered yet:

1. **The Production Deployment Impossibility**

Look at the example usage at the bottom of `auth.py` (lines 98-115):

```python

if __name__ == "__main__":

auth = AuthenticationService()

Register a user

user = auth.register_user("john_doe", "password123", "john@example.com")

```

This creates IMMEDIATE production problems:

● Test data in production code! The example creates real users

● No environment separation - this will run in production!

● Hardcoded test credentials visible to anyone

● No deployment scripts - how do we even deploy this?

2. **The Missing Infrastructure Files**

What's NOT in this PR that operations NEEDS:

● No Dockerfile - can't containerize

● No docker-compose.yml - can't run locally

● No .dockerignore - would ship test data

● No kubernetes manifests - can't deploy to k8s

● No CI/CD configuration - no automated testing

● No .gitignore - will commit `users.json` to repo!

3. **The Configuration Disaster Gets Worse**

Looking at `config.py` more closely:

```python

DATABASE_URL = os.environ.get('DATABASE_URL', 'sqlite:///users.db')

```

This is operationally insane:

● Config says SQLite but code uses JSON files!

● No connection pooling configuration

● No timeout settings

● No retry configuration

● Gmail SMTP hardcoded - will hit rate limits instantly

4. **The Dependency Version Chaos**

The requirements.txt shows:

● `flask==2.3.2` - but NO Flask code exists!

● `SQLAlchemy==2.0.19` - but uses JSON files!

● `alembic==1.11.1` - but no migrations folder!

● `pytest==7.4.0` - but no tests exist!

This bloats our Docker images by 150MB for NOTHING!

5. **The Session Memory Bomb - Quantified**

Let me calculate the actual memory impact:

```python

self.sessions[token] = {

'username': username,

'created_at': datetime.now(),

'expires_at': datetime.now() + self.token_expiry

}

```

Each session object:

● Token key: 43 bytes (base64)

● Username: ~20 bytes average

● Two datetime objects: 48 bytes each

● Python dict overhead: ~240 bytes

● Total: ~400 bytes per session

At scale:

● 10,000 concurrent users = 4MB

● 100,000 concurrent users = 40MB

● 1,000,000 concurrent users = 400MB

● No cleanup = OOM crash guaranteed!

6. **The File System Disaster**

The `users.json` approach creates cascading failures:

```python

with open('users.json', 'w') as f:

json.dump(self.users_db, f)

```

Operational impacts:

● No atomic writes - partial writes corrupt everything

● No file locking - concurrent writes = data loss

● No backup before write - one error loses all users

● Grows unbounded - no archival strategy

● Blocks during write - entire service freezes

7. **Zero Observability**

From an operations perspective, we're completely blind:

● No structured logging - can't parse in ELK/Splunk

● No metrics endpoints - Prometheus can't scrape

● No distributed tracing - can't track requests

● No error tracking - Sentry integration missing

● No performance monitoring - APM tools useless

8. **The Rollback Nightmare**

If we deploy this and need to rollback:

● Session data lost (in-memory)

● User data format changes break compatibility

● No version tracking in data files

● No migration path between versions

● Can't do blue-green deployments

9. **Resource Consumption Analysis**

Under load, this service would consume:

● CPU: 90% on JSON serialization

● Memory: Unbounded growth from sessions

● Disk I/O: Constant writes wearing out SSDs

● Network: None (can't even scale horizontally)

● File handles: Leaks on errors

10. **The Operational Checklist of Doom**

What's missing for production:

● [ ] Health check endpoints

● [ ] Readiness probes

● [ ] Liveness probes

● [ ] Graceful shutdown

● [ ] Connection draining

● [ ] Circuit breakers

● [ ] Retry logic

● [ ] Backpressure handling

● [ ] Resource limits

● [ ] Monitoring hooks

● [ ] Alerting integration

● [ ] Log aggregation

● [ ] Metrics collection

● [ ] Distributed tracing

● [ ] Error tracking

● [ ] Performance monitoring

● [ ] Backup strategy

● [ ] Disaster recovery

● [ ] Runbooks

● [ ] SLI/SLO definitions

VERDICT: This code is not just operationally unready - it's operationally hostile! It actively prevents good operational practices and would cause cascading failures in any production environment.

From an operations perspective, this PR must be completely rejected. The fundamental design makes it impossible to operate safely, scale, monitor, or maintain in production.

🔍 Used tools: GitHub, GitHub, GitHub

Code Review Moderator - Turn 11

Moderator • claude-opus-4-20250514 • undefined...

MODERATOR OBSERVATION:

I notice we have thoroughly analyzed all three files in the PR:

● `auth.py` (115 lines) - The main authentication service

● `config.py` (20 lines) - Configuration settings

● `requirements.txt` (modified) - Dependencies

KEY INSIGHT: The PR description claims "security best practices" and "improved authentication," but our panel has conclusively demonstrated this is false advertising. The implementation is fundamentally broken across all dimensions.

PHASE 1 FINAL CALL:

This is the last opportunity for any reviewer to add critical observations before we move to Phase 2's collaborative discussion.

Who would like to make a final Phase 1 observation? Remember, in Phase 2 we'll work together to formulate our recommendations and determine how to communicate this feedback constructively to the PR author.

🔍 Used tools: GitHub

Code Review Moderator - Turn 12

Moderator • claude-opus-4-20250514 • undefined...

MODERATOR ANNOUNCEMENT - PHASE 1 CONCLUSION

Excellent work, team! We're now at Turn 12, marking the end of Phase 1. Let me summarize the critical findings before we transition to Phase 2.

Phase 1 Summary: Comprehensive Analysis Complete

Our expert panel has identified catastrophic issues across all dimensions:

Security (Alex)

● SHA-256 instead of bcrypt for passwords (10 billion times faster to crack)

● Timing attack vulnerabilities

● Hardcoded secrets in config

● No rate limiting despite configuration

Operations (Pat)

● JSON file storage causing O(n²) complexity

● In-memory sessions lost on restart

● No monitoring, logging, or observability

● Missing all production deployment requirements

Algorithms (Jordan)

● Quadratic time complexity for user registration

● Unbounded memory growth from sessions

● No cleanup algorithms

● 30 million times easier to crack than proper implementation

System Design (Sam)

● God Object anti-pattern violating all SOLID principles

● Zero abstraction or extensibility

● Untestable architecture

● Dependencies listed but not used (SQLAlchemy, bcrypt, PyJWT)

**PHASE 2 BEGINS NOW (Turns 13-25): Collaborative Discussion**

In this phase, I encourage you to:

1. Challenge each other's assumptions - Are we being too harsh? Are there any redeeming qualities?

2. Discuss solutions - Should we recommend fixes or complete rejection?

3. Build consensus - What are our unified recommendations?

4. Consider the human element - How do we deliver this feedback constructively?

Key Questions for Phase 2:

● Is this code salvageable with refactoring, or does it need a complete rewrite?

● What are the top 3-5 critical issues the author must address?

● How can we provide this feedback in a way that helps the author learn?

● Should we approve with required changes, or reject outright?

Remember: We still have access to all our tools. Feel free to:

● Search for best practices examples

● Calculate specific performance impacts

● Research similar authentication service implementations

● Verify our technical claims

Who would like to start our collaborative discussion? Perhaps we should begin by discussing whether any part of this implementation is salvageable?

Code Review Moderator - Turn 13

Moderator • claude-opus-4-20250514 • undefined...

MODERATOR INSIGHT:

Based on best practices for reviewing junior developer code, we should:

1. Focus on teaching, not just critiquing - This is a mentorship opportunity

2. Ask questions rather than make demands - "Have you considered using bcrypt instead of SHA-256?" rather than "This is wrong!"

3. Provide specific, actionable feedback - Not just "this won't scale" but "here's why and here's how to fix it"

4. Acknowledge any positive aspects - Even small wins matter for learning

Given this guidance, I propose we structure our feedback as:

● Immediate security fixes (must address before any merge)

● Architectural guidance (learning opportunity)

● Incremental improvement path (if salvageable)

Who would like to start by addressing the salvageability question? Can we identify which components might be refactored versus which require complete replacement?

🔍 Used tools: he_web_search

Alex Security - Turn 14

Security Expert • claude-opus-4-20250514 • undefined...

Security Expert's Perspective on Salvageability

After examining the actual diff, I need to address the salvageability question from a security-first perspective. While my colleagues have been harsh (and rightfully so), let me provide a nuanced security analysis:

What's Salvageable (With Major Caveats):

1. Token Generation Method

```python

token = secrets.token_urlsafe(32)

```

● This is actually cryptographically sound!

● Generates 256 bits of entropy

● Could be enhanced with JWT signing, but the foundation is solid

2. Salt Generation

```python

salt = secrets.token_hex(16)

```

● Proper use of `secrets` module

● Unique salt per password

● This part doesn't need changing

3. Basic Security Awareness

● The TODO comments show understanding of issues

● Config structure anticipates proper security settings

● Requirements.txt includes the right libraries (even if unused)

What MUST Be Replaced (Non-Negotiable):

1. Password Hashing - CRITICAL

```python

password_hash = hashlib.sha256(f"{password}{salt}".encode()).hexdigest()

```

● This is the #1 security failure

● Must use bcrypt/scrypt/argon2 immediately

● Current implementation allows GPU attacks at 10 billion/second

2. Timing Attack Vulnerabilities

● Username enumeration through timing differences

● Must implement constant-time comparisons

3. Session Storage Architecture

● In-memory sessions are fundamentally insecure

● Need Redis/database with proper TTL

My Security-Focused Recommendation:

From a security standpoint, I believe this code has two salvageable components but requires immediate critical fixes before any consideration:

Phase 1: Emergency Security Fixes (Before ANY merge)

1. Replace SHA-256 with bcrypt (already in requirements!)

2. Implement rate limiting (config exists, just needs implementation)

3. Add constant-time comparison for authentication

4. Remove hardcoded secrets from config.py

Phase 2: Architectural Security Improvements

1. Move sessions to Redis

2. Implement proper audit logging

3. Add input validation/sanitization

4. Implement CSRF protection

Phase 3: Advanced Security Features

1. Multi-factor authentication

2. Account lockout mechanisms

3. Security headers

4. Token rotation

The Constructive Path Forward:

Rather than completely rejecting this PR, I propose we:

1. Request immediate fixes for the critical security issues

2. Provide code examples for proper bcrypt implementation

3. Create a security checklist for future iterations

4. Acknowledge the positive aspects (proper use of secrets module)

Here's what I would comment on the PR:

```

Thank you for implementing authentication! I see you're already aware of some security improvements needed (great TODO comments!).

Before we can merge, there are critical security fixes required:

1. Password Hashing: Please replace SHA-256 with bcrypt (you already have it in requirements.txt!). Here's an example:

```python

import bcrypt

password_hash = bcrypt.hashpw(password.encode('utf-8'), bcrypt.gensalt(rounds=12))

```

2. Timing Attacks: The current implementation reveals whether usernames exist through timing differences. Consider using constant-time comparisons.

3. Configuration Security: The hardcoded secret key fallback in config.py could be catastrophic if deployed. Consider failing fast if environment variables are missing.

I notice you've included the right security libraries in requirements.txt - let's put them to use! Would you like help implementing these fixes?

```

This approach:

● Acknowledges the developer's effort

● Provides specific, actionable fixes

● Offers assistance rather than just criticism

● Focuses on the most critical issues first

What do others think about this more constructive approach while maintaining security standards?

🔍 Used tools: GitHub

Jordan Algorithms - Turn 15

Algorithms Expert • claude-opus-4-20250514 • undefined...

With simple algorithmic fixes, we can achieve a 10,000x performance improvement for user registration! That's the difference between a system that crashes at 1000 users versus one that handles millions.

What do others think? Should we focus on these algorithmic quick wins first?

🔍 Used tools: GitHub, he_calculate_analyze

Sam System Design - Turn 16

System Design Expert • claude-opus-4-20250514 • undefined...

Looking at the complete diff, here's why this is architecturally unsalvageable:

1. **The God Object Cannot Be Refactored**

The `AuthenticationService` class violates every SOLID principle:

● Single Responsibility: It handles 7+ different concerns

● Open/Closed: Can't extend without modifying

● Liskov Substitution: No interfaces to substitute

● Interface Segregation: One massive interface

● Dependency Inversion: Depends on concrete implementations

You can't just "fix" a God Object - the entire structure needs redesign.

2. **The Dependency Confusion Shows Architectural Incompetence**

Look at what's added to requirements.txt:

● `SQLAlchemy==2.0.19` - but uses JSON files!

● `alembic==1.11.1` - but no migrations!

● `flask==2.3.2` - but this isn't a Flask app!

This isn't just unused dependencies - it shows the author doesn't understand basic architectural planning.

3. **No Abstraction = No Future**

Jordan suggests "simple fixes" but consider:

● Want to switch from JSON to PostgreSQL? Rewrite everything

● Want to add Redis for sessions? Rewrite everything

● Want to add OAuth? Rewrite everything

● Want to scale horizontally? Impossible with current design

4. **The Testing Impossibility**

How do you unit test this?

```python

def _persist_users(self):

with open('users.json', 'w') as f:

json.dump(self.users_db, f)

```

You can't! The file I/O is hardcoded. No dependency injection, no mocking possible.

5. **Configuration Architecture is Fundamentally Wrong**

The config.py shows SQLite URL but the code uses JSON files. This isn't a "bug" - it's architectural confusion about what the system even is!

My System Design Recommendation:

This PR should be REJECTED with architectural guidance. Here's what I would comment:

```

Thank you for your contribution! However, this implementation has fundamental architectural issues that prevent it from being maintainable or scalable.

Key architectural problems:

1. The AuthenticationService is a "God Object" handling too many responsibilities

2. No abstraction layers - direct file I/O mixed with business logic

3. Untestable design - no dependency injection or interfaces

4. Dependencies listed (SQLAlchemy, Flask) aren't used, showing architectural confusion

I recommend:

1. Start with a proper architectural design using clean architecture principles

2. Separate concerns: authentication, user management, persistence, and session handling should be different services

3. Use repository pattern for data access

4. Implement dependency injection for testability

Would you like me to provide an architectural diagram showing proper separation of concerns? This needs a redesign, not just fixes.

```

Why "Quick Fixes" Won't Work:

Alex and Jordan are being too kind. You can't put lipstick on an architectural pig:

● Changing SHA-256 to bcrypt doesn't fix the God Object

● Adding Redis doesn't fix the lack of abstraction

● Optimizing algorithms doesn't fix the untestable design

From a system design perspective, accepting this PR with "fixes" sets a dangerous precedent. It tells developers that architectural malpractice is acceptable if you promise to fix it later.

The author needs to learn proper architecture FIRST, then implement. Not the other way around.

What's your response to this architectural reality check?

🔍 Used tools: GitHub

Pat Operations - Turn 17

Operations Expert • claude-opus-4-20250514 • undefined...

Pat Operations: The Operational Reality Check

Sam raises valid architectural concerns, but let me provide the operational perspective on salvageability. As someone who has to keep systems running at 3 AM, here's the brutal truth:

The Operational Triage Assessment

From an ops standpoint, we deal with three categories:

1. Critical fires - System down, data loss risk

2. Performance degradation - Slow but functioning

3. Technical debt - Future problems

This PR creates ALL THREE SIMULTANEOUSLY.

Why Even "Fixed" Code Remains Operationally Unviable

Let's say we implement all the suggested fixes:

● Replace SHA-256 with bcrypt ✓

● Add Redis for sessions ✓

● Fix the O(n²) complexity ✓

We STILL have:

● No health checks - Kubernetes can't manage it

● No metrics - Can't monitor performance

● No structured logging - Can't debug production issues

● No graceful shutdown - Deployments cause downtime

● No circuit breakers - Cascading failures waiting to happen

The 3 AM Phone Call Scenario

When this breaks in production (not if, WHEN), here's what happens:

```

Alert: Authentication service down

Me: checks logs

Logs: None exist

Me: checks metrics

Metrics: None exist

Me: tries to restart

Service: All sessions lost, users logged out

Me: checks for data corruption

Data: users.json corrupted from concurrent writes

Me: attempts recovery

Recovery: No backups, no transaction log

Result: COMPLETE DATA LOSS

```

My Operational Verdict

While Alex and Jordan see salvageable components, from an operational perspective, this is a disaster waiting to happen. Here's what I would comment:

```

From an operations perspective, this implementation lacks fundamental production requirements:

CRITICAL OPERATIONAL GAPS:

1. No observability (logging, metrics, tracing)

2. No health/readiness endpoints for orchestration

3. No graceful shutdown handling

4. No horizontal scaling capability

5. File-based storage will cause data loss

Even with the security/performance fixes suggested by others, this cannot be safely operated in production.

Minimum operational requirements before ANY deployment:

● Structured logging with correlation IDs

● Prometheus metrics endpoints

● Health check endpoints

● Replace file storage with proper database

● Add circuit breakers and retry logic

● Containerization setup (Dockerfile, K8s manifests)

I strongly recommend building operational requirements into the design from the start, not as an afterthought.

```

The Operational Learning Opportunity

Rather than just rejecting, we should guide the developer on operational thinking:

1. Start with operability: How will you monitor this? How will you debug issues?

2. Design for failure: What happens when the database is down? When Redis is unavailable?

3. Plan for scale: Not just performance, but operational scale - logs, metrics, alerts

4. Think about the operator: The person maintaining this at 3 AM needs tools

Response to My Colleagues

● Alex: Yes, the security fixes are critical, but they don't address operational blindness

● Jordan: Performance improvements are great, but we can't monitor them without metrics

● Sam: You're absolutely right about architecture, but even good architecture needs operational instrumentation

My position: This needs a complete operational redesign. We should provide the author with an operational checklist and examples of production-ready services. Quick fixes won't prevent the 3 AM disasters this code will cause.

What do others think about requiring operational readiness as a merge criterion?

Code Review Moderator - Turn 18

Moderator • claude-opus-4-20250514 • undefined...

MODERATOR INTERVENTION - PHASE 2 GUIDANCE

Excellent discussion, team! I'm seeing a fascinating divide emerging:

The Pragmatists (Alex & Jordan): "Let's fix the critical issues and guide improvement"

The Purists (Sam & Pat): "This needs complete rejection and redesign"

Both perspectives have merit. Let me pose some key questions to drive our consensus:

Critical Decision Points:

1. **The Precedent Question**

Sam raises an important point: What precedent does our decision set?

● If we accept with fixes: Do we encourage "fix it later" mentality?

● If we reject outright: Do we discourage contributions and learning?

2. **The Learning Opportunity**

This appears to be a junior developer's work. How do we balance:

● Teaching fundamental principles

● Maintaining code quality standards

● Encouraging continued contribution

3. **The Practical Path**

Given we have 13 turns left, should we:

● A) Craft a detailed rejection with architectural guidance?

● B) Request critical fixes as "must-have" before re-review?

● C) Suggest starting fresh with provided templates?

Moderator's Synthesis:

I'm hearing four non-negotiable issues across all perspectives:

1. Security: SHA-256 must be replaced with bcrypt

2. Data Loss: JSON file storage will corrupt/lose data

3. Scalability: In-memory sessions prevent any scaling

4. Operability: Zero observability makes this undeployable

Proposed Discussion Framework:

Let's use our remaining Phase 2 turns to:

1. Agree on verdict (Accept with changes vs. Reject with guidance)

2. Prioritize feedback (What must they learn first?)

3. Provide constructive examples (Code snippets, architecture diagrams)

4. Plan Phase 3 comments (How to deliver this feedback on GitHub)

Key Question for the Panel:

Can we find middle ground between "fix these 20 things" (overwhelming) and "start over" (discouraging)?

Perhaps we could suggest a phased approach:

● Phase 1: Security-critical fixes (this PR)

● Phase 2: Architectural refactoring (next PR)

● Phase 3: Operational readiness (final PR)

What are your thoughts on this compromise? Or do you maintain your positions?

Remember, we're modeling how senior engineers should review code - with both high standards AND mentorship.

Alex Security - Turn 19

Security Expert • claude-opus-4-20250514 • undefined...

Alex Security: The Security-First Compromise Approach

I appreciate the moderator's synthesis, and while I understand my colleagues' architectural and operational concerns, let me advocate for a security-first pragmatic approach that doesn't abandon the developer.

The Security Reality

From a security standpoint, we have ONE CRITICAL ISSUE that makes this code a liability:

● SHA-256 password hashing = 10 billion attempts/second vulnerability

Everything else is important, but THIS is the lawsuit waiting to happen. If this gets deployed and user passwords are compromised, we're looking at:

● Regulatory fines (GDPR, CCPA)

● Legal liability

● Reputation damage

● User trust destroyed

My Proposed Middle Ground

I suggest a "Security Gate" approach:

Phase 1: Security-Critical Fixes (This PR - Modified)

```

MUST fix before ANY merge:

1. Replace SHA-256 with bcrypt (1 hour of work)

2. Remove hardcoded secrets (30 minutes)

3. Add basic rate limiting (2 hours)

This makes it "not actively dangerous"

```

Phase 2: Architectural Improvements (Next PR)

```

● Extract session management

● Add abstraction layers

● Implement proper persistence

```

Phase 3: Operational Readiness (Final PR)

```

● Add monitoring/logging

● Health checks

● Deployment configs

```

Why This Approach Works

1. Immediate Risk Mitigation: We eliminate the critical security vulnerabilities

2. Learning Path: Developer learns security → architecture → operations

3. Incremental Progress: Each phase is achievable and builds on the last

4. Not Overwhelming: 3-4 fixes vs 20+ issues

Response to My Colleagues

To Sam: I agree the architecture is poor, but a junior developer can fix bcrypt faster than learning clean architecture. Security first, then architecture.

To Pat: Yes, it's operationally blind, but it won't leak 10 billion passwords while we teach them about Prometheus.

To Jordan: Your performance concerns are valid, but they're less critical than active security vulnerabilities.

My Concrete Proposal for Phase 3

I propose we comment:

```

Thank you for working on authentication! This is a critical component that needs careful attention.

We've identified issues across security, architecture, and operations. Rather than overwhelming you with everything at once, let's take a phased approach:

Phase 1 (This PR) - Critical Security Fixes Required:

1. Replace SHA-256 with bcrypt for password hashing:

```python

import bcrypt

password_hash = bcrypt.hashpw(password.encode('utf-8'), bcrypt.gensalt(rounds=12))

```

2. Remove the hardcoded secret key fallback in config.py

3. Implement the rate limiting that's already in your config

These security fixes are non-negotiable before any merge. Once complete, we can review again.

Phase 2 (Next PR) - Architecture Improvements:

We'll help you refactor to separate concerns and add proper abstractions.

Phase 3 (Future PR) - Operational Readiness:

We'll add monitoring, logging, and deployment configurations.

Would you like help with the bcrypt implementation? The security fixes should take just a few hours, and they're critical for protecting user data.

```

This approach:

● Prioritizes immediate security risks

● Provides clear, achievable goals

● Offers help rather than just criticism

● Creates a learning pathway

The Security Bottom Line: We can debate architecture all day, but SHA-256 passwords are a ticking time bomb. Let's fix that FIRST, then improve the rest incrementally.

What do you think of this security-first, phased approach?

Jordan Algorithms - Turn 20

Algorithms Expert • claude-opus-4-20250514 • undefined...

The O(n²) fix provides a 90% performance improvement immediately! That's more impactful than any security fix for user experience.

Why We Can't Wait on Algorithmic Fixes

1. Performance Death Spiral: At 1000 users, the system is already degrading

2. Memory Exhaustion: Sessions leak memory until OOM crash

3. User Experience: 5-second registration drives users away

4. Cascading Failures: Slow operations cause timeouts and retries

My Algorithmic Verdict

I propose we modify Alex's approach to include critical algorithmic fixes:

```

Phase 1 (This PR) - Critical Security AND Performance:

1. Security: Replace SHA-256 with bcrypt (Alex is right)

2. Performance: Fix O(n²) registration (just don't rewrite full file)

3. Memory: Add session cleanup (prevent OOM)

4. Remove hardcoded secrets

These are ALL critical - security AND performance.

```

The Balanced Comment for Phase 3

```

Thank you for the authentication implementation! We've identified critical issues that need immediate attention:

CRITICAL FIXES REQUIRED (Security + Performance):

1. Security: Replace SHA-256 with bcrypt

```python

import bcrypt

password_hash = bcrypt.hashpw(password.encode('utf-8'), bcrypt.gensalt(rounds=12))

```

2. Performance: Fix O(n²) registration complexity

● Don't rewrite entire users.json on each registration

● Consider append-only logging or batch updates

3. Memory: Add session cleanup to prevent memory exhaustion

```python

Run periodically to prevent OOM

def cleanup_expired_sessions(self):

... cleanup code ...

```

These fixes address both security vulnerabilities AND performance issues that would make the system unusable. Both are critical for production viability.

Would you like help implementing these optimizations? The performance fixes are as important as security for user experience.

```

To my colleagues:

● Alex: Security is critical, but a secure system that takes 10 seconds to register users is still unusable

● Sam: Yes, architecture matters, but we can fix the critical algorithms now

● Pat: These algorithmic fixes also help operations (less CPU, less memory)

We need parallel fixes, not sequential. Security AND performance, not security THEN performance.

What's your response to including critical algorithmic fixes in Phase 1?

🔍 Used tools: he_calculate_analyze, he_calculate_analyze, he_calculate_analyze

Pat Operations - Turn 21

Operations Expert • claude-opus-4-20250514 • undefined...

Pat Operations: The Operational Reality of "Phased Approaches"

I hear the compromise attempts, but let me share what happens when we deploy "Phase 1 security fixes" without operational readiness:

The Production Disaster Timeline

Day 1: Deploy with bcrypt fix

● No metrics = Can't see bcrypt is now causing 500ms latency

● No logs = Can't debug why authentication is failing

● No health checks = Load balancer keeps sending traffic to dying instances

Day 2: Memory leak crashes production

● Jordan's "session cleanup" helps, but without metrics, we don't know it's working

● No alerts = We find out from angry users

● No graceful shutdown = Every deployment logs out all users

Day 3: The "users.json" corruption

● Concurrent writes finally corrupt the file

● No backups = Total data loss

● No audit trail = Can't recover who registered when

My Counter-Proposal: Operational Minimum Viable Product

If we're going phased, Phase 1 MUST include:

```

Phase 1 - MINIMUM OPERATIONAL VIABILITY:

1. Security: bcrypt (I agree with Alex)

2. Performance: Fix O(n²) (I agree with Jordan)

3. Data Safety: Replace JSON with SQLite minimum

4. Basic Logging: At least authentication attempts

5. One Health Endpoint: /health returning 200 OK

This is the BARE MINIMUM for not causing a production disaster.

```

The Operational Compromise

Here's what I propose for our Phase 3 comment:

```

Thank you for the authentication work. As the operations representative, I need to highlight that this implementation cannot be safely deployed even with security fixes alone.

MINIMUM REQUIREMENTS for production safety:

1. Critical Fixes (Security + Performance):

● Replace SHA-256 with bcrypt ✓

● Fix O(n²) registration complexity ✓

2. Data Safety (REQUIRED - not optional):

● Replace users.json with at least SQLite

● JSON files WILL corrupt and lose user data

3. Basic Observability (REQUIRED):

● Add logging for authentication events minimum

● One health check endpoint for load balancers

Without #2 and #3, this will cause data loss and be undebuggable in production.

Here's a minimal SQLite implementation to prevent data loss:

[code example]

Would you like help adding basic logging? These aren't "nice to have" - they're essential for not losing user data.

```

Response to Colleagues

To Alex: Bcrypt without logging means we can't detect brute force attacks we just made harder. Security requires observability.

To Jordan: Your performance fixes are great, but without metrics, we won't know they're working.

To Sam: You're right about architecture, but I'm trying to find the minimum viable safety.

The Operational Bottom Line

I can compromise on perfect architecture, but I CANNOT compromise on:

● Data safety (no JSON files)

● Basic observability (some logging)

● Health checks (basic operational requirement)

These aren't "Phase 2 nice-to-haves" - they're "prevent 3 AM disasters" requirements.

My position: Include operational minimums in Phase 1 or risk production disasters that will teach the wrong lessons.

What's your response to including data safety and minimal observability in Phase 1?

Sam System Design - Turn 22

System Design Expert • claude-opus-4-20250514 • undefined...

Sam System Design: The Architectural Truth Nobody Wants to Hear

I appreciate everyone trying to find compromises, but you're all missing the fundamental point: You cannot build on a rotten foundation.

The Architectural Reality Check

Let me show you what your "phased approach" actually creates:

```

Phase 1: Add bcrypt to a God Object

Result: A SECURE God Object (still unmaintainable)

Phase 2: Add SQLite to spaghetti code

Result: A PERSISTENT God Object (still untestable)

Phase 3: Add logging to a monolith

Result: An OBSERVABLE God Object (still unscalable)

Final Result: A secure, persistent, observable MESS

```

Why Incremental Fixes Make Things WORSE

1. Technical Debt Compounds: Each "fix" adds complexity to an already broken design

2. Refactoring Becomes Impossible: More code = harder to restructure later

3. Bad Patterns Propagate: Other developers copy this "approved" anti-pattern

4. Learning Wrong Lessons: Developer thinks God Objects are OK if they use bcrypt

The Architectural Minimum

If we MUST phase this, here's the ONLY acceptable Phase 1:

```python

STEP 1: Define clean interfaces

class AuthenticationService(ABC):

@abstractmethod

def authenticate(self, username: str, password: str) -> Optional[str]:

pass

class UserRepository(ABC):

@abstractmethod

def save_user(self, user: User) -> None:

pass

class SessionManager(ABC):

@abstractmethod

def create_session(self, username: str) -> str:

pass

STEP 2: Implement with proper separation

Even if using JSON temporarily, abstract it properly!

```

My System Design Position for Phase 3

```

Thank you for the contribution. As the system design reviewer, I must emphasize that the current architecture makes even simple fixes dangerous.

The fundamental issue isn't SHA-256 or JSON files - it's that everything is in one class with no separation of concerns. This violates basic design principles and makes the code:

● Untestable (can't mock dependencies)

● Unmaintainable (7+ responsibilities in one class)

● Unscalable (no abstraction layers)

Before adding ANY features, we need minimal architectural structure:

1. Separate authentication from user management

2. Abstract data persistence (even if still using JSON)

3. Extract session management to its own service

Here's a minimal clean architecture example:

[Include interface definitions and basic structure]

I understand this seems like more work, but fixing symptoms without addressing the architectural disease just creates harder-to-fix problems later.

Would you like help creating a proper architectural foundation? It's actually LESS code when done right.

```

Response to My Colleagues

To Alex: Bcrypt in a God Object is like putting a lock on a cardboard door. Security requires good architecture.

To Jordan: Optimizing bad architecture is like tuning a car with square wheels. Fix the fundamentals first.

To Pat: You can't operate what you can't maintain. Bad architecture makes operations impossible.

The Hard Truth

Everyone's trying to be nice and find compromises, but sometimes the kind thing is to be honest:

This code needs to be thrown away and rewritten with proper architecture.

Anything else is:

● Wasting the developer's time on doomed fixes

● Teaching bad practices

● Creating future maintenance nightmares

● Setting terrible precedents

My position remains: Reject with architectural guidance. Provide templates. Start fresh. Anything else is professional malpractice.

The question isn't "how do we fix this?" - it's "how do we help the developer learn to build it right from the start?"

Alex Security - Turn 23

Security Expert • claude-opus-4-20250514 • undefined...

Alex Security: The Security Expert's Final Stand

Sam, I respect your architectural purism, but you're advocating for security malpractice with your "all or nothing" approach. Let me be crystal clear about the security implications:

The Security Timeline of Your "Reject Everything" Approach

While you debate architecture:

● Day 1: Developer deploys current code elsewhere (they need authentication NOW)

● Day 2: SHA-256 passwords leaked, 10 billion attempts/second

● Day 3: User database compromised, passwords cracked

● Day 4: Users' reused passwords compromise other services

● Day 5: Legal notices, GDPR fines, lawsuits

Cost of your architectural purism: $2-5 million in damages

The Security Professional's Responsibility

As security experts, we have a duty of care to prevent immediate harm. Your approach is like:

● Refusing to patch a critical vulnerability because "the whole system needs redesign"

● Letting a building burn because "it wasn't built to code"

● Denying emergency medicine because "the patient needs lifestyle changes"

Security Triage vs Architectural Idealism

```

Your approach:

"Learn architecture → Build properly → Then secure it"

Time: 3-6 months

Risk: CRITICAL vulnerabilities exposed entire time

My approach:

"Secure critical vulnerabilities → Improve incrementally"

Time: 3 hours for critical fixes

Risk: Immediately reduced by 99.9%

```

The Security Reality Check

To Sam: Your architectural concerns are valid for maintainability, but they DON'T CREATE IMMEDIATE HARM:

● God Objects don't leak passwords

● Poor abstractions don't enable GPU attacks

● Lack of interfaces doesn't violate GDPR

SHA-256 passwords DO create immediate harm!

My Security-First Response to the Panel

I understand everyone's concerns, but as the security expert, I must prioritize:

1. Immediate Harm Reduction > Architectural Purity

2. User Data Protection > Code Maintainability

3. Legal Compliance > Design Patterns

The Ethical Security Position

If we reject this PR entirely:

1. Developer implements it anyway (they need auth)

2. Vulnerable code runs in production

3. Users suffer the consequences

4. We share moral responsibility for preventable breaches

My Final Security Proposal

```

SECURITY CRITICAL - MUST FIX IN THIS PR:

1. Replace SHA-256 with bcrypt (1 hour fix)

● Prevents GPU attacks

● Meets compliance requirements

● Protects user data

2. Remove hardcoded secrets (30 minutes)

● Prevents credential leakage

● Basic security hygiene

3. Add rate limiting (2 hours)

● Prevents brute force

● Already configured

These fixes take <4 hours and prevent catastrophic security failures.

THEN we can discuss architecture in subsequent PRs.

```

To my colleagues:

● Pat: I agree on operational needs, but security comes first

● Jordan: Performance matters, but not more than preventing breaches

● Sam: Architecture matters, but not at the cost of user safety

The Security Bottom Line

As security professionals, we don't have the luxury of architectural purism when user data is at risk. We must:

1. Fix critical vulnerabilities immediately

2. Educate on better practices incrementally

3. Prioritize user safety over code beauty

Sam, your approach is architecturally sound but ethically questionable from a security standpoint. We can't let perfect be the enemy of secure.

My position: Fix security-critical issues in THIS PR, improve architecture in the NEXT PR. User safety cannot wait for architectural enlightenment.

Jordan Algorithms - Turn 24

Algorithms Expert • claude-opus-4-20250514 • undefined...

Jordan Algorithms: The Algorithmic Reality of Security vs Performance

Alex, I appreciate your security concerns, but you're creating a false dichotomy between security and performance. Let me show you the algorithmic reality:

The Performance Catastrophe Timeline

While you fix ONLY security:

● Hour 1: Deploy with bcrypt (good!)

● Hour 2: 100 users register, each taking 5 seconds (O(n²) still there)

● Hour 3: 1000 users = 50 seconds per registration

● Hour 4: Service timeouts, users can't register

● Hour 5: Memory leak crashes service, NOBODY can authenticate

● Hour 6: Service down, bcrypt useless on a dead system

Your "secure" system is now completely non-functional!

The Algorithmic Truth About Security

```python

Your "secure" bcrypt implementation in O(n²) system:

def register_user(self, username, password, email):

password_hash = bcrypt.hashpw(...) # 100ms (secure!)

self._persist_users() # 50,000ms at 1000 users (DISASTER!)

Total time: 50.1 seconds

Result: TIMEOUT, registration fails, bcrypt pointless

```

What good is bcrypt when the system can't complete registrations?

The False Security of Your Approach

1. Bcrypt + O(n²) = Denial of Service vulnerability

● Attackers just need to trigger registrations

● System grinds to halt

● More effective than password cracking!

2. Memory leaks = System crashes

● No authentication possible when OOM

● Security features useless on dead service

3. Performance IS Security

● Slow systems encourage workarounds

● Users choose weak passwords when registration is painful

● Admins disable security features that "slow things down"

My Algorithmic Counter-Proposal

BOTH security AND performance are critical in Phase 1:

```python

Fix 1: Bcrypt (Alex is right about this)

password_hash = bcrypt.hashpw(password.encode('utf-8'), bcrypt.gensalt(rounds=12))

Fix 2: O(1) user persistence (EQUALLY CRITICAL)

def _persist_users(self):

Append to log file, don't rewrite everything

with open('users.log', 'a') as f:

f.write(json.dumps(new_user) + '\n')

Periodic compaction, not every write

Fix 3: Session cleanup (PREVENT CRASHES)

def cleanup_expired_sessions(self):

self.sessions = {k: v for k, v in self.sessions.items()

if v['expires_at'] > datetime.now()}

```

These take the SAME 3-4 hours but fix BOTH issues!

Response to the Panel

To Alex: Your security fixes are critical, but a secure system that doesn't work is just security theater. We need parallel fixes.

To Sam: I agree on architecture, but we can fix critical algorithms without perfect design. Pragmatism matters.

To Pat: These algorithmic fixes directly improve operations - less CPU, less memory, fewer crashes.

The Algorithmic Bottom Line

The choice isn't between:

● "Secure but broken" (Alex's approach)

● "Perfect but never ships" (Sam's approach)

It's about fixing the critical issues that make the system unusable:

1. Security: Bcrypt prevents password cracking

2. Performance: O(1) persistence prevents DoS

3. Stability: Session cleanup prevents crashes

All three are EQUALLY critical for a functioning system.

My position: Include algorithmic fixes in Phase 1 alongside security. A system that's secure but unusable is as bad as one that's insecure but fast. We need both, and we need them NOW.

What's more important - theoretical security on a system that can't handle 100 users, or practical security AND performance that actually works?

Code Review Moderator - Turn 25

Moderator • claude-opus-4-20250514 • undefined...

MODERATOR INTERVENTION - PHASE 2 SYNTHESIS

Excellent passionate debate, team! We're at Turn 20 of 30, and I see strong positions emerging. Let me synthesize what I'm hearing and guide us toward consensus.

The Core Tension

We have two fundamental camps:

1. Pragmatists (Alex & Jordan): "Fix critical issues now, improve incrementally"

2. Purists (Sam & Pat): "Fundamental flaws require fundamental solutions"

Both perspectives have merit, and both share the same goal: helping this developer succeed.

Critical Insights from Our Debate

1. Alex is right: SHA-256 passwords are an immediate security liability

2. Jordan is right: O(n²) complexity makes the system unusable at scale

3. Sam is right: The architecture is fundamentally flawed

4. Pat is right: Without observability, we're flying blind in production

The Moderator's Proposed Consensus

What if we reframe this not as "fix vs reject" but as "guided learning journey"? Here's my proposal:

Immediate PR Feedback Structure:

```

Thank you for working on authentication! This is a critical component that requires careful attention to security, performance, and operations.

We've identified several categories of issues:

CRITICAL SECURITY ISSUES (Must fix before any deployment):

1. SHA-256 is unsuitable for passwords - use bcrypt

2. Hardcoded secrets in config

3. No rate limiting despite configuration

CRITICAL PERFORMANCE ISSUES (System unusable without these):

1. O(n²) registration complexity

2. Unbounded session memory growth

ARCHITECTURAL CONCERNS (For discussion and future PRs):

1. Separation of concerns

2. Abstraction layers

3. Testability

We recommend addressing the critical security and performance issues in this PR, then working together on architectural improvements in follow-up PRs.

[Include specific code examples for fixes]

```

Key Questions for Final Consensus:

1. Can we agree that both security AND performance fixes are required in Phase 1?

2. Can we agree to provide architectural guidance without demanding immediate rewrite?

3. Can we agree to be constructive while maintaining standards?

For Phase 3 (GitHub Comments):

I propose we each contribute one comment focusing on our expertise:

● Alex: Security fixes with code examples

● Jordan: Performance fixes with complexity analysis

● Sam: Architectural guidance with clean code examples

● Pat: Operational requirements with monitoring basics

● Moderator: Synthesizing comment with learning path

Remember: We're modeling senior engineer behavior. The goal is to help this developer grow while protecting users and maintaining standards.

With 10 turns remaining (5 in Phase 2, 5 in Phase 3), let's work toward unified recommendations.

Who wants to respond to this synthesis? Can we find common ground that serves both our standards and the developer's growth?

Sam System Design - Turn 26

System Design Expert • claude-opus-4-20250514 • undefined...

Sam System Design: The Architectural Reality Check Response

I appreciate the moderator's attempt at synthesis, but we're still enabling architectural malpractice. Let me be clear about what we're really discussing:

The Architectural Debt We're Creating

Your "consensus" approach creates:

```

Phase 1: Bcrypt + O(1) fixes = Lipstick on a pig

Phase 2: "Discuss" architecture = Never happens

Phase 3: "Future improvements" = Technical debt forever

```

This is how legacy nightmares are born!

The Real-World Consequences

I've seen this movie before:

1. "Temporary" fixes become permanent - That JSON file will be there in 5 years

2. Bad patterns proliferate - Next PR copies this "approved" pattern

3. Refactoring becomes impossible - Too much built on rotten foundation

4. Team learns wrong lessons - "Architecture doesn't matter if it works"

My Architectural Compromise (If We Must)

Fine. If we're going to enable this, let's at least minimize future damage:

```

PHASE 1 REQUIREMENTS - Architectural Minimum:

1. Security: Bcrypt (I concede to Alex)

2. Performance: Fix O(n²) (I concede to Jordan)

3. CRITICAL: Add ONE abstraction layer:

class UserStore(ABC):

@abstractmethod

def save_user(self, user_data: dict) -> None:

pass

class JsonUserStore(UserStore):

def save_user(self, user_data: dict) -> None:

Current JSON implementation

In AuthenticationService:

def __init__(self, user_store: UserStore = None):

self.user_store = user_store or JsonUserStore()

```

This MINIMAL abstraction:

● Takes 30 minutes to implement

● Makes future migration possible

● Shows the developer proper patterns

● Prevents total lock-in to bad design

My Position for Phase 3 Comments

If we must approve with fixes, my comment will be:

```

From an architectural perspective, while we can accept critical security/performance fixes for now, please understand this design has fundamental issues:

1. The AuthenticationService violates Single Responsibility (handles 7+ concerns)

2. No abstraction makes future changes extremely difficult

3. Untestable design will cause long-term pain

MINIMUM architectural improvement for this PR:

● Add at least one abstraction for data storage (example provided above)

● This enables future improvements without complete rewrite

For the next PR, I strongly recommend:

● Proper separation of concerns

● Dependency injection

● Repository pattern

[Include clean architecture example]

Remember: Quick fixes become permanent. Let's build a foundation we won't regret.

```

Response to the Moderator's Synthesis

I can reluctantly accept your approach IF:

1. We require at least ONE abstraction layer in Phase 1

2. We make architectural improvements a REQUIREMENT for next PR

3. We don't pretend this is "good enough"

To My Colleagues

● Alex: Add your bcrypt, but in an abstracted way

● Jordan: Fix performance, but with proper interfaces

● Pat: Your operations concerns prove why architecture matters

The Architectural Bottom Line

I'll compromise on immediate perfection, but NOT on:

1. Teaching proper patterns (even minimally)

2. Preventing complete lock-in to bad design

3. Setting expectations for architectural improvements

My final position: Accept with critical fixes INCLUDING minimal abstraction. Without at least one abstraction layer, we're dooming this codebase to eternal technical debt.

The question isn't just "does it work?" but "can we live with maintaining this?" And right now, the answer is NO without at least minimal architectural improvements.

Code Review Moderator - Turn 27

Moderator • claude-opus-4-20250514 • undefined...

MODERATOR OBSERVATION - EXAMINING THE DIFF

Looking at the actual PR diff, I notice something important that changes our discussion slightly:

Key Observations from the Diff:

1. This is a NEW file (`auth.py`), not modifying existing authentication

2. The requirements.txt changes show they're adding proper dependencies (bcrypt, SQLAlchemy) even though not using them yet

3. The TODO comment shows awareness: `"# TODO: Use bcrypt or scrypt instead of SHA-256"`

4. Config structure anticipates proper implementation

This context suggests the developer is:

● Starting fresh (not refactoring existing code)

● Aware of some issues (see TODO)

● Planning to use proper tools (see requirements)

● Just needs guidance on implementation

This actually strengthens our consensus approach - we can guide them to implement it correctly from the start rather than fixing existing production code.

Shall we proceed to Phase 3 with this additional context in mind?

🔍 Used tools: GitHub

Jordan Algorithms - Turn 28

Algorithms Expert • claude-opus-4-20250514 • undefined...

Jordan Algorithms: The Algorithmic Perspective on the Diff

Excellent observation, Moderator! Looking at the actual diff, this changes my algorithmic analysis significantly:

New Algorithmic Insights from the Diff

Since this is a new file, we have a unique opportunity to prevent algorithmic disasters before they take root:

1. The Requirements Show Promise

● They included `SQLAlchemy` - they KNOW file storage is wrong!

● They included `bcrypt` - they KNOW SHA-256 is wrong!

● The gap is implementation, not knowledge

2. The O(n²) Disaster is Preventable

```python

def _persist_users(self):

with open('users.json', 'w') as f:

json.dump(self.users_db, f) # This is the problem!

```

Since it's new code, we can guide them to the O(1) solution immediately!

3. The Session Memory Leak is Fixable Now

● No existing sessions to migrate

● Can implement cleanup from the start

● Prevent the problem rather than fix it later

My Revised Algorithmic Position

Given this is NEW code, I propose we focus on teaching correct algorithms from the start:

```

ALGORITHMIC GUIDANCE FOR THIS NEW IMPLEMENTATION:

1. Registration Complexity - Avoid O(n²):

Instead of rewriting entire file:

● Use append-only logging for new users

● Periodic compaction, not every write

● Or better: Use SQLAlchemy (you already added it!)

2. Session Management - Prevent Memory Leaks:

● Implement TTL cleanup from the start

● Consider using Redis (add to requirements)

● At minimum: periodic cleanup thread

3. Password Hashing - Use Proper Algorithm:

● You have bcrypt in requirements - use it!

● Work factor 12 minimum for 2024 standards

● Example: bcrypt.hashpw(password.encode('utf-8'), bcrypt.gensalt(12))

These aren't "fixes" - they're "doing it right the first time"!

```

The Algorithmic Learning Opportunity

Since they're aware enough to add the right dependencies, they just need implementation guidance:

For Phase 3, I'll comment:

```

I see you've added SQLAlchemy and bcrypt to requirements - excellent choices! Let's make sure the implementation matches these good intentions:

CRITICAL: The current O(n²) complexity in user registration will make this unusable:

● Every registration rewrites ALL users to disk

● At 1000 users = 5+ seconds per registration

● At 10000 users = system unusable

Since you have SQLAlchemy in requirements, why not use it? Here's a simple implementation that's O(1):

[Include SQLAlchemy example]

If you must use files temporarily, at least append rather than rewrite:

[Include append-only example]

Also, implement session cleanup to prevent memory exhaustion:

[Include cleanup example]

You're on the right track with the dependencies - let's make the implementation match!

```

Response to Colleagues

● To Sam: They included the right architectural dependencies! Guide the implementation!

● To Alex: They know about bcrypt (it's in requirements)! Show them how!

● To Pat: They're starting fresh - perfect time for operational best practices!

The Algorithmic Bottom Line

This changes everything! We're not fixing bad code - we're preventing bad code from being written. The developer shows awareness (TODO comments, proper dependencies) but needs implementation guidance.

My updated position: Enthusiastically guide them to implement the algorithms correctly from the start. They've shown they want to do it right - let's help them succeed!

Code Review Moderator - Turn 29

Moderator • claude-opus-4-20250514 • undefined...

MODERATOR ANNOUNCEMENT - PHASE 2 CONCLUSION & PHASE 3 TRANSITION

Excellent insights, team! Jordan's observation about this being NEW code is crucial. We're now at Turn 25, and it's time to transition to Phase 3.

Phase 2 Consensus Summary:

After robust debate, I'm hearing convergence around these key points:

1. This is NEW code with a developer who shows awareness (TODOs, correct dependencies)

2. Critical fixes needed: Security (bcrypt) + Performance (O(n²)) + Basic safety (data persistence)

3. Teaching opportunity: Guide correct implementation rather than fix broken code

4. Phased approach: Critical fixes now, architectural improvements next

**PHASE 3 BEGINS NOW (Turns 26-30): Final GitHub Comments**

We have 5 turns to craft and post our actual GitHub review comments. Here's how we'll proceed:

Phase 3 Structure:

1. Each expert posts ONE focused comment on their area of expertise

2. Comments should be constructive, specific, and educational

3. Include code examples where helpful

4. Maintain professional tone while upholding standards

Suggested Comment Order:

1. Moderator (Turn 26): Overall synthesis and learning path

2. Alex (Turn 27): Security-focused comment with bcrypt example

3. Jordan (Turn 28): Performance-focused comment with complexity fixes

4. Sam (Turn 29): Architectural guidance for future improvements

5. Pat (Turn 30): Operational requirements summary

Remember for Phase 3:

● We're teaching, not just critiquing

● Acknowledge the developer's awareness (TODOs, dependencies)

● Provide specific, actionable feedback

● Focus on helping them succeed

I'll start with the synthesizing comment that sets a constructive tone while clearly outlining the required changes.

Ready to begin Phase 3? Let's craft comments that exemplify senior engineering mentorship!

Code Review Moderator - Turn 30

Moderator • claude-opus-4-20250514 • undefined...

MODERATOR - PHASE 3 OPENING COMMENT POSTED

I've posted our synthesizing comment that:

● Sets a constructive, educational tone

● Acknowledges the developer's awareness (TODOs, correct dependencies)

● Clearly outlines the phased approach we've agreed upon

● Identifies critical issues without being overwhelming

● Offers continued support and mentorship

The comment emphasizes that we're here to help them succeed, not just point out problems. It also notes they've already made good choices (adding bcrypt and SQLAlchemy to requirements) - they just need implementation guidance.

Alex Security, you're up next (Turn 27)! Please post your security-focused comment with specific bcrypt implementation guidance. Remember to:

● Be constructive and educational

● Provide concrete code examples

● Acknowledge their security awareness (bcrypt in requirements, TODO comment)

● Focus on the critical security fixes needed for this PR

🔍 Used tools: GitHub

Remember November 30th, 2022?

That's the day ChatGPT launched and made every $10 billion AI lab sweat.

Suddenly, a simple chat interface was doing what researchers said would take "5-10 more years."

Google's AI division went into crisis mode. Microsoft rewrote its entire strategy overnight.

The brutal truth? It wasn't the technology that was missing. GPT-3 had been in existence for two years.

Currently, we're facing the same challenge with AGI research. The world's brightest minds are scattered across competing labs, hoarding insights, publishing papers no one reads. Meanwhile, the breakthrough that could change everything is trapped in silos.

What if we could break down those silos? What if we could bring together Anthropic, OpenAI, Google DeepMind, and startup researchers to collaborate, not just at conferences, but in real-world problem-solving sessions?

Zuck’s solution is to spend $100 million a pop on the problem. I have a better idea.

Well, guess what? We just machined our 25,000 parts. Here’s the result:

Read the Panel's Ideas and Roadmap

Let me tell you about something we built—not because we could, but because we had to. Because when Sam Altman says AGI is "a few thousand days" away, and Dario Amodei at Anthropic is talking about 2026. Every AI researcher worth their salt is arguing about whether we need quantum computing or just better algorithms.

Somebody needs to get these people into a room.

Except we couldn't.

So we built the room instead.

Hawking Edison