Two AI Systems, Twenty Ideas, One Real Lesson

Two days ago we ran a structured brainstorm. Two AI systems, working from different starting points, generated about twenty new capability ideas for the same problem space.

The twenty ideas weren't the interesting part.

Is this actually true, or does it just sound true?

Almost everything that mattered over the next two days came down to one question: is this actually true, or does it just sound true?

We found a table our own planning documents said existed. It didn't. It had been built under a different name months earlier, and nobody had gone back to fix the reference.

We found a data column a new feature's design assumed would be populated. It wasn't. Not once, anywhere in the dataset.

We found a test dataset that looked current and was actually weeks out of date relative to the real evidence it was supposed to be checked against.

None of these were dramatic. They're the completely normal gaps that accumulate in any fast-moving system. The only thing that mattered was catching them before they shipped — as a default, not an exception.

The most useful moment wasn't agreement

The most useful moment of the whole exercise wasn't when the two systems agreed. It was when they didn't.

One proposal, generated independently, would have quietly crossed a line we'd deliberately drawn around our own product months earlier. Not a preference — a principle about what we will and won't build. It got caught, in plain language, before a single line of code existed, because we checked the new idea against the old commitment instead of assuming a good-sounding idea is automatically a safe one.

That's the real takeaway. Running two independent perspectives side by side isn't valuable because they agree. It's valuable because of what shows up when they don't.

What's still hard

The part of building with AI that used to be hard — generating ideas fast, writing code fast — isn't the hard part anymore. Any capable system does that on demand now.

What's still hard, and what actually separates good work from fast work, is the boring discipline of treating "this is documented" and "this is true" as two different claims, every time, and reconciling them before you trust either one.

We weren't the only ones learning this

At VB Transform this week, Intuit's VP of Applied AI described scrapping their internal agent architecture twice in four months — because errors compound silently every time agents hand work off to each other, and nobody was independently checking.

Different company. Different scale. Same root lesson: speed without an independent check is just a faster way to ship a quiet failure.

The discipline we're building for

Not more ideas. Not more velocity.

Receipts that survive contact with reality.