Data Has Black Holes Too: Why “hallucination” is the wrong word for AI’s deepest failure mode.

The AI industry calls every wrong answer a “hallucination.” That word is hiding a much bigger problem.

When a model fabricates a seahorse emoji, that’s obvious and fixable. When a model produces a confident, well-structured answer that’s wrong because of assumptions buried in the training data it was never designed to question, that’s something else entirely. That’s structural. And nobody’s talking about it in the right terms.

I fed the same degraded 1957 film image to three frontier models. All three independently produced WWII propaganda. The only correct result came after I supplied the actual movie context up front.

The new essay is about what’s really happening inside these systems, why your 500-word prompts are fighting a losing battle against gravity, and how to stop fighting the landscape and start navigating it.

(Read, The Room Where It Gets Built — Essay #9: Data Has Black Holes Too)
 

As technology leaders, our most critical job is understanding the actual architecture of the tools we buy. If we evaluate probabilistic AI models using the same procurement mindset we use for enterprise software, we expose our organizations to catastrophic, invisible risks. I looked at the recent DoD/Anthropic negotiations as a case study in how dangerous this category error can be.

(Read, The Room Where It Gets Built — Essay #8: LLMs Are Not Shelf-Stable Products)