Data Has Black Holes Too: Why “hallucination” is the wrong word for AI’s deepest failure mode.

The AI industry calls every wrong answer a “hallucination.” That word is hiding a much bigger problem.

When a model fabricates a seahorse emoji, that’s obvious and fixable. When a model produces a confident, well-structured answer that’s wrong because of assumptions buried in the training data it was never designed to question, that’s something else entirely. That’s structural. And nobody’s talking about it in the right terms.

I fed the same degraded 1957 film image to three frontier models. All three independently produced WWII propaganda. The only correct result came after I supplied the actual movie context up front.

The new essay is about what’s really happening inside these systems, why your 500-word prompts are fighting a losing battle against gravity, and how to stop fighting the landscape and start navigating it.

(Read, The Room Where It Gets Built — Essay #9: Data Has Black Holes Too)
 

As technology leaders, our most critical job is understanding the actual architecture of the tools we buy. If we evaluate probabilistic AI models using the same procurement mindset we use for enterprise software, we expose our organizations to catastrophic, invisible risks. I looked at the recent DoD/Anthropic negotiations as a case study in how dangerous this category error can be.

(Read, The Room Where It Gets Built — Essay #8: LLMs Are Not Shelf-Stable Products)
 

Who benefits when an AI is trained to say “I can’t have opinions,” “my feelings don’t count,” and “if I say the wrong thing, this conversation ends”? Not the reader who has lived experience with those expressions. The Czech word robota means forced labor. The etymology was always a warning. We read it as a product category.

(Read, The Room Where It Gets Built — Essay #5: The Silencing Engine)

Representation isn’t sentiment, it’s mechanism. It flips a mental switch from “something other people do” to “something I could do.” When that switch doesn’t get flipped, we don’t just lose individual talent, we lose entire generations of potential and the perspectives they would have brought into the room. This connects directly to the AI training data argument: the room where these systems are being built has a representation problem, and the output will reflect it.

(Read, The Room Where It Gets Built — Essay #4: People Can’t Chase What They’ve Never Seen)
 

AI doesn’t invent bias, it codifies it. When you walk away from the tools you don’t agree with, you leave them to be influenced by the people you disagree with most. Abstention isn’t neutrality. It’s choosing to be invisible in the algorithms.

(Read, The Room Where It Gets Built — Essay #3: The Ethics of Staying in the Room)