I configured my server by the book. Ten AI models still couldn’t read my documents.

I set up a technical documentation bundle: eight markdown files, a PDF, a diagram source, and an llms.txt manifest with absolute URLs, token estimates, and a reading order. Correct MIME types. Validated headers. Clean UTF-8. Every URL resolves.

Then I asked ten commercial AI models, including frontier-class, to fetch the manifest and read the documents. Same server, same files, same day and none of them succeeded cleanly. Believe it or not, this was a step up from having them ingest from an HTML landing page with direct file links!

But, two couldn’t reach the server at all (blocked by their own infrastructure, not mine, although they both blamed my server). Two rejected every markdown file because their fetchers don’t accept text/markdown. The correct configuration makes content less accessible than serving the wrong type.

Four silently truncated the largest document. One stopped at 32 KB (13%). Another at 120K characters (49%). None knew they had partial copies with one exception: Claude Opus 5.1 (High) told me its parser only retained the first and last thirds of my document and asked me to resubmit. That’s the behavior I want to see but doesn’t help much in an unattended mode.

Four served cached versions despite headers requiring revalidation. Three models returned three different stale versions of the same file on the same day. Every one blamed my server. There was definitely a trend to blame me or my infrastructure when things failed and I’m not pleased with that. Either the models don’t have full visibility to their failure modes or they’re just hallucinating. The latter is probably the more dangerous though.

One decoded UTF-8 as Latin-1. Every diacritic destroyed. Not critical but still irritating. I’ve fixed that on my server to enforce decoding in the future.

One fabricated a finding… reported a file that doesn’t exist, cited footers that don’t reference it, and presented invented observations alongside real ones at identical confidence. This set off a scramble to figure out if a stray error had gotten into the text somehow, but it ended up being a confabulation based on a phrasing expanded to imagined content.

The dangerous failures were silent. Truncation, caching, encoding corruption, and fabrication produced no errors. The model answers confidently from whatever it retrieved, and nothing in the output tells you retrieval failed.

The industry sells “give it a URL” as a core capability. For most models on most documents, it isn’t as reliable as opening the link in a browser. And you can’t tell from inside the conversation unless you’re actively probing secondary to whatever it is you’re actually trying to get done.

If you’re building an llms.txt: the format works. The implementations don’t. Plan for rejected MIME types, ignored headers, silent truncation, and stale caches. Verify independently and across models.

I’ve been talking with smaller construction firms at local meetups lately, and the energy around AI is genuinely exciting. Owners are automating proposals, generating scripts, pulling data out of documents faster than ever. The enthusiasm is real, and a lot of what they’re building is legitimately impressive.

But I keep noticing the same gap, and it’s one I’ve seen play out time and again over the years.

There’s a meaningful difference between ”an AI can write a script” or “configure your settings” and “an AI can architect, secure, and maintain a production environment.” That’s not a knock on anyone’s intelligence or the tools. It’s just that most smaller firms have never needed to think at the enterprise level before and any professional IT experience they’ve encountered was probably on the user side of someone else’s environment. AI is quietly pushing them into territory that looks and feels genuinely good.

What I’m watching happen though is firms handing the keys for a sports car to an algorithm that doesn’t know what a brake pedal is, simply because the car goes fast.

Here’s what that looks like in practice:

When you swap reliable, deterministic software for a chat-based AI workflow, you’re introducing a system that can fail in ways that don’t look like failures. It doesn’t throw an error code. It just confidently gives you the wrong number on a bid, or silently drops a compliance document, or racks up compute costs that don’t show up until the invoice hits. And if nobody on the team knows what a failure state looks like in that stack, nobody catches it.

The owners I talk to think they’re saving on headcount. What they’re actually building is invisible technical debt. The kind of debt that can surface as a legal exposure or a cost blowout on exactly the wrong project.

And I get the appeal. These tools are incredible. They’re validating in a way that most software never has been. They give you a sense of command over problems that used to feel out of reach. That feeling is real! But it’s not the same thing as a sound architecture.

AI is a fantastic tool for generating code, pre-visualization, drafting documents, and accelerating work you already understand. It falls apart when you ask it to be the architect, especially when nobody on the team actually knows what an enterprise IT architecture even looks like, let alone how to maintain one. That’s when you lose the person in the loop who knows when to step in, redirect, or just say “no, we’re doing this the boring reliable way.”

So if you’re a construction firm leaning hard into AI right now, here’s my honest ask: get a technical review. Not from your AI vendor. From someone who’s built and maintained production infrastructure. Have them look at where your AI tools touch your actual operations… your bids, your compliance docs, your financials, and map out where a silent failure would hurt you most.

You don’t need to slow down. You just need to know what you’re standing on.