I recently began doing Generative Engine Optimization (GEO) work for a client whose setup was a Linux VM sitting over my own Windows desktop — a clever, deliberate policy-, tech-, and security-layer stacked atop my personal environment that I genuinely appreciate. Along with it came an org-owned Claude Cowork account.
That last part is the most important, because I was instantly sold on how powerful Cowork can be if used properly. Until I noticed (well, more like confirmed, anecdotally) something.
I could do more with Cowork but finish less.
I don’t know exactly how everyone else on our shared local directory works. But between what I hear secondhand and what’s been surfacing in the trade press about agentic, multi-tool, multi-access, multi-capability, multi-everything genAI, one thing has become clear.
For most people, being handed something like Cowork is a lot like being handed a do-anything card with no statement of account attached.
You can feel the power the moment you swipe it. You don’t feel the balance until someone else does the arithmetic.
It isn’t limited to individuals, either. Large organizations with equally large budgets have been quietly running the same experiment, at a scale where the arithmetic becomes newsworthy.
One unnamed company reportedly burned through roughly $500 million in Claude credits in a single month after failing to put usage guardrails in place [1].
Amazon’s own internal engineers reportedly logged a smaller but stranger case: a routine task — matching author names to product listings — that quietly ran 860% over its allocated budget, generating $1.8 million in charges before anyone noticed, a full five months later. Engineers reportedly described the pattern as turning what used to be “trivially cheap” mistakes into something else entirely once an AI agent was doing the work [2].
One could argue the do-more-but-finish-less problem and the overspending problem are distinct — one’s a workflow story, the other’s a finance story. They’re not unrelated. They’re the same story, told from two different desks.
With Great Power Comes Great Scope Creep
If you can suddenly do five times more with a shiny new tool — don’t.
Not without some proper administration. Call it workflow governance, if you want the term that survives a slide deck.
The knowledge work I was already bolstering with free or individually-metered genAI, before Cowork, expanded the moment the tool got bigger. The tasks didn’t change. But the cost of trying felt negligible.
A subtask that would once have earned a “not now, maybe later” gets done anyway, because it’s fast, because it’s cheap, because the tool is right there and idle capacity feels like waste if you don’t use it. I’ve definitely adopted a habit of making sure an AI agent is doing something while I sleep — I mean it’s already earmarked usage costs!
The task’s actual definition of “done” doesn’t move.
What moves is everything orbiting it — the nice-to-haves, the “while we’re at it”s, the polish nobody asked for on a deliverable that was already finished three steps ago. And I was already having a hard time on single-minded “perfectionism” because of my ADHD, but anyway.
But just because trying something no longer registered as a cost at all doesn’t mean there’s no overhead. Every one of those additions still needs a human to look at it. Human post-verification doesn’t scale down just because the generation got faster — it’s the opposite.
If you added five more subtasks because they were nearly free to produce, you now owe five more (admittedly smaller) rounds of review before you can call any of it trustworthy. Quality goes up. So does the bill for checking it.
Those aren’t the same bill, of course, and conflating them is exactly how a promising four-hour task turns into an eight-hour one that feels more productive the entire way through.
And that’s before the meter even gets involved.
Major providers like OpenAI, Anthropic, and Google currently enforce a five-hour rolling usage window stacked on top of a separate weekly cap. Anthropic reportedly added the weekly ceiling specifically because the five-hour window alone had become too easy to route around by spreading heavy use across a longer day [3].
If your organization doesn’t cap usage above that, you get the Amazon-shaped outcome. If it does, you get a new, quieter kind of overhead: constantly asking yourself whether this particular task is worth spending your five-hour allowance on, which is project management wearing a different hat — workflow governance, as we used above. And nobody put it on the org chart.
Some companies solved this the way companies solve things when they’d rather have a number than a conversation: leaderboards.
Track who’s burned the most tokens, reward the top of the list, call it adoption. Amazon, JPMorgan, Meta, and Disney have all reportedly run internal AI usage leaderboards of this kind:
- In one case, a single Disney employee reportedly interacted with Claude 460,000 times over nine days [4].
- Meta’s internal dashboard reportedly went so far as to hand out tiers, “Token Legend” among them, for the heaviest users [5].
- Uber’s engineers reportedly burned through the company’s entire annual AI budget in about four months; its COO later told colleagues the relationship between rising token usage and any actual gain in useful output hadn’t been established [6].
- IBM’s own SVP of consulting put the underlying mechanism plainly: absent a real metric, organizations built usage leaderboards, and employees quickly learned how to game them [7].
None of this is a training problem. It’s the same mistake as measuring programmer output in lines of code — you get more lines, not more software. Ironically developers are obsessed with lean code and optimized execution time (well, at least in principle). Their Big O notation of code execution tries to make sure code is engineered in a way that runtime is at least linear (if not logarithmic or if you’re coding a small project: constant); runtime shouldn’t quadratically or exponentially increase proportional to input.
Yet these token leaderboards were doing the exact opposite.
Tokens spent is not value created. Like developers, the more experienced genAI users try to skirt around token use and usage limits like they would input context windows; they want less of it, not more. Tokens spent is not value created. Volume isn’t as a proxy for maturity.
If you run an org and you’re capping AI usage but not capping the temptation to do more just because it’s there, the cost might not be bleeding dollars on the invoice. The cost is that nobody — not the person doing the task, not the person reviewing it, not the CFO reading the monthly total — can actually see where the number came from. That’s the same gap, wearing a dollar sign instead of a clock.
At the Mercy of the Machine
The meter you can’t see is just one problem. The thing you can’t see your LLM agent do is a different, arguably worse one.
Rely on an agentic tool heavily enough and you’re at the mercy of its technical limitations, whether or not you notice. Retrieval limitations are a frustrating favorite of mine.
Depending on what you’re using, you might be pulling from a live cache, a stale one, or something claiming to be current that hasn’t been for a while. Perplexity’s own citation pipeline sorts every URL it touches into a small taxonomy — live, dead-but-archived, or apparently fabricated because nothing about it can be found anywhere, ever [8]. That’s just hallucination with a longer alias : a citation that resolves to nothing, anywhere, is functionally indistinguishable from something the model simply invented and dressed up as sourced.
Even when a citation resolves, it doesn’t mean the content behind it says what the model claims it says.
Researchers have described a distinct failure mode where the retrieved source is completely real — the page loads, the author is credible — but the footnote pointing to it has drifted to the wrong place entirely, a kind of second-hand hallucination sitting one layer removed from outright fabrication [9]. This isn’t a Perplexity-specific quirk; a recent academic study found that citation hallucination persists even inside retrieval-grounded systems generally, not just in models working purely from memory [10].
Then there’s the content that’s simply never retrieved in full.
Feeding a model an entire long document costs input context, so systems built for deep research frequently return truncated versions of what they find. It’s a documented trade-off, not an accident, and one that can quietly drop the exact section that mattered [11]. You’ve likely lived the milder version of this yourself: a thirty-page report handed to an LLM assistant, and the assistant returns two or three data points from the first few and last few pages, as if the other twenty or so in-between were mostly filler.
Robots.txt exclusions add a second, adjacent layer. Most major AI crawlers do respect these files, which means a restrictive rule written years before an AI crawler existed can quietly wall it out today, with the silence built directly into the mechanism. The only symptom is invisibility in a channel nobody happened to be measuring [12].
I’ve built a small personal workaround for this (and many others of a similar flavor): if a source looks good but the tool can’t or won’t use it, I pull the content manually — copy, paste, keep the URL attached — rather than let it get quietly dropped. Certainly not elegant. Certainly works better, though. Mostly because I’m the one watching for it (and I really shouldn’t be spending time copy+pasting).
Not everyone is watching for it, and not every failure announces itself as cleanly as a blocked crawl (my Claude Cowork account is certainly guilty of just quietly dropping a great source because there are others). A year or so ago, on an earlier research tool, we were building profiles of a list of social media figures — a listicle format, “fifteen people who,” that kind of thing. The task was person number six. What came back was accurate, well-written, and entirely real — a paragraph that genuinely existed on the page, describing something that genuinely happened. It had simply been filed under person number seven instead.
The retrieval worked. The attribution didn’t. Nothing about the output looked wrong until someone who already knew better happened to check — and luckily I did before it went to the client.
This isn’t free-trial-level hallucination. It’s more like pro-plan hallucination. There were no invented facts; just misattributed ones. Very close to human error, really.
Not every part of this story is bad news. Different models genuinely have different strengths, trained on different data with different blind spots. And there’s real, published value in running the same question across more than one of them rather than trusting a single answer (often called “ensembling” models; sort of like botsourcing instead of crowdsourcing). So it’s closer to a panel of different-strength evaluators than a single generalist, and this panel’s results consistently outperform any one model working alone [13].
If there’s a lesson buried in the retrieval mess, it’s not “don’t trust any of it.” It’s “don’t trust only one of it.” And really, the standard that hasn’t changed: “trust only yourself (or your org’s SME).”
The Tool That Can’t Fix Itself
Retrieval limitation amounts to just one kind of blindness. There’s a second kind that’s stranger, because it isn’t about the outside world at all — it’s about the tool losing track of its own instructions.
I keep a daily log — one running thread I return to throughout the day, meant to be updated once, at the end of it. I also have a personnel file that’s org-wide that I instruct Claude to update at the end of the day as well. I’ve told Cowork this directly. It’s in memory instructions, in the system prompt, in a file it’s supposed to check every time.
And for several days running, it kept updating the file mid-thread anyway, against its own standing instructions, and when asked about it (repeatedly), it admitted — genuinely, not performatively (it’s tried to fix it several times through the supposedly appropriate means) — that it didn’t know how to stop.
Turns out this has a name and a paper trail. A filed bug report describes almost exactly this mechanism: global instructions get stored in the same configuration file as a pile of unrelated internal app state, each active session holds its own in-memory snapshot of that file from the moment it started, and any session writing its internal housekeeping back to disk does so using its now-stale copy — silently overwriting whatever you saved in the meantime. The system reportedly even logs that a conflict happened. It just doesn’t stop the write [14].
A related report describes the same failure from a different angle: simply reopening an old session can quietly revert your current global instructions to whatever that older session remembered, with no notification that anything changed at all [15].
It might not be my problem exactly, but it certainly seems to be the shape of it.
When you’re playing chess, in a position that looks stable, sometimes it’s not the pieces in the center of the board that decide the outcome. It’s the pawn nobody’s been watching, three files over, that’s quietly become the one thing holding the whole structure together. The person who doesn’t know how to mount a Drive folder, doesn’t know what “context window” means, doesn’t know their assistant has no memory between sessions by default — at the risk of overextending the chess metaphor — that person is that pawn.
I caught this bug because I already knew to look for it. Most people using these tools don’t know there’s anything to look for. That means the failure isn’t just invisible to them; it’s a category they don’t yet know exists.
From One Desk to the Whole Building
Zoom out once, and this is a story about a careful person managing an expensive tool.
Zoom out twice, and it’s a story about organizations discovering, nine figures at a time, that nobody had been watching the meter either. Worse: the meter doesn’t really show the leak.
Zoom out a third time, and something changes in kind, not just in size: this stops being a cost-management problem at all. It becomes a question of whether an organization can meaningfully govern work it has delegated to something it cannot fully audit, mid-task, in real time. And it’s not because nobody cares, but because the tool itself often can’t tell you what it’s doing wrong, even when you ask it directly and it tries to answer “honestly.”
That’s a different kind of problem than “the bill was too high.”
A high bill gets a budget line. A tool that can’t see its own failure mode gets neither a budget line nor a fix, until someone vigilant enough happens to notice, and writes it up, and files it somewhere a future user might eventually find. Like every useful StackExchange or Reddit thread ever.
Capability Without Visibility
I don’t think the promise of these tools is fake.
I think the promise was made much sooner than the instrumentation needed to trust it came into fruition. And in that gap, not any single bug or any single invoice, is where we live right now. Every version of the pattern above is the same shaped gap wearing different clothes: a capability granted well ahead of the visibility needed to govern it.
Cost visibility.
Retrieval visibility.
Session-state visibility.
Take your pick of domain; the structure repeats.
There’s a real, honest case that this is temporary. That it’s a maturity ceiling we haven’t hit yet, on the far side of which these tools genuinely do get cheaper and more legible than the humans they’re replacing for low-value work. I still believe that’s at least the direction this ship is sailing. But we’re traversing a gap that’s arguably growing wider even as we brave it.
A visibility gap. And visibility begets governance. And that visibility we need is getting buried under the capability we’re being given.
FAQs
The visibility gap is the difference between what modern AI systems can do and what users or organizations can actually observe while those systems are working. AI capabilities have advanced faster than the tools for auditing costs, retrieval behavior, memory state, and reasoning processes.
AI dramatically reduces the cost of generating drafts, analyses, and ideas, but every additional output still requires human review. As generation becomes cheaper, scope expands. The work shifts from producing content to verifying, selecting, and governing it.
Not necessarily. High token consumption measures activity, not value. Like measuring programmers by lines of code, rewarding token usage can incentivize inefficient behavior rather than better outcomes.
Modern AI systems often retrieve real sources but may misattribute them, truncate important sections, or cite content that doesn’t fully support their conclusions. These failures frequently appear convincing until someone independently verifies the original material.
More capable systems amplify both productivity and hidden risk. Organizations increasingly need governance around workflows, verification, spending, retrieval quality, and operational visibility—not because AI is outright unreliable, but because powerful systems require equally powerful oversight.
Sources
Generative AI Disclaimer:
This article was written with the assistance of generative AI, and reviewed with full human post-editing. It uses the proprietary frameworks of SupraGraphos and went through reviews with the bylined author. Generative AI was used to speed up research and synthesis.
- Android Authority, “A company spent $500 million in one month after forgetting to set AI usage limits” — https://www.androidauthority.com/skyrocketing-enterprise-ai-costs-3672603/
- The Next Web, “Amazon’s $1.8m Claude blunder shows AI’s runaway costs” — https://thenextweb.com/news/amazon-catastrophically-expensive-ai-cost-overruns-claude
- tokenkarma.app, “Claude Usage Limit Explained 2026” — https://tokenkarma.app/blog/claude-usage-limit-explained-2026/
- CIO, “Tokenmaxxing: When AI adoption metrics go bad” — https://www.cio.com/article/4178320/tokenmaxxing-when-ai-adoption-metrics-go-bad.html
- The National CIO Review, “Token Spend Is the New Lines of Code” — https://nationalcioreview.com/articles-insights/leadership/token-spend-is-the-new-lines-of-code/
- gHacks Tech News, “Leaked Amazon Documents Detail $1.8 Million Overrun on a Single Claude AI Task Missed for Five Months” — https://www.ghacks.net/2026/07/31/leaked-amazon-documents-detail-1-8-million-overrun-on-a-single-claude-ai-task-missed-for-five-months/
- IBM Think, “Tokenmaxxing is dead, long live valuemaxxing” — https://www.ibm.com/think/insights/tokenmaxxing-dead-long-live-valuemaxxing
- rizz.dev, “Perplexity Isn’t Shutting Down. Its Citation Corpus Might Shrink” — https://rizz.dev/feed/is-perplexity-down-lawsuits-not-outages
- AI Q&A Hub, “Perplexity Fake Citations: How to Fix Them in 2026” — https://www.aiqnahub.com/perplexity-fake-citations-how-to-fix-them-in-2026/
- arXiv 2604.03173, “Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents” — https://arxiv.org/html/2604.03173v1
- arXiv 2602.21456, “Revisiting Text Ranking in Deep Research” — https://arxiv.org/html/2602.21456.pdf
- tranx.io, “robots.txt Blocking AI Crawlers: Why GPTBot Gets Silently Blocked” — https://tranx.io/blog/robots-txt-blocked-ai-crawlers
- Weights & Biases, “Ensembling and ensemble learning methods” — https://wandb.ai/byyoung3/ML_NEWS3/reports/Ensembling-and-ensemble-learning-methods–Vmlldzo5OTMwMjYw; arXiv 2512.23213, “Scoring, Reasoning, and Selecting the Best!…” — https://arxiv.org/pdf/2512.23213
- GitHub, anthropics/claude-code Issue #40175, “[BUG] Cowork: Global instructions silently revert to older version after saving” — https://github.com/anthropics/claude-code/issues/40175
- GitHub, anthropics/claude-code Issue #40731, “[BUG] Opening a prior Cowork session silently overwrites global instructions with the version from that session” — https://github.com/anthropics/claude-code/issues/40731
All URLs above were live and retrievable at time of writing (August 2026).