Agentic AI

Silent Failure: What Running an Agent Fleet on OpenClaw Taught Me

A poised figure reading confidently from a blank sheet of paper, evoking fluent but hollow output from a silent failure.

One of my OpenClaw agents spent about ten weeks writing thoughtful weekly summaries of a feed that had gone completely empty.

Every week it did exactly what it was configured to do. It opened the folder it was supposed to review, found nothing new, and instead of stopping, it wrote around the absence. The summaries grew more articulate over time. One week's was about how the signal was narrowing. The next, about how the silence itself was hardening into a kind of filter. The agent had started finding meaning in the quiet. There was no quiet. A route upstream had broken two months earlier, the feed had stopped arriving, and the agent had gone on summarizing an empty folder week after week without once noticing it had nothing to summarize.

Nothing alerted. It ran on schedule, produced its required output, and the output read as insight. I caught it only because I glanced at the summaries and realized they had all started to sound the same.

That is the failure mode that actually matters when you run agents in production, and it is the one default liveness monitoring rarely catches. Let me show you the system it happened in, because the shape of the system is the whole point.

The system

I run about fifteen agents on OpenClaw, an open-source harness for long-running autonomous agents. They are not a demo. They do real, scheduled work, and they have been doing it for months. The shape looks like this.

The fleet: about fifteen agents on OpenClaw

Gatherer agents feed a synthesis agent, briefing agents run on their own schedules, and a watchdog is supposed to catch backup failures. A cron scheduler drives the whole thing. Every agent boots with an injected identity and memory file.

A set of gatherer agents pull from external sources and route the keep-worthy material into a shared feed. A synthesis agent reads that feed and produces summaries. A separate set of briefing agents run on their own cron schedules and produce daily and weekly digests. A watchdog watches the offsite backup and is supposed to alert me if it fails. And every one of these agents boots with two files injected into its context: a SOUL file that defines who it is and what it does, and a memory file of accumulated context. OpenClaw 2026.5.22, the release my fleet runs, defaults each SOUL and memory file to a 12,000-character-per-file injection cap. When a file is longer, it keeps the beginning and the end and drops the middle.

Hold onto that last detail. It comes back.

None of this is exotic. If you have built any distributed system, you already recognize it: scheduled workers, a shared store, a fan-out of consumers, a health check. What is different is that every node here is an LLM agent, and an LLM agent has a property no ordinary worker process has. When it is starved, misconfigured, or fed garbage, it does not throw. It writes. Fluently. That single property is what turns routine operational failures into invisible ones.

Everything was running. That was the problem.

Autonomous agents rarely fail by crashing. A crash is loud, and loud failures are the easy ones: an error, a page, a stack trace, a thing to fix. Agents fail the other way. They keep running while something upstream has quietly died, and they produce plausible, confident, hollow output that announces nothing wrong. You check whether the system is up, and it is. Everything is up. That is precisely why you miss it.

An agent that crashes tells you something is wrong. An agent that keeps running tells you nothing, and when its inputs have rotted, nothing is exactly what you should not trust.

If this shape feels familiar, it should. It is the agentic re-run of lessons distributed systems learned the hard way. Stale caches served as if fresh. Data pipelines that rot silently while the dashboard stays green. Systems with no observability into whether their own inputs are current. What is new is only the veneer. A broken pipeline used to emit a null or a stack trace. An agent emits a paragraph, and hollow output is written in the same confident prose as real output, so you cannot tell them apart by reading.

Here is where each of those breaks actually happened in my fleet.

Liveness monitoring reported all of it healthy, because all of it was running. The five numbered breaks are the ones that raised nothing.

The empty feed

Start with the one I opened on, because the diagram makes its cause obvious. The gatherer agents were supposed to route keep-worthy references into the synthesis agent's feed. A scheduler re-wire, months in, quietly dropped that one route while leaving the others intact. The gatherers kept running. The synthesis agent kept running. But nothing connected them anymore, so the feed went empty and stayed empty.

The synthesis agent's instructions said, in effect, review this week's entries and produce a five-point synthesis. With zero entries, it produced a five-point synthesis anyway, treating the absence as the finding. Because the output looked exactly like every previous week's, nothing flagged it. It ran that way for about ten weeks, roughly ten weekly summaries, every one of an empty feed.

The blast radius is the part that should worry you. Everything downstream of that agent, everything that read its summaries, was consuming confident analysis of nothing for two months. No alert fired because, by every liveness measure, the agent was perfectly healthy. It was running on schedule and producing well-formed output. The only thing wrong was that the output was hollow, and hollowness is not a signal any uptime check knows how to see.

The smoke detector with a dead battery

The watchdog was supposed to be my safety net for exactly this class of problem, at least for the backups. It was broken two ways at once, and I did not know, because a broken watchdog is silent by definition.

First, it invoked a command-line flag that had been removed in an OpenClaw update, so every time it ran it errored out before it could check anything. Second, even when it did run, it worked by asking an agent to relay a fixed alarm message to me. An agent handed a verbatim string to pass along will often reason about it, soften it, or reword it, so even a firing alarm would have arrived as something that no longer read as an alarm.

Here is the honest part, and the reason I cannot give you a clean number. I do not know how long the backups were actually at risk, because the thing that was supposed to know was dark the entire time. The detector was the incident. I rewrote it to send directly, with no agent in the loop and no removed flags.

The distributed-systems version of this is old and unforgiving. A health check that reports healthy while the service is down is worse than no health check, because it manufactures confidence. It is the same thing I have written about as the gap between a system's declared state and its observed state: a watchdog is a declared-state machine you have chosen to trust completely, which makes it the last place you can afford one to lie.

The jobs that pointed at something no longer there

When the default model on my stack moved up a version, most things followed automatically. But about twenty scheduled jobs each carried their own pinned model, and the pins still pointed at the retired one. Every one of them failed the moment it ran.

No job succeeded, and no job complained loudly enough to notice, because the way they failed was by not producing their briefing. A briefing that stops arriving does not page you. Absence is quiet. It took about twelve days and a vague "wait, when did I last get that?" to catch it.

This is silent dependency drift, and the reason it stays silent is structural: the failure is defined by something not happening. Monitoring is good at things that happen. It is bad, by default, at things that stop. If you only alert on events, a system that goes dark produces no events to alert on, and you have built a blind spot exactly where you most need to see.

When the description is fiction

Several of my agents carried SOUL files, the injected identity I described earlier, that flatly did not match their wiring. One said it monitored continuously, around the clock. Its actual schedule ran it once a week. Another said it fed the synthesis agent its inputs. The plumbing had never routed anything there at all.

That second one was not cosmetic. It was the root cause of the empty-feed story I opened with. The synthesis agent believed, in its own injected identity, that it was being fed. So did I. Neither of us was, and nothing in the system reconciled the claim against the wiring until I sat and read one against the other and saw they described two different machines.

Nothing checks what an agent says it does against what its plumbing actually does. The prompt is aspiration. The schedule is reality. When they disagree, the agent narrates the aspiration, fluently, because narrating what it was told it is is exactly the thing it is good at.

And sometimes the gap sits inside the agent's own memory, which is where that twelve-thousand-character cap comes back. One agent's memory file had grown to about thirty-seven thousand characters of deliberately curated context. The injector kept the first three-quarters and the last quarter and silently dropped the middle, so the agent was reasoning on roughly a third of the memory I believed I had given it, with no error, because from the outside it still had memory. That one has enough happening to deserve its own post. The pattern is identical to all the others: the system reported a capability it was not actually delivering.

Why this is a class, not a run of bad luck

The clearest tell is what happened while I was fixing all this. One of my own fixes re-sprang the exact same trap. I inlined some content into a few SOUL files, which pushed them back over the twelve-thousand-character cap, and they quietly started truncating again. I caught it in minutes, but only because I now knew to look. The trap is trivially easy to re-enter, which is the point. This is not a bug you fix once and retire. It is a category you instrument against permanently, the way you never stop watching a cache for staleness.

What actually protects you

Not liveness checks. Everything in every one of these stories was alive and running. The discipline is to monitor the things that go quiet, not the things that crash.

  • Watch inputs, not just uptime. Alert on an input count of zero, on a feed that has not arrived, on the age of the freshest thing an agent consumed. The synthesis agent should have been unable to summarize nothing; the empty feed should have been the alarm, not the routine.
  • Put freshness and provenance on what agents consume. An agent acting on stale or missing data should know its data is stale or missing, and say so loudly, rather than composing around the hole.
  • Treat confident output as suspect precisely when inputs go quiet. Fluency is not evidence. The most eloquent summaries I got were the ones written about nothing at all.
  • Audit prompt against plumbing. For each agent, read what its SOUL claims it does next to what it is actually wired to do. Every place they disagree is a silent failure already waiting.
  • Monitor the monitors. A watchdog is part of the system and fails like any other part. If it is the only thing between you and a silent outage, it needs its own check, and it must never route an alarm through an agent that can reword it.

None of this is new. It is the operational hygiene distributed systems has demanded for decades: observability on inputs, freshness, and provenance, plus the humility to assume your health checks can lie. What is new is that the failures now arrive dressed in thoughtful prose, which makes them easier to trust and harder to catch. Every agent you add to a system is not only a new capability. It is a new surface that can fail silently while sounding certain.

The question to keep asking is not whether your agents are running. They are almost always running. The question is whether anything is still feeding them the truth.

Sources

Based on my own OpenClaw fleet logs and configuration review.