Designing an Agent Runtime From Scratch
What Goes in the Loop
Most agent runtimes fail at the same place: the loop.
The loop is the thing that wakes up, checks for new input, assembles the context, calls the model, dispatches the action, and checkpoints state so the whole thing can survive a crash. It runs thousands of times a day. Every architectural decision you make about the loop compounds across every turn. A bad context assembly strategy doesn’t fail once. It fails on every single turn after the context window fills up. A missing checkpoint doesn’t lose a single action. It loses every action since the last save, and if you never checkpointed at all, it loses everything.
The loop is where the runtime earns its name. Everything else, the skill system, the memory tier, the tool integrations, the chat platform adapters, hangs off the loop. Get the loop wrong and nothing else matters. Get it right and the rest is composition.
Here is the loop that every agent runtime needs, walked through layer by layer: event intake, context assembly, tool discovery, model invocation, action dispatch, state checkpointing, and recovery. This is not a tutorial. It is a decision framework for the architectural choices every runtime designer faces, grounded in how Hermes Agent, OpenClaw, and Restate each solve different pieces of the problem.
The first decision is how events enter the loop. OpenClaw treats everything as an event with a unified input queue: chat messages, cron ticks, webhooks, heartbeat signals all flow through the same pipe. The runtime wakes up, dequeues the next event, and processes it. This has an important property: every event gets handled exactly once, in order, with a clear boundary between “what triggered this turn” and “what happened during it.” The alternative is polling: the runtime checks N sources on every tick and processes whatever it finds. Polling is simpler to implement but harder to reason about. If two sources produce events simultaneously, which one gets processed first? If one source is slow to respond, does it block the entire tick? The unified queue pattern costs more upfront but pays off the first time you need to debug why an agent took an action that made no sense. With a unified queue, you can trace the exact event that triggered the turn. With polling, you are guessing.
Hermes Agent takes a hybrid approach. Cron events and scheduled triggers enter through a structured queue, but direct chat interactions are handled inline. The tradeoff is deliberate: scheduled work needs guaranteed processing order. Interactive work needs low latency. Treating them the same way would force interactive sessions to wait behind batch jobs or force batch jobs to be deprioritized below chat. The hybrid model acknowledges that not all events are equal, and the runtime should not pretend they are.
The second decision is context assembly. This is where most runtimes fail silently. The context window has a hard limit: 200k tokens for Claude Sonnet 4, 128k for GPT-4o, 8192 for Llama 3.1 8B. When the conversation history, tool outputs, and system instructions exceed that limit, something must be truncated. The default strategy in most frameworks is “keep the most recent N tokens and drop the rest.” This is catastrophic for long-running agents because the dropped tokens often contain the instructions the agent needs most: the goal, the constraints, the output format. An agent that forgets its system prompt on turn 40 is worse than useless. It is actively dangerous.
The correct strategy is relevance-based pruning with instruction anchoring. The system prompt, safety policies, and output format constraints are pinned to the top of the context window and never truncated. Tool outputs are summarized rather than dropped, using a cheaper model if necessary. Conversation history is pruned by relevance to the current turn, not by recency: a critical instruction from turn 3 matters more than a status update from turn 39. Hermes Agent implements this through its memory scoring system. OpenClaw handles it through configurable prompt templates with token budgeting. The implementation differs; the principle is the same. Context assembly is not a trimming problem. It is a relevance problem.
The third decision is tool discovery. How does the runtime know what tools are available, and how does it present them to the model? The simplest approach is hardcoding: the developer registers tools at startup and the runtime passes the full list on every turn. This works for five tools. It fails for fifty because the tool descriptions consume context window tokens and the model gets worse at selecting the right tool as the list grows. The MCP pattern solves this differently: tools are discovered dynamically from MCP servers, and the runtime can filter the available tool set based on the current context. An agent processing a database query does not need to know about the image generation tool. Presenting only relevant tools reduces token consumption and improves tool selection accuracy.
The fourth decision is where the model call happens and what happens when it fails. The model call is the simplest part of the loop conceptually: send the assembled context to the model, get back a response, parse the response. In practice, it is the highest-failure-rate operation in the entire system. Models return malformed JSON. Models hallucinate tool names. Models exceed token limits. Models time out. The runtime must handle every failure mode and retry with backoff before the agent even knows something went wrong. The retry strategy matters more than the model selection. A naive retry with the same malformed context will produce the same malformed output. A smart retry trims the context, adds explicit formatting instructions, or falls back to a more reliable model for the retry attempt. Hermes Agent does this through its provider-agnostic retry layer: if the primary model fails three times with the same error class, it falls back to a simpler model for the reformatting pass, then returns control to the primary model.
The fifth decision is action dispatch. The model has produced a tool call. The runtime must execute it. This is where security, sandboxing, and credential management converge. The runtime should never pass raw credentials to the model. It should never execute a tool call without validating that the call is permitted under the current security policy. It should sandbox execution so that a hallucinated destructive command does not reach the filesystem. Claude Code’s permission pipeline is the reference implementation here: every tool call passes through a policy engine that checks whether the agent is allowed to make this call, with these parameters, in this context. If the call is denied, the agent receives a structured denial rather than a raw error, which lets it recover gracefully rather than retrying blindly.
The sixth decision is state checkpointing. This is the one integration most runtimes skip, and it is the one that determines whether your agent survives a crash. After every action dispatch, the runtime must save enough state to resume from exactly this point: the full conversation history, the current tool outputs, the agent’s internal state, the skill memory, the context window composition. If the process dies on turn 37, the runtime resumes at the start of turn 38, not turn 1. Restate’s entire product is built around durable execution: treat every workflow as a state machine, persist the state after every transition, guarantee exactly-once execution semantics. It is the most rigorous implementation of this pattern, and it is worth understanding even if you never run Restate yourself, because the pattern is what every runtime eventually needs.
Hermes Agent implements checkpointing through its transaction log: every state mutation is written to a durable log before the action is dispatched. If the process crashes, the runtime replays the log from the last checkpoint and resumes. OpenClaw’s event-driven architecture makes checkpointing simpler in one respect: because every turn is triggered by an event, the runtime can checkpoint after every event is fully processed. The tradeoff is that checkpointing an event-driven system requires serializing the entire event history, not just the agent state. Neither approach is wrong; they optimize for different failure modes.
The loop, end to end: an event arrives through the queue or the cron scheduler. The runtime assembles the context window, pinning critical instructions and pruning by relevance. It discovers available tools through MCP servers or a local registry, filtering to only those relevant to the current context. It calls the model with retry logic that handles malformed responses and falls back to simpler models when necessary. It dispatches the resulting action through a permission pipeline that validates the call, injects credentials without exposing them to the model, and sandboxes execution. It checkpoints the entire agent state so the next turn starts from known ground. Then it waits for the next event.
If you are building a runtime from scratch, start with the checkpoint. You can bolt on tool discovery later. You can improve the context assembly strategy over time. You can swap models without changing the loop. But if you do not have state checkpointing on day one, you are building something that will lose everything it learns the first time it crashes. And it will crash. Every runtime crashes. The question is whether it wakes up where it left off or wakes up at zero.
The second thing to build is the retry layer around the model call. Not because the model is unreliable. Because the model is the only component in the loop whose failures are visible to the user. When the model returns garbage, the user sees garbage. When the retry layer catches the garbage before the user sees it, reformats, and retries, the agent looks reliable even when the model is not. Most “agent reliability” problems I have debugged were not model problems. They were retry-layer problems that nobody had built yet.
The third thing is the permission pipeline. Build it before you connect the agent to anything that can do damage. Filesystems, databases, payment APIs, production infrastructure. The model will eventually produce a destructive tool call. It is not a question of if. The only question is whether your runtime catches it before it executes.
The loop is not the exciting part of building an agent. The model is exciting. The skills are exciting. The autonomous behavior is exciting. The loop is the plumbing. The loop is also the part that determines whether your agent is a demo or a product. Most agent projects I have reviewed have a model and a set of tools and something that looks like a loop if you squint. What they are missing is the recovery infrastructure, the checkpoint discipline, and the permission pipeline, the three things that turn a script into a runtime. That gap is not a missing feature. It is the feature.
If this was useful, forward it to one engineer who needs less noise in their feed.