ReAct loop, the canonical formula, and why this is the foundational mental model every expert shares.
01 · Core Concept
What an Agent Actually Is — The Engineer's Definition
Forget "AI that can do things." That's marketing. Here's how the field actually defines it.
In computer science, an agent is any system that perceives its environment, makes decisions, and takes actions to achieve goals. The field has studied agents for 40 years (think robotics and game-playing AIs). What's new since 2022 is using a Large Language Model (LLM) as the reasoning engine inside that agent — replacing hand-coded logic with a model that can read instructions in plain English.
Each term has a precise meaning:
🧠
LLM — The Reasoning Engine
The LLM is not the agent itself — it's the brain. It reads the current situation (observation + history) and outputs the next reasoning step or action. It has no persistent state; every call is stateless. The agent scaffolding (the code around the LLM) is what gives it memory, tools, and a loop.
Think of the LLM like a very capable consultant you can call anytime — but they forget the last conversation the moment you hang up. The scaffold is your assistant who briefs them each time and executes what they advise.
💾
Memory — State Across Time
Memory lets the agent track what happened so far. There are four types (we go deep on this in Day 2): in-context (the conversation window), external (a database the agent can query), episodic (past interaction summaries), and semantic (factual knowledge, often in a vector store). Without memory, the agent is amnesiac — every step starts from scratch.
📋
Planning — Breaking Goals Into Steps
Planning is the agent's ability to decompose a high-level goal ("research competitors and write a report") into a sequence of executable sub-tasks. Naive agents just react to each step; sophisticated agents build a plan first, then execute it, and replan when they hit unexpected results. Day 3 covers planning algorithms in depth.
🛠️
Tool Use — Reaching Into the World
Tools are the hands of the agent. Without tools, the LLM can only talk — it can't read a URL, write a file, query a database, or send a message. Each tool is a function the LLM can call by name. The agent framework intercepts the LLM's output, detects a tool call, executes it, and feeds the result back into the next LLM call as an "observation."
This is exactly why OpenClaw lives in Signal or Telegram — those messaging platforms become the tool through which the agent perceives your world and delivers results.
01b · The Mechanism
The ReAct Loop — How Agents Actually Execute
The most important algorithm in the field isn't a neural network trick — it's a simple loop called ReAct (Reason + Act), introduced in a 2022 paper from Princeton & Google. Almost every production agent runs some version of this loop.
👁️
OBSERVE
Read current state
→
💭
THINK
Reason in natural language
→
⚡
ACT
Call a tool or respond
→
📥
OBSERVE
Read tool result
↺
The key insight of ReAct is the interleaving of reasoning traces and actions. Before the paper, researchers tried either pure reasoning (chain-of-thought) or pure action. ReAct showed that alternating between verbal reasoning ("I need to look up X because Y") and concrete actions ("SEARCH[X]") dramatically outperforms either approach alone.
What a single ReAct cycle looks like in text
Thought: The user wants the Q4 revenue for Company X. I should search their latest earnings report.
Action: search("Company X Q4 2025 earnings")
Observation: [Result: Company X reported $4.2B revenue in Q4 2025, up 18% YoY...]
Thought: I have the number. I can now answer.
Answer: Company X's Q4 2025 revenue was $4.2B, up 18% year-over-year.
Each "Thought" is the LLM reasoning out loud. Each "Action" calls a tool. Each "Observation" is the tool's return value. The loop continues until the agent decides it has enough to answer.
From a formal CS perspective, this is a POMDP (Partially Observable Markov Decision Process) — the agent can't see the full world, only what its tools return. This is why agents can be wrong or hallucinate: they're reasoning under uncertainty with incomplete information. When engineers say "grounding," they mean connecting the agent's beliefs to real-world facts via observations.
02 · Papers to Know
The Research Foundation
These three papers form the intellectual DNA of the field. Engineers will reference them in interviews, blog posts, and architecture discussions. Know them.
Seminal · ICLR 2023
ReAct: Synergizing Reasoning and Acting in Language Models
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, Yuan Cao — Princeton & Google · 2022
Before ReAct, AI researchers treated "reasoning" (thinking step by step) and "acting" (calling external tools) as separate capabilities. This paper proved they work dramatically better together — interleaved in a tight loop. The agent reasons in plain text to form a plan, takes an action, reads the result, reasons again, and repeats. Tested on question answering, fact-checking, and game navigation tasks, ReAct outperformed all baselines. Crucially, the reasoning traces are human-readable — you can see exactly why the agent made each decision.
Why it matters: ReAct is to agents what the Transformer paper is to LLMs. The entire generation of agents built in 2023–2026 — LangChain, AutoGen, Claude's tool use, OpenClaw — runs some variant of the ReAct loop. If you say "ReAct loop" in a conversation, every engineer in the room will know exactly what architecture you mean.
↗ arxiv.org/abs/2210.03629
Framework · 2023
Cognitive Architectures for Language Agents (CoALA)
Theodore Sumers, Shunyu Yao, Karthik Narasimhan, Thomas Griffiths — Princeton · 2023
This paper brings 60 years of cognitive science into AI agent design. It maps human cognitive architecture concepts (working memory, long-term memory, procedural knowledge) onto LLM agent components — and argues that most agent failures are memory failures. It proposes a systematic framework for thinking about how agents store and retrieve information across time. More useful as a thinking tool than as an engineering spec.
Why it matters: It gave the field a shared vocabulary for memory. When engineers debate "episodic vs semantic memory in agents" or "when to consolidate to long-term storage," they're using the concepts from this paper. You'll hear this terminology constantly.
↗ arxiv.org/abs/2309.02427
Survey · Jan 2026
Agentic AI: Architectures, Taxonomies, and Evaluation of LLM Agents
Multiple authors · arXiv · January 2026
The most comprehensive 2026 survey — 200+ pages covering how the entire field has structured itself. It proposes a six-dimensional decomposition of any agent: Core Components (perception, memory, action, profiling), Cognitive Architecture (planning, reflection), Learning mechanisms, Multi-Agent Systems, Environments, and Evaluation. Written from a formal POMDP framing, it's the reference document researchers use when designing new systems.
Why it matters: If you cite this paper in a conversation, you signal you're tracking the 2026 state-of-the-art, not 2023 blog posts. It also gives you a mental checklist — when evaluating any agent product, you can run through all six dimensions and identify where it's strong or weak.
↗ arxiv.org/html/2601.12560v1
04 · What the Community Is Saying
Voices That Shape the Field
These are the people engineers and researchers actually follow. Their posts move the field. Knowing their mental models helps you enter those conversations.
"All LLM frontier labs will do this. It's the final boss battle." — on autoresearch completing 700 experiments autonomously in 2 days.
Context: In March 2026, Karpathy released autoresearch — a 630-line Python script that implements a pure ReAct loop for ML experimentation. An agent reads training code, forms a hypothesis, modifies it, runs a 5-minute experiment, evaluates the metric, and commits or reverts via git. The agent ran for 2 days, conducted 700 experiments, and discovered 20 real optimizations resulting in an 11% speedup. Karpathy called this "the loopy era of AI" — the beginning of agents improving themselves. 21,000 GitHub stars in days.
"The concept of memory in agents is inspired by human cognition. Planning gives agents structure. Tool use gives them reach. Together they turn an LLM into an autonomous actor."
Context: Weng's 2023 blog post "LLM Powered Autonomous Agents" is the most-cited non-paper resource in the field. Her formula — Agent = LLM + Memory + Planning + Tool Use — is the mental model. Engineers still cite it in 2026. Her blog (lilianweng.github.io) is considered required reading; she explains complex concepts with rigor and clarity that neither dumbs down nor alienates.
"The debate has shifted from 'should agents be autonomous?' to 'what's the right level of human-in-the-loop for this task?' Every use case has a different answer."
Context: Chase has been vocal in 2026 that the "fully autonomous agent" hype is settling into nuanced practice — some tasks warrant full autonomy, others need human checkpoints. LangGraph was built to make these checkpoints explicit and controllable. His posts are excellent signals for where production engineering is heading vs. research.
06 · Vocabulary Master List
12 Terms You Must Own
These are the exact terms engineers use. For each: the real definition, what people get wrong, and how to use it correctly.
ReAct Loop
The agent execution pattern: Observe → Reason → Act → Observe, repeated until task completion. Named after the 2022 paper. The reasoning step is explicit natural language written by the LLM.
⚠ Don't say: "ReAct is a type of model." It's an execution pattern, not a model.
✓ Use: "Our agent runs a ReAct loop with up to 10 tool calls before forcing a final answer."
Scaffold / Scaffolding
The code surrounding the LLM that implements the agent loop — managing state, routing tool calls, injecting context, parsing outputs. The LLM is the brain; the scaffold is the nervous system and body.
⚠ Don't confuse with the model itself — they are separate.
✓ Use: "OpenClaw's scaffolding is open-source, but it can run any LLM as its brain."
Grounding
Connecting the agent's beliefs to real-world, verifiable facts via observations from tools (web search, database queries, file reads). An "ungrounded" agent is reasoning from its training data alone — prone to hallucination on factual questions.
⚠ Don't conflate with "instruction following" — grounding is specifically about factual accuracy via external verification.
✓ Use: "We ground the agent's financial claims with real-time database queries before it drafts the report."
Observation
The return value of a tool call — what the agent "sees" after taking an action. In the ReAct loop: Action → Observation is the tight coupling that grounds the agent's next reasoning step in reality.
⚠ Don't use "result" interchangeably — "observation" has the specific meaning of "what the environment returned."
✓ Use: "The observation from the search tool changed the agent's plan entirely."
Action Space
The complete set of tools and actions an agent can take. Larger action spaces = more capable agents, but also harder to manage (more decisions, more failure modes). Designing the right action space for a use case is a core engineering challenge.
⚠ Don't assume bigger is always better — a focused action space often beats a broad one for specific tasks.
✓ Use: "For a customer support agent, we restricted the action space to 8 tools to limit error surface."
Tool Registry
The catalog of available tools the agent can call — each defined by a name, description, and parameter schema. The LLM reads the registry to decide which tool to call and with what arguments. MCP standardizes how this registry is structured and shared.
⚠ Don't call it an "API list" — a registry includes semantic descriptions the LLM uses for tool selection, not just endpoint specs.
✓ Use: "We added a CRM lookup to the tool registry so the agent can pull customer history without asking the user."
Orchestrator
In multi-agent systems, the orchestrator is the agent that manages other agents — assigning tasks, tracking progress, and synthesizing results. It does not execute tasks itself; it delegates.
⚠ Don't confuse with "controller" or "manager" — orchestrator has a specific meaning in the multi-agent architecture pattern.
✓ Use: "The orchestrator agent broke the research request into 4 sub-tasks and dispatched them to specialized worker agents."
POMDP
Partially Observable Markov Decision Process — the formal mathematical framework agents operate in. "Partially observable" means the agent can't see the full world state, only what its tools return. This formalism explains why agents can be wrong: they're always reasoning under uncertainty.
⚠ You don't need to derive the math, but knowing the term signals you understand agents have epistemic limits by design.
✓ Use: "The POMDP framing makes it clear why adding more tool calls doesn't always improve accuracy — the agent's beliefs can still diverge from ground truth."
Context Window
The maximum amount of text (measured in tokens ≈ 0.75 words each) an LLM can process in a single call. This is the agent's short-term memory. Claude Sonnet 4.6 has a 1M-token context window in beta — roughly 750,000 words or ~10 full novels.
⚠ Don't say "memory limit" — that implies persistence. Context window is session-scoped, not long-term storage.
✓ Use: "With a 1M context window, the agent can hold the entire codebase in working memory during a debugging session."
MCP
Model Context Protocol — Anthropic's open standard for connecting LLMs to tools and data sources. Like USB for AI: instead of each agent building custom integrations, MCP provides a universal plug. Now supported by OpenAI and Google.
⚠ Don't confuse with "API" — MCP is a protocol layer on top of APIs that adds agent-specific semantics (tool discovery, schema negotiation, streaming).
✓ Use: "We built our CRM integration as an MCP server so any MCP-compatible agent can use it without custom code."
Agentic Loop
Informal synonym for the ReAct loop — the iterative cycle by which an agent makes progress toward a goal through alternating reasoning and action steps. Often used to describe any autonomous execution pattern, not just ReAct specifically.
⚠ Use "ReAct loop" when you mean the specific 2022 paper's pattern; use "agentic loop" when speaking more generally.
✓ Use: "The agent ran 12 iterations of the agentic loop before producing a final answer."
Reasoning Trace
The chain of visible "thought" steps the agent writes before taking an action — the "Thought:" lines in a ReAct loop. Making traces visible enables debugging, auditing, and trust. Hiding traces speeds up the agent but makes behavior opaque.
⚠ Don't call it "internal monologue" in technical settings — "reasoning trace" is the precise term.
✓ Use: "We log the full reasoning trace for compliance — auditors can see exactly why the agent took each action."
07 · Questions You Can Now Ask
5 Questions That Signal Deep Understanding
These are not beginner questions. Ask any of these and a senior engineer will lean forward.
Q1
"When you design your agent's action space, how do you handle tool selection ambiguity — do you expose the full tool registry to the LLM on every step, or do you use retrieval to surface only relevant tools dynamically?"
Q2
"How do you handle mid-task replanning when an observation contradicts the agent's prior assumptions? Does your loop support backtracking or is it strictly forward-only?"
Q3
"Is your agentic loop synchronous or async? For long-running tool calls — like spawning a browser or running code — how do you avoid blocking the reasoning trace?"
Q4
"In your architecture, where does the ReAct trace live? Is the reasoning visible to the orchestrator and logged for auditability, or does it only live inside the LLM's context window?"
Q5
"Karpathy's autoresearch uses git as the agent's memory and commit/revert as its error recovery mechanism — do you use any similar environment-level checkpointing, or does your agent handle failure recovery purely in the LLM reasoning trace?"