Stanford’s CS329Z starts with a deceptively important question: who decides the next step? A language model can be part of a sophisticated system without making that system an agent. If code fixes the sequence of operations, it is a workflow. If the model chooses actions as it observes the environment, it is acting as an agent. Both are compound AI systems: models combined with retrieval, tools, code, and other components.
Key insights #
- An agent is defined by model-directed action and observation, while a workflow has developer-directed steps. Both can be powerful compound systems.
- Reasoning, memory, and tools solve different problems: deciding what to do, preserving useful experience, and reaching capabilities beyond the model.
- Task completion alone is an inadequate score. Reliability across runs, cost, safety, and recoverable failures belong in the evaluation.
- More autonomy is a design tradeoff. It is valuable when the task genuinely needs decisions that cannot be fixed ahead of time.
The agent is a loop, not a model #

Source: CS329Z Lecture 1, PDF page 32.
The lecture’s anatomy diagram puts an LLM at the core, surrounded by planning and reasoning, memory, tools, and an environment. The agent takes an action, receives an observation, and decides what to do next. This makes autonomy a property of control flow, not a marketing name for an API call.
The distinction matters in practice. Consider answering a question from a knowledge base. A fixed pipeline might retrieve passages, rerank them, and generate an answer. An agent might first decide whether retrieval is necessary, revise a failed query, inspect a source, or stop when the evidence is sufficient. The second design can handle open-ended tasks, but every extra decision creates another opportunity for error.
Diyi Yang’s historical tour makes this less novel than it sounds. The idea of a software entity that senses and acts predates LLMs. What language models changed is the range of natural-language instructions and environments a single controller can handle. The lecture then uses ReAct to illustrate the modern loop: reasoning can guide an action; the action yields an observation; the next decision uses that observation. Planning, reflection, and multi-agent orchestration are extensions of this core pattern.
The history in the slides runs from the actor model in the 1970s, through 1990s definitions of agents as autonomous, reactive, proactive, and social systems, to reinforcement learning’s agent-environment loop. The LLM era adds flexible language reasoning and tool selection. This history matters because an LLM is only one possible controller; the enduring engineering problem is how a controller senses, acts, and receives feedback.
Reasoning is an internal action #
The lecture treats intermediate reasoning as an update to an agent’s own state. In its cooking example, the agent sees that salt is unavailable and reasons that soy sauce might provide a savory substitute. The reasoning step bridges an observation and a grounded next action. ReAct makes this explicit by alternating a proposed action with a fresh observation. Compared with reasoning only in text, the agent can check facts in the environment instead of extending an unverified chain of guesses.
Other strategies address different failure modes. Self-consistency samples several reasoning paths and aggregates their answers. Reflexion takes feedback from a failed attempt and writes a short lesson into memory for the next attempt; the lecture notes that this is linguistic feedback, not weight updates through reinforcement learning. Debate or orchestration can provide alternative perspectives, though those approaches also introduce communication and coordination costs. The lesson is to select a reasoning mechanism for a measured failure, not add every mechanism by default.
The three things to engineer #

Source: CS329Z Lecture 1, PDF page 56.
One slide reduces the course to system, data, and evaluation. The system includes architecture and control flow. Data includes demonstrations, memories, retrieved context, and training examples. Evaluation asks whether the system completed the task, how much it cost, how reliably it did so, and whether it behaved safely. This is a useful antidote to treating the model name as the whole design.
Memory gets its own taxonomy: episodic memory records experiences, semantic memory stores distilled knowledge, and procedural memory stores reusable skills. The lecture notes that simply extending the context window is not a full memory design. An enormous event log may still make the relevant fact difficult to find or use.
The slides give concrete write and read patterns. An episodic store can append events and retrieve them by relevance. Semantic memory can use model reasoning to turn events into facts that can later be searched. Procedural memory can save code-based skills and retrieve an appropriate one for a future task. These are different kinds of information, so one generic “memory” field will rarely serve them equally well.
The tool section adds a second distinction. A model can select a tool and fill typed arguments, but the harness must validate and execute the call, then return the result as an observation. The lecture previews Toolformer, Gorilla, RAG, and MCP to show why tool use is both a learning problem and a systems interface problem. An agent that can call a calculator or search an API has expanded its reach, but it also has new ways to make consequential mistakes.
Why agent success is a harder claim #
The closing section names five engineering challenges: reliability, training, long-horizon context, safety, and evaluation. A chain of individually plausible steps can fail as errors compound. A task can be completed while violating a safety requirement. Longer histories can drift away from the original instruction. Sparse success signals make improvement expensive.
The reliability slides ask three separate questions: does the same task succeed consistently across runs; does it survive realistic changes to prompts and tools; and, when it fails, is that failure predictable and recoverable? The safety examples include distracting pop-ups, privacy-sensitive actions, reward tampering, and coordination failures between agents. They illustrate why “the final answer looked right” cannot establish that the process was acceptable.
The opening course discussion also raises a human consideration: AI assistance can accelerate a task while weakening the learner’s own understanding if it replaces the work of debugging and explanation. For a course about building agents, that is a useful reminder to inspect traces and learn from failures rather than hide the process behind a polished final response.
The assigned AI Agents That Matter sharpens the evaluation point: accuracy alone can reward an unnecessarily expensive system, and weak holdout practices can reward benchmark-specific tricks. For a real application, I would compare a fixed workflow with an agent on the same tasks, reporting success, cost, latency, and failure modes. Add autonomy only when it buys enough capability to justify the extra decisions.
Further reading #
- Lecture 1 slides and video with captions
- Zaharia et al., The Shift from Models to Compound AI Systems
- Lilian Weng, LLM Powered Autonomous Agents
- Kapoor et al., AI Agents That Matter