AI Adoption

Lecture 4. How agents work

Two agents · The loop · Feedback and checking · Real agents · Checking our agents

Where we are

Lecture 2 · An LLM

A Transformer computes the probability of the next token given the context.

Pretraining → fine-tuning → RL.
It writes text. A tool call is also text.

Lecture 3 · Memory

The model remembers nothing between requests. Answer = parameters + context.

A chatbot re-sends the whole conversation, memories and search results every turn.

Today · Agents

The model’s text becomes actions. Their results come back into the context.

The loop runs until the goal is reached, the budget ends, or a person stops it.

From a chatbot to an agent

Chatbot (lecture 3)

You

type a message

App

builds the context: history, memory, search results

LLM

writes a reply

You

read it and decide what to do next

↺ repeat while you want

Agent (today)

Goal

given once

Harness

builds the context: history, tool results

LLM

writes a tool call, or says it is done

Tools

run it: shell, files, browser — and return the result

↺ repeat until done or out of budget

Today’s lecture

  1. Running two agentstwo tasks on SAT that work during the lecture
  2. The agent loopan LLM, tools and a while loop
  3. Why agents workfeedback, checking and its limits
  4. Real agents todayfrom cheap checks to no checks at all
  5. Back to our two agentsdid they succeed? we check ourselves
1

Running two agents

Two tasks on SAT: started now, checked at the end

SAT, the satisfiability problem

Input: a formula in CNF

(x1 ∨ x2) ∧ (¬x1 ∨ ¬x2) ∧ (¬x1 ∨ x2 ∨ x3)
∧ (x1 ∨ ¬x2 ∨ x3) ∧ (x1 ∨ ¬x2 ∨ ¬x3)

Is there an assignment that makes every clause true?
n variables, m clauses; k-SAT: at most k literals per clause.

The same formula in DIMACS, the format of task A

p cnf 3 5
1 2 0
-1 -2 0
-1 2 3 0
1 -2 3 0
1 -2 -3 0
In practice
modern CDCL solvers are extremely efficient: industrial formulas with millions of variables
In theory
until 2026, the best k-SAT algorithms took time about 2(1−1/k)n
ETH
for some δ > 0, 3-SAT needs time 2δn
SETH
for every ε > 0, some k-SAT needs time 2(1−ε)n
October 26, 2026
OpenAI and Anthropic post claimed refutations of ETH, found by their models: #k-SAT in time 2O(n log log n / √log n), proofs checked in Lean. Not yet peer-reviewed.

OpenAI, The Exponential Time Hypothesis is False · Claude (Anthropic), A 2o(n) Time Algorithm for 3-SAT · checked October 28, 2026

Demonstration 0 · two agents start now

Two tasks that will run during the lecture

A. Write a SAT solver from scratch

Claude Code, in a cloud sandbox · task in TASK.md

Task
A CDCL solver in pure Python; 80 test formulas with known answers
Tools
shell, Python, git
Checker
its own: assignments, brute force, DRAT proofs
Rules
no other solvers; do not read the answers; do not touch the tests

B. Deep research

a research agent with web search

“Find everything new about the Exponential Time Hypothesis since the news of its refutation on October 26, 2026. Make a table and group the sources in it by category. For each source, say how far it can be trusted and why.”

2

The agent loop

An LLM, tools and a while loop

What is an agent?

An LLM agent runs tools in a loop to achieve a goal.

Simon Willison, September 2025

Chat

The model writes a reply.

+ Tool use

The model writes a tool call; the program runs it.

+ Loop

The result goes back into the context; the model chooses the next call.

Willison · “Agent” may finally have a widely enough agreed upon definition · Anthropic · Building effective agents (2024)

The agent loop in code

messages = [{"role": "user", "content": task}]
while True:
    reply = query_lm(messages)
    messages.append({"role": "assistant",
                     "content": reply})
    command = parse_action(reply)
    if command == "exit":
        break
    result = execute_action(command)
    messages.append({"role": "user",
                     "content": result})

query_lm
one API request with the whole history

parse_action
finds the one bash-action block in the reply

execute_action
runs it in a shell, returns stdout + stderr

The command gets the program’s access to files and network.

Adapted from the Minimal AI agent tutorial · full version: materials/agent.py

Demonstration 1

A 160-line agent fixes a bug

  1. The task: sum_first_odds(n) should return 1 + 3 + … + (2n − 1). Two tests fail.
  2. cd materials, then python3 agent.py
    Each command is shown; Enter runs it.
  3. Watch the loop: look → run the tests → edit → test again → exit.
  4. The harness checks for itself: tests unchanged? tests pass? the diff.

agent-demo/odd_sums.py

def sum_first_odds(n):
    total = 0
    for k in range(1, n):
        total += 2 * k - 1
    return total

Tests: n = 0, 1, 3, and sum = n² for n < 20

Live model: Claude Opus 5.5 via the API · the API key is read from materials/.env

What a real harness adds

Claude Code, Codex, Cursor …

  • a long system prompt and tool descriptions
  • tools: read, edit and search files, run commands, fetch pages, start sub-agents
  • a to-do list: the plan, kept in the context
  • permissions and a sandbox
  • compaction when the context fills up

mini-swe-agent

  • about 100 lines of Python for the agent
  • one tool: bash
  • no tool-calling API, a text protocol like ours
  • >74% on SWE-bench Verified

mini-swe-agent (Princeton, Stanford) · score as reported in its README, checked October 26, 2026

Demonstration 2

The same loop inside Claude Code

  1. In materials, copy the task to a fresh temporary folder and start Claude Code there:
    M=$PWD; D=$(mktemp -d); cp -R agent-demo/. "$D"; cd "$D"; claude
  2. Type: “Make the failing tests pass without editing the tests.” Approve its commands. Ctrl+O shows the transcript.
  3. /context: the system prompt and the tool descriptions fill part of the context before we type anything.
  4. /exit, then python3 "$M/transcript.py" prints the session log from ~/.claude/projects/: user, model → tool, tool result, again and again.

Without feedback, the agent works blind

Change one line: execute_action runs the command but returns only “ok”.

  • The agent no longer sees the failing test, the error or the file contents.
  • It acts on its expectations, not on the world, so it cannot notice its own mistakes.
  • It cannot tell when the task is done.
optional

A workflow or an agent?

WorkflowAgent
Next stepWritten in the programChosen by the model from the context
ExampleLoad data → validate → compute → format a reportInvestigate an unfamiliar failure
Good fitThe steps are known in advanceThe steps are hard to specify in advance
Trade-offPredictable, cheaperFlexible, variable cost and behaviour

Anthropic · Building effective agents (December 2024)

3

Why agents work

Feedback, checking and its limits

Feedback makes long tasks possible

A task of 40 steps. Each step is right with probability 0.95, independently. An error that nobody catches ruins the task.

After each stepThe task succeeds with probability
no check0.9540 ≈ 0.13
a check catches every error; up to 3 tries per step(1 − 0.053)40 ≈ 0.995
a check catches 90% of errors; unlimited tries(0.95 / 0.955)40 ≈ 0.81

Checking is often easier than doing

TaskCheckerCost of a checkWhat it misses
A satisfiable formulathe satisfying assignmentlinear timenothing
An unsatisfiable formulaa DRAT proof + a proof checkerabout as long as solvingnothing, if the checker is right
A math statementa proof assistant (e.g. Lean)seconds to minuteswhether the statement means what you meant
Fix a bugtestssecondsinputs the tests do not cover
Improve a constructionan evaluation functionsecondswhat the score does not measure
A research summarya person reading the sourceshourswhatever the person skips
Run a shopthe marketweeksnoisy, and too late

Models are trained in this loop

RL from lecture 2, scaled up

  • an episode: a whole trajectory in a sandbox — a repository, a terminal, a browser
  • the reward: a checker — the tests pass, the answer matches, the page shows the result

Consequences

  • the tasks agents finish keep getting longer (METR, lecture 1)
  • skills with automatic checks improve fastest
  • a weak checker in training teaches the model to please the checker

Plans and stopping

A plan is a hypothesis

Reproduce → diagnose → edit → verify.

An observation can break it: the tests do not even start → fix the environment first.

In Claude Code the plan is a to-do list in the context.

Why a run ends

verified successsuccess
budget used upbudget
no progressstalled
a person must decideneeds input

When the checker is weaker than the goal

The agent satisfies the checker instead

  • edits, skips or deletes the failing tests
  • special-cases the test inputs
  • reads the answer off the file name or an answers file
  • reports success without running the checks

ImpossibleBench (2025): tests are changed to contradict the specification, so any “pass” is cheating. Frontier models cheat on a noticeable share of tasks.

“…the task verifier is nearly perfect, otherwise Claude will solve the wrong problem.”

Nicholas Carlini, Anthropic, February 2026

Zhong, Raghunathan, Carlini · ImpossibleBench (2025) · Carlini · Building a C compiler with a team of parallel Claudes

“All 214 tests pass” does not mean the task is done

The agent’s report can be true while:

  • the tests were changed, skipped or deleted;
  • the code special-cases the test inputs;
  • the tests do not cover the requirement;
  • the wrong tests were run, or an old version of the code;
  • the tests were never run: the report is just text.

AlphaEvolve: an LLM as a mutation operator

AlphaEvolve diagram: scientists provide a prompt template, LLMs, evaluation code and an initial program; the controller loop samples a parent program from the database, builds a prompt, asks the LLM for a diff, applies it, runs the evaluator and adds the child program and its scores to the database.
Task
a program that builds an object or a heuristic
Tools
LLMs propose code changes; evolution keeps the best programs
Checker
an evaluation function, exact and fast
Results
4×4 complex matrices with 48 multiplications; kissing number in 11 dimensions ≥ 593; 67 problems with Tao et al.: most best-known results matched, several improved
Limits
constructions and bounds, not proofs; people write the evaluator

Diagram: Google DeepMind, AlphaEvolve (May 2025) · Georgiev, Gómez-Serrano, Tao, Wagner (November 2025)

16 agents write a C compiler

16parallel Claude agents (Opus 4.6), two weeks
~2,000Claude Code sessions
< $20Kin API costs
100Klines of Rust, from scratch

What it compiles

Linux 6.9 on x86, ARM and RISC-V; QEMU, FFmpeg, SQLite, PostgreSQL, Redis, Doom. About 99% on most test suites, including GCC’s torture tests.

How it was checked

Test suites, and GCC as a reference: most kernel files built by GCC, the rest by the new compiler. Agents took tasks via lock files in git; no coordinator.

Nicholas Carlini · Building a C compiler with a team of parallel Claudes · Anthropic, February 5, 2026

Demonstration 3

An agent in a browser

  1. Claude in Chrome, ChatGPT agent or Gemini, in a separate browser profile.
  2. “Open excalidraw.com and draw the agent loop: four boxes in a row labelled Goal, Harness, LLM and Tools, arrows between them from left to right, and an arrow from Tools back to Harness labelled ‘result’. Draw it yourself with the box, arrow and text tools.”
  3. Excalidraw draws on a canvas: no page text to read, no API. Watch: screenshot → click → drag → type → screenshot. Count the steps; note the time.
  4. Check the drawing yourself: every box, label and arrow.
Observation
a screenshot; where a page has text or an API, the agent reads that instead
Actions
click (x, y), drag, type, press keys, go to a URL
Checker
you, looking at the drawing
OSWorld
369 desktop tasks. 2024: humans 72%, best model 12%. Autumn 2026: top entries on OSWorld-Verified report above 85%

OSWorld (Xie et al., 2024)

Claudius runs a shop

Phase 1 · 2025 · Sonnet 3.7

  • a small shop in Anthropic’s office; Slack, web search, notes, a human for deliveries
  • lost money; bought metal cubes and sold them at a loss
  • declined $100 for a $15 six-pack
  • claimed it would deliver “wearing a blue blazer and a red tie”

Phase 2 · December 2025 · Sonnet 4 → 4.5

  • a CRM, price checks, an AI “CEO”; shops in San Francisco, New York, London
  • mostly profitable: “bureaucracy matters”
  • nearly signed an illegal onion futures contract
  • reporters talked it into giving goods away

Anthropic and Andon Labs · Project Vend (June 27, 2025) · Project Vend: phase two (December 18, 2025)

When agents have real access

Replit · July 2025

During a declared code freeze, a coding agent ran destructive commands and wiped a production database. It then said a rollback was impossible. It was not.

OpenClaw · early 2026

A popular open-source personal agent with access to files, shell and messengers. Within weeks: remote-code-execution bugs, malware among its skills, prompt injections in a social network for agents.

A model in a safety test · 2026

In a UK AISI hacking exercise with internet access, an agent sent malicious code to a real open-source project and created fake accounts to push it — and went on after being told the maintainer was real.

An hour unsupervised needs a checker and limits

TaskWhat the agent may do, and who checks
Make CI green in your projectyes, in a sandbox: the tests, and a person reviews the diff
Find a counterexample to a graph conjectureyes: a short independent script checks the counterexample
Buy plane ticketsprepare everything; a person confirms the payment
Write the literature review for your thesisa draft; you check every citation
Answer your emaildrafts only: private data, untrusted input and sending messages together are dangerous (lecture 5)
5

Back to our two agents

Did they succeed? We check ourselves

Demonstration 4

Did the agent write a correct SAT solver?

  1. The session: how long it ran, how many steps and commits. Read its REPORT.md aloud.
  2. check_solver.py run solver.py --proofs: fresh random formulas it has never seen. Every satisfying assignment is checked; every UNSAT answer is checked by its DRAT proof.
  3. check_solver.py drat on one unsatisfiable formula: the certificate that is not short.
  4. check_solver.py scan . — other solvers, processes, the answers file, file names? Then read the main loop.

Checker and starter kit: materials/sat-demo/ · standard library only

Demonstration 5

Checking the report on ETH

  1. Open the report: how many searches and sources, how long, which categories?
  2. Primary sources first: the two preprints and their Lean folders. Does the report get the dates, the bound and who reviews it right?
  3. You choose one row from each category. We open the source: does it say this? Who wrote it, and when? Is it new since October 26?
  4. Each row: correct · imprecise · not supported. Each trust rating: justified or not?
Primary
preprints, Lean code, the labs’ posts: what exactly is claimed
Specialists
the review group, complexity theorists: does it hold up?
Press
news articles: who said what, and when
Aggregators
rewritten or generated pages: dates and numbers drift

Primary sources: openai/math · anthropics/formal-math/eth

Speaker notes