Seminar 4. Setting up a coding agent: day-0 baseline or wait for a signal

Seminar 4 · ~86 min · 57 slides

01 · An agent's configuration is a set of decisions taken as the work on a project goes along: some of them pay off from the first day, some cannot be taken in advance

"The standard way to set an agent up on a new project is to open a repository that looks mature and copy its configuration across: a three-hundred-line instruction file, a dozen skills, a few connected servers. Part of that is universal engineering hygiene that pays off on any project. Part of it is decisions made for somebody else's repository, taken on signals that were never present in yours.

Today we work through this not in the abstract but on a single project from the beginning: we take a request from a client, pin down the requirements, stand up a repository, get a working prototype — and then, in the course of ordinary work, we run into the decision point 'do we write this into the agent's configuration or not' six times.

The question for the session: what is worth putting in place right away, on day 0, and what cannot be known in advance, because it is specific to your project?"

Straight on to the request from here, no pause.

02 · The task arrives in the client's wording: what the result should be is there; how the code is laid out, what the readiness criteria are and where the boundaries of action lie is not

"Here is the task in the form it arrives in: what the result should be is stated; how it is put together inside, what counts as done, and what must not be done are not stated. This text is what you sit down to work with."

The open question: "Do we hand this wording to the agent as is — or do we pin down the requirements first?"

The format is a discussion, 2-3 voices; arguments are collected, not graded. The spread is usually predictable: some hand it over as is and squeeze out the details as they go, others write the wording out first. Both positions work, and the point of the step is to get that said out loud.

If the conversation drifts into architecture ("do we even need a backend here") — log it as a separate question and move on.

Bridge: "You can hand it over — the agent will deal with what is written. The question is what it is left with when little is written. Let's pin down the requirements."

03 · The requirements break down into functional and non-functional ones and go into spec.md in the first commit; README describes how the repository is laid out and is set up in the same commit

"The first ten minutes of work on a project do not go into code. We break the client's wording down into functional requirements — what has to be there — and non-functional ones — what properties it has to have — and put them into spec.md in the first commit.

The 'acceptance criterion' line is worth a separate look. The client never said it; we derived it. Without it, 'done' is indistinguishable from 'the test is green', and those are different statements about the system.

The same commit sets up README.md. The spec answers the question of what the result should be; the README answers the question of how the repository is put together. The split is a practical one: two different readers, two different lifespans. Requirements change by agreement with the client; how the repository is put together changes as refactorings happen.

The task for the agent is now worded differently: not 'build a landing page with a form' but 'here is spec.md, initialize the repository and build the project from it'."

04 · The agent built a working prototype from the spec without a single clarifying question — and chose the repository's structure itself along the way, because spec.md does not describe it

"The directory is empty — that is the real output of the command before the run. Then the agent gets the task from the spec and in a single pass hands back an assembled project: the markup, the form handler, the package manifest with the build and test scripts, a Playwright test. The prototype works — the commands from its final answer run.

There were no clarifying questions at all; the captured session records this outright: all the parameters of the task were derivable from the text.

And here it is worth pausing on what was not in that text. The acceptance criterion was there — we wrote it into the spec ourselves. But the structure: what to name the files, where to put the handler, how to lay the project out across folders — none of that is in the spec, and it should not be; that is how the code is put together, not a requirement on the product. The agent took those decisions itself, and silently: src/main.js, tests/form.spec.ts, the scripts in package.json.

We have a prototype. From here there are many days of work ahead in this repository — and this is where the question about the agent's configuration begins."

05 · When setting up an agent in a new repository, everyone works in one of two modes — put the baseline in place right away, or wait for a real problem

Two voices; the answers are stated aloud and left without comment: the breakdown comes with the next slide.

Fallback wording if the room stays silent: "Put another way: here is a list — a readiness gate, a decision log, a repository overview, nested files, complex memory. Which of these do you set up on the first day, and which do you defer until there is a real problem?"

06 · Both modes work — the question is which type of decision they are applied to: what is specific to the project waits for a signal, universal hygiene is set up right away

"There is no single correct mode — there are two types of decision with different costs of getting it wrong.

The first type is what is specific to the project. Whether you will need it in this particular place cannot be known in advance: the answer depends on the specific repository. An overview of the structure, nested convention files, multi-level memory — these belong here. Waiting for a real signal here is not laziness but normal engineering practice: while there is no signal, there is nothing to write down.

The second type is universal hygiene. Setting it up is cheap, and the cost of not having it is unpredictable: you do not know at what moment it will be needed or what will have been lost by then. A readiness gate, tracking of tasks and decisions — these belong here. The signal here may never come: the shortfall shows up after the fact, not as one explicit event.

Separately — so as not to mix this up with Lecture 3. The thesis about a thin agent was about structural complexity: subagents, skills, external access. It still holds and still requires a signal. Universal hygiene is a different class of decision.

Six decision points lie ahead of us in the project. Which type each one belongs to we decide as we go, on the spot."

07 · Besides the spec and README, three more things go into the repository the same day — not one of them waited for a signal

"Before we go further — what appeared in this repository on the first day, besides the code.

The spec and the README — we have already seen them, they went in with the first commit. The same day sets up CLAUDE.md, the instruction file for the agent. For now it holds one line: what counts as 'done' and the requirement to check the result by hand. There is no description of how the repository is put together in it — there is no reason for one yet. DECISIONS.md is set up, the decision log: what was decided and why. And the file-per-task practice is set up — for every multi-step piece of work, a separate file with the plan and notes.

Not one of these waited for a signal, and here is why. Each of them costs one line today. And the cost of not having it cannot be known in advance and surfaces after the fact: the signal that would have made it clear that it is time may never come at all — you find out at the moment when it is already too late to go looking.

From here on all four will be met by name in the work, each in its own decision point."

08 · An agent's configuration breaks down into five slots; today's project uses two of them — instruction-rules and memory

The slide starts out empty: five empty slots in a row. The slots are not read off the screen — the room pulls them out of memory, they were worked through in Lecture 3.

The opening line: "In Lecture 3 we worked through what an assistant agent is made of on top of the loop 'plan → act → check → repeat'. In the chapter this is called the harness, and the slots are named Memory, Instruction-rules, Skills, Subagents, Access/MCP — the same names we will use from here on, on the slides and in conversation. There are five slots. Name the ones you remember."

Reveal them one at a time, with one line on each: memory — "not the context of the current conversation, but what survives the end of the session"; instruction-rules — "what you write yourself and what gets loaded at the start of every session"; skills — "a manual you take down off the shelf, not a regulation lying on your desk"; subagents — "a separate performer with a clean context"; access to the outside — "every connection is a new trust boundary". Any slot the room does not name, name yourself and move on.

Then, in one sentence: "In today's project two slots are at work: instruction-rules and memory. The other three, and the mechanical layers — hook and process — are a session of their own, Seminar 5."

The ladder rule — verbatim, under the map: "Stay on the lowest rung that closes the requirements of the task. Climb to the next one only when you can name a requirement that the current rung does not close. Every climb is paid for with new cost, new failure modes, and a new attack surface."

Bridge: "Back to the repository. The prototype is built, there are many days of work ahead with it — and the first decision about the agent's configuration is taken right now."

09 · Section 1 — the instruction file: three decision points about one and the same file, in the order in which they come up while working on a project

A one-sentence transition: "The project's instruction file — CLAUDE.md, also known as AGENTS.md. Over the course of today's project it will be the subject of a decision three times: what not to write into it, what to write into it, and where each piece of knowledge should live. The three decision points come in the order in which they come up in the work."

10 · Case 1.1 — the agent's role and a general description of the repository: what an empty instruction file gets filled with when there is one run behind you

A one-sentence transition: "The signup-landing repository has product code, a spec and a README in it. Agent configuration: zero lines. The first decision point: what the empty CLAUDE.md gets filled with."

11 · The prototype is built, the next session is tomorrow — CLAUDE.md already carries the readiness-gate line, the rest is empty, and the standard first section of such a file is called 'Repository overview'

"The prototype is built, but the work isn't finished: there are still several days of tasks ahead on this same repository, and each one is a new agent session from scratch. The developer sets up CLAUDE.md — the file that gets loaded at the start of every session — and opens it: there is one line in it, the readiness gate, the rest is empty.

Open almost any public repository that has such a file and it starts with a "Repository overview" section: what this project is, how it is put together, where things live. That is an established convention, and the question of the first decision point is exactly about it: which of that is worth writing down here, in this repository, at this step."

12 · A choice among five options: which of the items listed will go into the file now, before the next session

The question is asked with the set of options already on the screen. Two or three voices. Answers are recorded neutrally and without judgment: "repository structure — noted", "list of technologies — noted".

Any option that comes up is recorded the same way, including "I'd describe the architecture": "Got it, we'll come back to that in a minute."

The vote happens with not a single number on the screen — the evidence and the breakdown come on the next slide, in one pass.

13 · A repository overview measurably does not raise success and does cost context — and a structured description, if it is needed, lives in README.md and is not duplicated in CLAUDE.md

"The controlled experiment by Gloaguen et al. was the first to compare against a third arm — no file at all. The result for a repository overview, verbatim: "are not helpful". Cost goes up by twenty to twenty-three percent, the success rate does not. The only confirmed exception is repositories with no other documentation at all.

The phenomenon has a name: presence paradox. The presence of text in the context costs money regardless of whether it helped solve the task. We will need the term twice more: in cases 1.2 and 1.3, and later in the section on memory.

Briefly on the remaining sources: the ablation confirms the mechanism — they removed the README and the overview became useful. Lulla et al. is a strong argument for the file, but about non-discoverable conventions, not about a general description. Shepard & Albrecht — static, unverified guidance is worse than dynamically verified guidance.

On the gap, honestly: there are no direct experiments with a deliberately stale overview. The failure mode you do see in practice — the description survived a refactoring and the agent edits the file according to it rather than according to the code — is plausible, but unmeasured, and we are not going to pass it off as proven.

Now the breakdown. For our repository the target answer is "nothing yet": the agent sees the six-file structure itself, and the description of the project is already in the README. A structured description must exist, but not duplicated in the instruction file. The answer at this step is not "what text to write", it is "none for now"."

14 · A description of the repository's structure does not appear in CLAUDE.md: there is no signal for it — and the file's first content will appear in the very next case, for a different reason

"The outcome of the first decision point: there is no description of the structure in the file — there is no signal for it, while the cost of its presence is real.

But an empty file is not a position for the whole project. Part of the content of CLAUDE.md will appear in it as early as this week, and for a different reason: not because something broke, but because it is a baseline practice that costs little. That is the next decision point."

15 · Case 1.2 — baseline processes: the readiness gate and the 'plan → code → improvement' loop, written into the instruction file

Transition in a single phrase: "The first decision point was about what not to write into the file. The second one is about what has already been sitting in it since day one: the readiness gate. We are in the first week of work on the repository."

16 · The readiness gate has been sitting in CLAUDE.md since day 0 — and twice in a row 'done' was said before anyone checked by hand that the submission was delivered

"We write the readiness gate into the instruction file the same day as the spec: always run the tests and verify the result by hand before you say done. The line is cheap, and it has been in the file from the very beginning.

It is the first week. The client writes: I submitted a request from your site — it never arrived. The agent reads the code, finds the problem in the form submit handler, fixes src/main.js, runs the test. The test is green. The agent writes: done. And our test checks exactly one thing — that the form does not submit with empty fields. It knows nothing about delivery. You close the task, and the client writes again: it still is not arriving.

You remind the agent about the gate by hand — this time it opens the page manually, finds a second cause, fixes it. Two days later, in a new session, on a different bug — the same thing: automated test green, "done", nobody checked by hand.

The gate line had been sitting in the file all those days and was loaded at the start of every session. Both times, "done" was said before the manual check."

17 · The gate line has been there from the very start and did not fire — a choice among five options for what to do next so that it actually fires in the next session

A pause, 2–3 voices — with not a single number on screen. Every option voiced is recorded the same way, including "I'll rewrite it in caps": "Noted, we'll come back to this in a minute."

18 · A textual gate without tooling is not merely useless but harmful — regressions rose to almost 10%; none of the room's five options solves the problem systemically, except handing the check over to a mechanism outside the text

"Here the result is genuinely counterintuitive. The TDAD study took a purely procedural instruction, "write the tests first", without a single change to the tooling — and regressions did not fall, they rose to almost ten percent. Worse than no instruction at all. The improvement only appeared once the same requirement was wired into a skill or into the tooling rather than into text. And separately, for the "plan - code - improvement" loop: self-critique with no external signal systematically makes the result worse — ICLR confirms that firmly. Self-repair with a real test run, on the other hand, gives plus five to thirty percentage points. The difference is fundamental, not terminological: did the agent look at its own code, or did it run a real check.

There is no separate controlled experiment on an isolated gate line in an instruction file — here we have practitioners' best-practice observations, not a measurement of its own.

Now the breakdown. The option "I'll rewrite it in caps" is exactly the path TDAD tested: they strengthened the wording with text, with no tooling. Regressions rose. Caps do not turn a request into a law. "I'll duplicate it as a second bullet" — the presence paradox from the first case: an extra line costs context in every session. "A separate checklist file" — the same class of solution, text again, a request again. "I'll live with it, that's normal" — for a rule whose violation is expensive, that is a decision not to decide.

And only one option out of five — "I'll hand the check to a mechanism outside the text" — works systematically: the mechanism gives twenty percentage points where switching the model gives one. That is the right direction. The mechanism itself is on the next slide."

19 · Rules in the prompt are requests, rules in code are laws; a textual gate catches an honest mistake but does not hold under pressure — the boundary for a mechanism runs where the action, the cost and the frequency are named

"The option "I'll rewrite it in caps" is exactly the path TDAD tested. Caps do not turn a request into a law. And none of the five options except "I'll hand it to a mechanism outside the text" solves the problem systematically — and we are not going there today, that is the next session. So the honest answer today is: we will still leave the gate as text, because that is better than nothing — but we will tell ourselves plainly that it is unreliable, even having been set up on the first day, and we will name the direction to go when it lets us down a second time.

And here is the formula that holds today's whole breakdown together: rules in the prompt are requests, rules in code are laws. The textual gate we have had since day one is a request, and that is fine for a basic practice. But calling things by their names is part of the job: what we wrote down as text did not become a law just because we wrote it down proactively.

Now a direct question worth answering here rather than leaving it hanging: can we get by without a hook today — and if we can, why are we talking about it at all.

We can get by today, and here is the basis for that. A textual gate catches the honest mistake: the case where the check was not worked around, people simply did not remember it. That is real work, and it costs one line. In our case you can see it literally: out of three times the gate worked once — the time it was mentioned out loud. What it does not do is hold under pressure. A deadline, a rush, "the test is green, let's report it" — and the line in the file stops existing. That has been measured too: strengthening text with text raised regressions to almost ten percent, while the mechanism gives twenty percentage points where switching the model gives one.

The boundary does not run along how important the rule is, but along whether you can name the specifics. A hook is needed when the cost of a violation is high and predictable: there is a concrete action, a concrete command, concrete damage. Until the action has been named, it is too early to set up a hook — it charges you on every call and it checks a pattern match, not meaning.

In our project the action has in fact been named: "done" without a manual check, twice in a row, and the cost is that the request never arrived while the client has a demo coming up. So the honest conclusion today is not "the text is enough" but "the text is enough up to exactly this point" — and from there the material of the next session begins.

And that gives the answer to the question "why do we need to know about the hook if we are not setting one up today". So as not to mistake a textual gate for a solution. A person who does not know about the next level strengthens the text: in caps, with a duplicate, with a third checklist — exactly the path TDAD measured. Knowing about the hook gives you the criterion by which you recognize the moment: name the action, the cost of a violation and the frequency. Named them — time to set up a hook. Have not named them — the text is enough.

And the second point of the breakdown, about "improvement": if it is "agent, look at your code again" — do not do that, the evidence is unambiguous. If it is "agent, open dist/index.html and submit the form by hand" — always do that."

20 · The final file is fifteen lines: the 'done' criterion, the gate warning, an honest caveat about its limit, and one one-off step without which the tests fail for no good reason; AGENTS.md is a symlink to the same file

"Here is the result of case 1.2: fifteen lines — the readiness criterion, the gate warning to verify delivery by hand, an honest boundary on deploys, and one one-off step after cloning without which the tests fail instantly and for no good reason. The file is loaded in full at the start of every session; the official guideline is up to two hundred lines, and we have fifteen. @-imports organize the file but do not save context — what is imported is still loaded in full.

A word about that last line on its own. npx playwright install chromium is exactly the kind of step the 'Build, test, verify' section exists for in the first place: without it, a clean clone fails every test at once, and fails them for no good reason — not because the code is broken, but because there is no browser on the machine. An agent that has read this will not start repairing code that works.

AGENTS.md has been with us since day one — a symlink to this same file, one source of truth for different engines. Note that it is set up right away, and it holds exactly as much content as CLAUDE.md, because it is one and the same file. The sharp edge here is not that some system behaves incorrectly, but that the behaviour of cp -R on a symlink is simply not defined by the standard: POSIX states outright that with -R and none of -H, -L or -P, the default is unspecified. On GNU coreutils the symlink survives the copy, but that cannot be relied on — -P states the intent explicitly. What does break the symlink for certain is -L: cp -RL will give you a second real copy, and from then on the two files quietly drift apart. The safest way to move the project is git clone or git archive."

21 · The gate is not the only cheap baseline: three more practices get set up on the first day and do not wait for a signal

"It is easy to take the wrong conclusion away from case 1.2: the only thing worth setting up in advance was one line, which we have just been told is unreliable. So there is not much to do in advance. That is not so, and here are three things that get set up on the first day.

The first one is the create, roast, improve loop. A draft of the plan, then a pass over it with a fixed list of questions: is this the simplest version that works; which assumptions are unverified; is there a premature abstraction; does every new file have an owner; are risky changes bundled into one batch. Then the edits — and only then show it and get sign-off. A minute ago we read that self-critique with no external signal makes the result worse — and this is not a contradiction. There are two external signals here: the list of questions, which does not depend on the text just written, and the human at the "show it" step. And what gets roasted is the plan, not finished code — that is where an edit is cheapest. The team making this course works exactly this way: the loop is written into its instruction file as mandatory for any non-trivial task.

The second one is a progress log for the task, in any form. A file per task, a list of steps with marks, just a log. Everything that is held only inside the session is unavailable by the next session. How such a log is kept and where it breaks — a separate case in the section on memory.

The third one is that a rule is written together with its reason. Next to the rule, a line: which case gave rise to it. A rule with no reason reads like a matter of taste a month later and is the first to be dropped. A rule with a named cost of violation is a ready-made criterion showing when the text needs to be backed by a mechanism. That very criterion: name the action, the cost and the frequency."

22 · Case 1.3 — where the project's documentation physically lives: the address of a piece of knowledge, not its content

A one-sentence transition: "Two decision points were about what to write into the instruction file. The third is about where to keep the project's documentation at all: a month has gone by, and more knowledge has piled up than fits in a single file."

23 · Over a month two pieces of knowledge of different natures have piled up — a convention for a whole folder and a detail about a single function — and the developer treats both the same way: by setting up nested instruction files

"A month of work on signup-landing. Over that time two pieces of knowledge have piled up in the repository that are in neither the root file nor the project description.

The first: in the tests folder an understanding has settled in — how check files are named, and that the page is worked with through a page object rather than through selectors directly. That is a rule for the whole folder: it is needed every single time somebody touches the tests.

The second piece is of a different kind. The form-submit function retries the request once on timeout — which means the handler on the server has to be idempotent, otherwise the application gets submitted twice. This knowledge is needed by exactly the person editing that function, and by nobody else.

The developer treats both the same way: he sets up tests/CLAUDE.md and src/CLAUDE.md. The bet is that the agent will read the file by itself when it works in that folder. That fixes the state; next comes the question."

24 · Where to put each of the two pieces of knowledge so that the agent gets it when it is needed and does not pay for it in all the other sessions — a choice among five cards, more than one allowed

A pause, 2–3 voices — with not a single number on screen.

Say this before the vote, not after: more than one card may be picked, and the two pieces of knowledge may be split across different cards — the question does not call for one answer covering both.

Every option voiced is recorded the same way, including "I'll just drop it in somewhere and be done": "Noted, we'll come back to this in a minute."

25 · 'Where to put it' is the question 'will the text reach the agent, and what does it cost'; five channels have five different answers, and three tools out of four independently converged on one mechanism — a declared path pattern

"The question 'where to put it' is really the question 'will the text reach the agent, and what does it cost'. The answer is mechanical, and it differs across the five channels.

The root file and every directory above it load at the start of the session: it always arrives, and we always pay. An import via the at-sign is expanded at launch too — it organizes the file, it does not save context; we did that arithmetic in the previous case. A comment in the code arrives when the agent has opened the file — and it has opened the file if it is editing the function. A nested file in a subfolder arrives only when the agent has read something out of that folder, and even then only if the tool can do that at all.

Next comes the fact this case exists for. Three tools, independently of one another, built the very same mechanism: a rule with a declared path pattern, sitting in one shared rules directory. In Claude Code that is the paths field, in Cursor globs, in VS Code applyTo. The wording is the same everywhere: the rule is attached when the agent works with a file matching the pattern. The trigger is declared explicitly, and the file sits where the tool always looks — you don't have to hope it wanders into a subfolder.

And 'just put a file in the folder', across those same four tools: closed as 'not planned', acknowledged as a bug by an employee, marked experimental and off by default, not implemented at all. Four tools, four different refusals.

The promise everyone leans on comes not from a tool but from the agents.md standard: agents automatically read the nearest file in the directory tree. That is a description of typical behavior, not a requirement. The promise goes unmet because nobody ever made it."

26 · Neither of the two controlled measurements found that spreading documentation around improves the result — so the channel is chosen by delivery and cost, not by promised benefit: knowledge about a function goes into the code, about a class of files — into a rule with a path pattern, about the project — into the root file as pointers

"Now the uncomfortable part. Two controlled measurements checked whether any of this helps at all. The first — sixteen hundred and fifty sessions, four structural variables of the config file: not one of them has a significant effect. The second, more recent — two hundred and eighty-eight runs across two agents and three repositories, three modes: no file, the file in every turn, separate documents to be loaded on demand. For Claude all three land within two percentage points of each other. For Codex the 'no file' variant turned out to be the best one. There is no significance. And the failure analysis showed that the tasks fell over on implementation craft, not on a shortage of knowledge about the repository.

The authors make the caveat themselves, and we will repeat it: across fifteen tasks the detectable effect is more than thirty percentage points. An effect of ten they would not have seen. 'Not found' is not 'not there'.

What follows from this in practice. We choose the delivery channel not because somebody proved it useful — nobody proved that. We choose by the two things we know for sure: does the text arrive, and what does it cost in the sessions where it isn't needed. Hence the rule of three addresses. Knowledge about a single function — next to it, in the code: the only channel where delivery follows from the mechanism. Knowledge about a class of files — into a rule with a declared path pattern. Knowledge about the whole project — into the root file, briefly.

And the answer to the question that usually hangs in the air: where does the corpus live. The corpus is not a separate document the agent is supposed to find. The corpus is a table of contents inside the one file that loads at launch: one line per document, where it sits and when to go there. Everything else — 'we have a wiki, go take a look' — is a hope, not a delivery channel.

We know the cost of that corpus too, from the only work in which it was built out in full: an hour or two a week to keep it up, and the main failure mode is going stale. Verbatim: agents trust documentation absolutely, and stale specifications produce silent failures. That is the experience of one developer on one project, with no control group — but we have no other numbers on the cost of a corpus."

27 · The root file stayed the same, fifteen lines long: the detail went into a comment on the function, the tests' convention already sits in the tests themselves, not a single nested instruction file was set up — and knowledge that appears as the work goes along has a different address, and that is section two

"On our repository the root file has not changed — the same fifteen lines. The detail about the retried submit went off into a comment on the function itself: you edit it, you read the file, you see the comment. The test convention did not get set up as a separate file, because it already sits in the three existing tests: an agent that opens any one of them sees the page object in the code. The same logic as in the first case — whatever is derivable, the agent will derive by itself — and with the same honest caveat: it follows from the mechanism, it has not been measured directly. Neither tests/CLAUDE.md nor src/CLAUDE.md ever appeared in the repository.

The threshold for each address. A comment in the code — no signal needed, it is useful to a human as well, and the agent gets it for free. A rule with a path pattern — when the convention has stopped fitting into the root file and you can name the pattern in a single expression; while it is two lines long, it's too early. A corpus pointer — when a document to point at has appeared, and a person who maintains it; a pointer with no document is a line of context in every session for nothing. A nested file in a subfolder — never as the only guarantee: only if you have checked through /context that your tool, in your version, picks it up, and are prepared to re-check after every update.

Three decision points, one and the same file: we decided what not to write into it in advance; we wrote down the readiness gate that we set up on day 0; and we decided that knowledge has an address — and a nested folder is not one. All of that is knowledge you already had before the work started: written down in advance, or checked on the first real task.

But there is knowledge of another kind: the kind that appears in the course of the work itself. You cannot write it down in advance, because you do not know it yet. And it has an address too. That is section two — memory."

28 · Section 2 — memory: what survives the end of a session, not what you wrote to the agent in advance; case 2.1 — the project's decision log

The transition in one phrase: section 1 closed the question of authorship — what you knew in advance and wrote down yourself. This section opens the question of memory — what appears in the course of the work and has to survive the end of a session. The first decision point is about the project's decision log, and we are going back to the same first week as in the second case.

29 · The end of the first week: a rejected decision about validation surfaces again — DECISIONS.md was set up on day 0, but there is no entry about this decision in it

"End of the first week. The client sent in a request with a typo in the email domain — and the form accepted it. You ask the agent to tighten the check. It reads the code and says: let's pull in a form-validation library, here's a suitable one. The very one you discussed and rejected a few days ago — the form has two fields, native required and pattern are enough.

The state of the repository at this moment: DECISIONS.md is there. It was set up on day 0, the same day as the spec and the README — like everything that is cheap and gets set up right away. There is no entry about the validation library in it. The decision was voiced in a working discussion, and between "we discussed this and decided" and "a line appeared in the file" there is a step that does not happen by itself.

And this is not "the agent forgot". The model has no state between calls: every request is processed independently, and what looks like a memory of the discussion is the text history sent in again. When the session ended, what was not written to disk was not forgotten. It is no longer there."

30 · Choose where to record the rejected decision — from the cards below — so that you do not have to explain it a third time

The question is read verbatim off the screen — the scenario and the diagnosis were already given on the previous slide, so do not retell them here, only the short one-line reminder at the top. The second half is deliberately built into the question — "and why not in the instruction file we have just set up" — without it the room will give an answer that is right in form and will not see that the layers are different. A pause, then 2–3 voices. The facilitator does not comment on the substance of the answers on this slide, only notes them out loud: "got it", "noted, let's move on". The breakdown, the numbers and the link to the evidence come on the next slide.

31 · The instruction file is loaded automatically at the start of every session, while DECISIONS.md is an ordinary repository file that the agent reads only on its own initiative or on an explicit reference; the presence paradox carries over to it in a conditional, but all-or-nothing, form

"Before we go through the answers — one mechanical thing that is easy to mix up, and expensive to mix up. In case 1.3 we worked through auto-loading from the official documentation: the root instruction file and all parent ones are loaded by themselves when the session starts, nested ones — only once a file from that folder is read.

DECISIONS.md does not fall under that rule at all. It is an ordinary repository file, no different from src/main.js: no tool feeds it into the context automatically. The agent opens it if it decided to open it — or if you pointed it there.

Hence a correction to the presence paradox. In case 1.1 a controlled experiment showed: the mere fact that text is present in the context costs plus twenty to twenty-three percent, regardless of whether the text helps solve the task. For a decision log this holds conditionally: the cost arises not in every session, but in every session where the file was read. But it is read in full — nobody reads a file for the one entry they need. Thirty entries on different topics are paid for in full for the sake of the one that is relevant today.

No separate measurement on decision logs specifically has been carried out — this is a transfer of a mechanism to a new object, not new proof, and that is how it is presented.

And the flip side, which is more honest to name right away: an entry the agent did not open costs nothing and gives nothing. In our final instruction file — the fifteen lines from case 1.2 — there is no pointer to DECISIONS.md. Which means we currently have no guaranteed path from the auto-loaded layer to the decision log. That is an open question, not a closed one."

32 · The difference between the layers is not the format but who initiates the record, who will see it later, and by what mechanism it gets into the context at all

One criterion of distinction, in one phrase: the difference is not in the format — the first three layers are all markdown — but in who initiates the entry, who will see it afterwards, and by what mechanism it gets into the context. You write the instruction file in advance, and it loads itself. Auto-memory is written by the agent, for itself, on this machine, without review. You write the decision log, for the team, into the repository, through review — but it does not load itself: the agent opens it on its own initiative or on an explicit pointer.

The fourth column is an external memory system: a vector or graph database, a separate service. There the entry is not composed by a human: the service extracts facts from the work itself and decides by itself what to supply into the context on request. On the plus side — it survives a change of machine, unlike auto-memory. On the minus side — nothing sits in the repository, which means no change history and no review, and you do not control the wording of the entry. And the gain from it is not automatic: on small corpora a simple file regularly beats it — the measurements for that are on the failure slide.

Four details, briefly: the memory index is only an index, beyond two hundred lines the content physically does not make it into the session; the agent itself skips what is derivable from the code; only files on disk survive context compaction — what was only said in conversation can disappear when the context is compacted; and complicating the layer does not improve the result by itself. The third detail is a direct answer to the previous section: the plan of "I'll explain it to the agent again as we go" does not work even inside a single long session.

Now, with this table in hand, let's go through what the room chose.

33 · A team decision with its rationale goes into DECISIONS.md, a personal preference into auto-memory, and anything derivable from the code nowhere

"The layers are in front of us — now to the options that were named.

"I'll add it to CLAUDE.md" — mechanically it will work, there is nothing to argue about; but imagine thirty such decisions over half a year — the file is read in full every session, the guideline is two hundred lines. "The agent will remember on its own" — answer not with "no" but with the mechanics: between two calls the model stores not a single bit. "I'll say 'remember this'" — half the answer, it goes into auto-memory, a machine-local layer with no sync and no review. "I'll write it in DECISIONS.md — it already exists" — acknowledge it: correct, the file is already there, there is nothing to create. The question is not "where to set it up" but "who actually writes into it, and when, at the moment of the decision" — and how the agent is going to reach that file, given that it does not load itself. "I'll install a memory system" — acknowledge that the direction is reasonable and defer it to the failure slide.

The target answer — both mechanisms at once, with the roles split — and that split gets assembled at the next step, live, from the cards."

34 · Not everything the agent has learned is worth recording at all — the third basket is just as legitimate as the first two

The facilitator takes the cards one at a time and asks, briefly: "which one?". The comment on each is one phrase, no more.

Two cards carry most of the load. "The form handler lives in src/main.js" — almost always someone proposes writing it down: the agent will read that in a second by ordinary file reading, while the memory index is loaded into every session and has a hard limit. "On September 24 a test failed because of a timeout" — this is the card the room most often pulls into DECISIONS.md: it is a "what", not a "why", and the "what" is already in the change history.

For the "nowhere" basket, the mechanics are stated once: the space in the memory index is finite, and everything put there "just in case" occupies it in every session that follows.

After the sorting — the facilitator asks part by part: "the date?", "the decision in one line?", "the reason?" — and reveals on the slide the entry that actually sits in the demo repository. It is important to say out loud that this is a real file from a real repository, not an example made up for the slide, and that the file itself has been in the repository since day one — only the line is new.

35 · What is new is not the file (it was set up on day 0) but the first substantive entry in it, dated the day the decision was recorded

Two mechanisms, not one. The runtime's auto-memory lives outside the repository, in a memory directory on the developer's machine: an index of the first 200 lines or 25 KB plus topic files that the agent creates itself. The command "remember: no third-party widgets in the form" creates a topic file exactly there — it does not make it into the tree on the right.

DECISIONS.md is the second mechanism, a git-tracked append-only log, set up on day 0. What is new here is not the file but the first substantive entry in it, and its date is today's: the decision was made a few days ago and was written down only now. This is a real file from a real demo repository, not an example made up for the slide — and the date in it is the real one, the one the commit actually carries in git. We deliberately do not reconcile the session calendar ("the end of the first week") with the git dates: the repository was assembled over two days, while the session runs on its own clock.

36 · Memory is read by default as trusted context rather than as data — and switching memory on does not necessarily improve the result

Two layers of a different nature, and they must not be merged: the first is about security, the second about effectiveness.

Poisoning — told as a story, with a date: September 2024, the ChatGPT desktop app for macOS, researcher Johann Rehberger. The agent read an external text — and an instruction was embedded inside it. That instruction went not into the conversation but into long-term memory. The conversation ended, the memory stayed. The vendor closed that specific leak channel the same month. Nobody closed the principle: by default the agent reads the contents of memory as trusted context, not as data.

The scale — with an explicit baseline: a dozen poisoned templates captured on the order of half of all subsequent memory retrievals, against zero before the poisoning; in an adjacent experiment five planted documents were enough to steer the output in more than nine cases out of ten.

The practical conclusion — as it applies to our file: today we wrote down our own decision, and here everything is fine. But the moment the agent starts writing into memory conclusions drawn from someone else's issue, someone else's page or someone else's ticket — that is already a different level of trust.

The second layer does not hit your paranoia, it hits your intuition. An independent run of one popular memory system: with memory — forty-six percent, without memory — fifty-seven point six, the same system, the same benchmark. In a live comparison of six memory systems against the trivial "put the whole session text into one file", some of the systems lost to that flat file by twenty-five to sixty-seven points. An honest caveat about the conditions: the corpora there are small, they fit into the context in full — but a small corpus is exactly our case today.

And a third detail: on twenty-four tasks, on four of them — seventeen percent — the model flatly refused to write down the fact that was needed. This is not a bug — by construction the model decides for itself what is worth writing down. The conclusion for an engineer: completeness of memory is not guaranteed, and if a fact is critical — you write it by hand.

37 · A flat file is enough as long as the history of decisions fits into the context whole and there are no links between the entries yet

The criterion — off the checklist, out loud, briefly.

Then the boundary is named directly: a flat file stops coping not instantly and not because of size in itself, but when two conditions hold at once — there are so many entries that finding the one you need by reading linearly takes longer than recalling and re-explaining the decision from scratch, and links have appeared between the entries for which there is physically nowhere to put them in a flat append-only log: decision number seven cancels part of decision number two, and next to decision number two there is not a word about it. This is not "we need a vector database" — this is exactly the next case: the very same decision log, only grown and now requiring structure.

38 · Case 2.2 — structured wiki memory: the same decision log four months later

A one-phrase transition: "The previous case ended at a boundary: a flat file stops coping not when it gets big, but when it has accumulated links that a linear log has no room for. Here is that same boundary played out — month four of work on the repository."

39 · The fourth month: DECISIONS.md holds two hundred entries, two of them are about the same question — and neither references the other

"The previous case ended at a boundary: a flat file stops coping not when it gets big, but when it has accumulated links that a linear log has no room for. Here is that same boundary played out.

Month four of work on the repository. The form started getting spam. By now there is a second person on the team, someone who was not around for the earlier discussions, and they ask the agent to put protection in place. The agent honestly reads the decision log — there are two hundred records in it by now — and answers from the record of September twenty-sixth: no third-party widgets in the form. So a captcha is out, and it proposes reopening the question from scratch.

But on November twenty-eighth a second record appeared in the log: anti-spam is a honeypot field, no captcha. It settles exactly this case, and it sits a hundred and fifty records further down, near the end of the file. Neither of the two links to the other.

Both records are in the same file. There is only one way to find both in a flat log — read all two hundred in a row."

40 · A decision log that has grown out of hand and a lost link between two entries — a reason to choose how to reorganize it, not to set it up again from scratch

The wording verbatim. The facilitator reads out the question and the five cards, takes 2 responses from the room — and gives no numbers on this slide: the evidence comes on the next one.

The answer "I'll install a graph memory system" is not taken apart here: it is recorded out loud on the same footing as the rest, and the comment comes on the next slide.

41 · Flat memory degrades over time, and structure can pay off for an agent precisely where it does not pay off for a human — but 'smarter' does not mean 'better' without a check

"Before I show any numbers — a short honest remark. Almost everything I am about to say about structured memory is research from literally this year, preprints that have not been peer-reviewed yet. The niche is growing faster than science can check it. I will say 'claimed in a preprint', not 'proven'.

There is a measurement: flat memory degrades over a horizon of roughly three days of work — minus fourteen percentage points in tool-call success. There are three reasons: the context saturates, older records lose weight, and the meaning of a record drifts away from the current state of the project.

And here is an interesting argument for why structure may pay off for an agent where it does not pay off for a human. A human sets up a wiki, and it dies: maintaining it costs more than it is worth. For an agent, the cost of editing a dozen and a half files in a single pass is almost zero. That is an argument from mechanism — there is no direct measurement of 'structure versus a flat log' at our scale.

And a counter-example. The Letta benchmark, the long-term memory task LoCoMo: a file store — automatically parsed and indexed, with semantic search available to the agent, not only grep — seventy-four percent. The graph-based memory system Mem0 — sixty-eight point five. The simple store beat the structured one. The reason the authors name: models are trained on file operations as the most familiar tool there is. We will need this number once more in the breakdown."

42 · Splitting it into separate files (one per decision) with cross-references is the target answer, but not a free one: the structure itself creates a new load

"'Leave it as is, just grep it' — acceptable while there are dozens of records; but we have just seen that flat memory degrades over time, and our file has already outgrown that boundary. 'I'll install a graph memory system' — well, what we have is essentially a graph of decisions. The target answer is still a different one: separate files, one per decision, with cross-links, not a graph system. The Letta benchmark showed it: a simple file store beat the graph-based Mem0. Structure is not a guaranteed win; it wins here because it solves a specific problem — the lost link between decisions — not because 'structured is always better than flat'.

'One shared SUMMARY.md' — rewriting in full kills the very append-only guarantee. 'The agent will find it itself by searching' — the same fallacy as 'it'll remember on its own' from the previous case: decisions are not facts about the code."

43 · The ADR practice: one decision — one numbered file, and the link between decisions is a line in the Status section that the tool fills in both directions at once

"We did not invent the structure here — we took a ready-made one. Architecture decision records, ADRs, were described by Michael Nygard in 2011, and the practice has a reference set of commands — adr-tools. The arrangement is exactly what you see on the slide.

One decision — one file: a four-digit sequential number plus a hyphenated title. Inside — a heading with the number, the date and four sections: Status, Context (what forced the decision), Decision (what was decided), Consequences (what gets easier and what gets harder).

The key part for our case is the Status section: besides the status, it holds links to other decisions. The November record carries Amends — it amends the earlier one — and a link to the September record; in the September record the tool added the matching Amended by line. And that is exactly what the flat log did not have: an agent that opens the September decision no longer has to read the file to the end — the link to the November amendment sits right there in it.

The key words are 'the tool added': the link in both directions is placed by a command, not by the author's discipline. The -s flag means 'supersedes' and moves the old record from Accepted to 'superseded by such-and-such record'; the -l flag simply links, leaving the status alone. Ours is the second case — the November decision does not cancel the September one, it amends it for one question. The table of contents is not hand-written either: adr generate toc prints the list of all records with links, and that goes into README.md.

Honestly about the strength of evidence: this is a practice for humans. It has been documented since 2011 and is widely used — a 2026 study collected more than four and a half thousand ADRs from seven hundred and fifty projects out of open repositories. But there is not a single controlled measurement that 'ADRs help a coding agent'. The content of our two records is illustrative. A snapshot of the demo repository does exist — the doc/adr/ directory, commit 3b4b289 — but it shows a different scene: it holds three records, there is no Amends / Amended by pair between them, and adr-tools was never installed there, so the table of contents was written by hand. The structure, the sections and the format of the link line come from the adr-tools template and sources — they are not invented."

44 · A file store beat graph memory — but what was measured was questions about a long dialogue, not finding a related decision in a project's log

"Before we accept this number, let us work out what stands behind it — otherwise it means something other than it seems to.

LoCoMo is a benchmark for memory in very long dialogues. One example is a conversation between two fictional interlocutors: up to thirty-five sessions, about three hundred turns, on the order of nine thousand tokens. Then they ask questions about facts that came up in that conversation and count the share of correct answers.

The company Letta put together the simplest possible solution: they dropped the whole conversation into a file and gave the agent three tools — semantic search, a text grep and 'answer'. The model was gpt-4o-mini, not the strongest one. The result was seventy-four percent. Mem0, a specialized graph-based memory system, has sixty-eight and a half in its own publication. The authors' explanation is simple: file operations were in the training data, the agent handles them confidently, and an unfamiliar retrieval mechanism, however smarter, regularly loses to a familiar one.

Now three caveats without which this number cannot be used. First: this is not a single run — the numbers come from two different publications; Letta's own authors call such a comparison 'apples to oranges' and write that today's memory benchmarks may not be very meaningful at all. Second: 'a file store' here is not 'grep only' — the file is indexed, and the agent did have semantic search. Third, the main one for us: what was measured was questions about facts from a conversation — not finding a related decision in a project's decision log. Nobody has measured our case.

What we take from this: structure by itself guarantees no win. Next — what it costs."

45 · Four costs of structure — not general categories but concrete files in our own doc/adr/: a generated table of contents, a redundant entry, an orphaned rationale and manual reconciliation

"And since we have set up a structure — let us be honest right away about what it costs. I will show it on our own directory.

Staleness. The table of contents was generated by a command in October. Records one hundred and eighty-eight through two hundred never made it in, yet it looks complete — an agent handed that table of contents will not see the November decision. In one flat file there was nothing to diverge.

Duplication. Someone did not find record one hundred and eighty-seven and set up record one hundred and ninety-three about the same honeypot. Both are accepted, and there is no link between them. The format does not forbid two decisions about the same thing, and the tool does not look for similar ones.

No cascading delete. Suppose the September decision was superseded — the tool honestly changed its status and placed the links. But the November record is justified by the phrase 'a captcha is banned by decision thirty-one', and it stayed accepted even though its rationale no longer holds. The tool fixes links, but not the meaning of a derived record.

And maintenance. Two hundred decisions — two hundred files: a number, a date, a status, links in both directions, a regenerated table of contents. Part of that is done by commands. But noticing that a new record is about the same thing as record one hundred and eighty-seven is done by nobody.

For people it is exactly this burden that usually kills a knowledge base like that — that is an observation, not a measurement. For an agent, editing a dozen and a half files in a single pass costs almost nothing, and it may turn out differently for it. That is a hypothesis about a mechanism, not a measured fact.

Structure is a trade-off. Not a free upgrade."

46 · A handful of links per entry and one or two people editing the memory — the structure is enough; a full memory system requires measurements at your own scale, not general advice

"If a record has a handful of links rather than dozens, one or two people edit the memory, and nobody has asked to 'find decisions semantically similar to this one' — what we have just built is enough for you. A full-blown memory system is the next step, and it requires measurements at your own scale, not general advice: we have just seen that 'smarter' does not mean 'better' without a check.

Now the change of axis. Everything we discussed in this section is about decisions that live forever. But what happens to the memory of a single task while it is not finished yet? The next case is about exactly that."

47 · Case 2.3 — working memory: not 'what has been decided forever' but 'what is happening right now in one task'

Transition in one sentence: "Both previous decision points were about decisions that stay in force for good. Now a different question: what happens to the memory of a single task while it is still unfinished?"

48 · The task file was set up on day 0 and the plan in it is ticked off for the first steps — but the agreement on how to do the next step, reached during the first session, never made it in

"The 'a file per task' practice is already set up on day 0 for us, next to the decision log: any task longer than one session gets its own file — a plan and a progress log. The task here is this: split the form into a step-by-step wizard — name, email, confirmation — keeping all the current validation intact. The file is created before the work starts, the plan is laid out in six steps.

The first session closes the first two steps. In the middle of the second step it becomes clear that the field values have to be held somewhere between steps, and right there in the work — not in the plan — a way is chosen: hold the wizard's state as a single object in sessionStorage, under the key signup-wizard, so that reloading the page on the third step does not wipe the fields already filled in. In the plan the third step is written down as a single line: 'state of the current step'; how exactly to do it is not said there. The chosen approach never makes it into the progress log. The session breaks off in the middle of the third step, the context is compacted.

The second session. The agent opens the file: the plan is there, two steps are checked off as done, the third is open. There is not a single line about sessionStorage in the file. The agent does the third step the shortest way — holds the state in a module variable, saving it nowhere. Switching between steps works, the fifth step's tests are green. The divergence surfaces only at the sixth step, during the manual check of the full walkthrough: a reload on the third step clears the form — exactly what sessionStorage was chosen for in the first session. The third step is redone from scratch, along with everything that had already been built on top of it.

At the entry into the second session the agent has exactly what is in the file: six steps and two marks. Everything else was in the context of the first session, which no longer exists."

49 · The task file existed from the very start and still misled — a choice among five options for what to do differently

The question is asked exactly as written on the card. The facilitator reads out the question and the five cards, calls for 2 voices — and gives not a single figure and not a single reference to a source on this slide.

One reaction is worth preparing for in advance, but it is acknowledged aloud without any comment on the substance: "Append it to DECISIONS.md" sounds like the right answer from the previous case — the facilitator simply notes the option and moves on, the comment comes on the next slide.

50 · The task file is precisely the written plan the next session works from; that is why the case runs into the measured question of incomplete and stale plans

"First one link, without which nothing further adds up. The task file is the written plan the next session works from. The agent that opens it does not remember the first session: it reconstructs the task entirely from what is written in the file. And it has nothing to tell 'this was agreed but not written down' from 'this never came up' — whatever is not in the file is not forgotten for it, it simply is not there.

Two consequences follow. First: the completeness of the task file is not tidiness for tidiness' sake, it is literally the completeness of the plan the executor will act on with no other sources. Second: so our decision point is a special case of a question that has been measured — what happens to an agent's work when the plan it is guided by is incomplete or has drifted away from reality.

First, about what people do on this score. A precedent for the pattern from Anthropic's engineering blog: an agent plays Pokémon for thousands of steps in a row, surviving context resets — and manages it thanks to simple structured notes: counters, the current goal. Manus has a similar technique called 'recitation': the agent continuously rewrites its to-do list to the end of the context — a model follows what it saw last more readily. On tasks on the order of fifty tool calls this noticeably reduces goal drift. And there is an open project, metasphere-agents, where this is done exactly the way we are discussing: a file per task that moves to an archive on completion.

And now — why the file let us down in our scenario anyway, even though it was set up correctly and from day one. There is a measurement across a large number of trajectories: a plan helps on average. But the authors' verbatim quote is 'bad plans hurt performance more than no plan at all'. A file that has stopped reflecting reality is not neutral — it creates false confidence precisely where vigilance is needed. I'll show the full picture with the figures in a few minutes, on the failure slide — for now it's enough to understand the mechanism."

51 · The target answer is a line in CLAUDE.md obliging the agent to append an agreement to the task file at the moment it is reached; the task file is not the same thing as the decision log

"The task file is not the same thing as the decision log. DECISIONS.md is about what stays in force for good, for all future tasks. The progress of this particular task is not a project decision, it is a status: what is done, what comes next, what small things we happened to settle along the way. How to store the wizard's state is a decision for exactly the duration of this task: once we're finished it is just a detail of the code, not a project rule. When the task ends, all of this can be thrown away. A completely different type of memory, not the next rung of the same ladder.

The target answer is not 'set up a file' — it already exists — and not 'do not forget to keep appending'. The file was there, the practice was there. What was missing was a single line — not in the log, but in CLAUDE.md: append an agreement to the task file the moment it is made, not 'when there is time'.

The addressee of that line is not the developer but the agent that keeps the file anyway: it reads the instructions at the start of every session, it checks off the steps, it writes the log. It simply was never told when to write. The solution to the case is to say that in the instruction file, rather than relying on someone remembering it on their own.

And exactly the same caveat as in the case about the readiness gate: a rule in text is a request, not a law, and it won't work every time. The difference is that the request is addressed to an executor that opens these instructions every session — whereas human attentiveness does not scale at all, in any way."

52 · What is new in this case is not the file but three lines in CLAUDE.md: they tell the agent when to write into that file

"One file per task: the plan, the progress log, the results — it was already there from the first step, that is not the news of this case. The news is this block in CLAUDE.md, three lines. The first describes the file itself. The second says when to write into it: an agreement made in the course of the work goes into the log right away, in the same reply. The third says to reconcile the plan with the repository at the start and at the end of a session; we will come back to it in a minute, when we look at the numbers — that is where it came from.

On the right — what this produces in the task file itself: the agreement about sessionStorage made it into the log before the session broke off. The structure of the file is the same, the practice is the same — the difference is exactly whether it was written down in time or not. And it is the agent that writes it, following an instruction, not you from memory: forcing yourself to remember the log in the middle of the work is precisely the solution that does not work.

The honest boundary, the same as in the case about the readiness gate: these lines are a request in text. The agent reads them and can follow them; they give no guarantee that it appended every time. That is why the third line, about reconciling, stands right next to the first two: one request backs up the other, but neither of them becomes a barrier.

And an explicit caveat about scale: this is a simplified version of the pattern. In real practice it scales up to a folder per task — a separate plan, a separate log, a separate folder for the results, especially when the task is not linear. For today's session one file is enough — we are not complicating it."

53 · A bad plan is worse than no plan at all — and the 100% failure was measured precisely without any check that the plan was still current, so the countermeasure is direct: reconcile the plan at the start and at the end of the session

"The numbers I promised. Twenty-one thousand one hundred and twenty trajectories, SWE-bench: having a plan helps on average. And right there, a direct quote from the authors: 'bad plans hurt performance more than no plan at all'. An agent's compliance with its own plan drops by about thirteen points on the harder benchmark.

And here is an experiment with a stricter condition: thirty controlled live working scenarios in which the plan was not checked for currency as the work went on. A stale plan was followed in literally all thirty cases. A task file that nobody updates is not a neutral piece of paper. It is a source of systematically wrong actions.

And read the condition of that experiment once more: a hundred percent refers to runs with no check of whether the plan was current. What was measured is not the inevitability of failure but the price of skipping one specific step. So the step has to be put back — and that is exactly the third line we wrote down a couple of slides ago: reconcile the plan with the state of the repository at the start and at the end of a session, and put a 'reconciled' line with the date into the file.

Three things in that wording are not accidental. The first is 'and at the end'. Reconciling at the start protects this session, reconciling at the end protects the next one: it writes down what came to light towards the end, while the context is still intact. In our scenario the session broke off in the middle of the third step, and no reconciling at the end happened — but reconciling at the start of the second session would have compared the plan against the code and shown: the step has been started, and how to do it is not stated in the file. Not a guarantee, but a place where the divergence becomes visible.

The second is the 'reconciled, date' line itself. Without it the next session cannot tell a fresh plan from a three-week-old one: both look equally confident, and that is exactly the mechanism that makes a bad plan worse than no plan. The date makes freshness visible. The third is to reconcile against the repository, not against memory: the code and the change history survive context compaction, memory does not.

And the boundary of this measure is the same as for the readiness gate from the first section: it is once again a request in text. It lowers the risk but does not remove it. With Manus, which we talked about on the previous slide, the same trick works inside a single session — they rewrite the to-do list to the end of the context; reconciling does the same thing between sessions. We are not putting up a mechanical barrier today that would make it impossible to work from an unreconciled plan.

There is also a case study very close to our situation: a real project, three hundred and ninety-one sessions in a month. The authors observe the agent losing touch with the project's actual semantics as the rules grow. The solution is the same separation we have been drawing all through this section: the stable kept apart from the current. The result — minus seventy-five percent in the volume of AI instructions, zero recurrences of the problem over the next hundred and fifty sessions. Honestly, up front: this is one project, one author, action research, not a controlled experiment.

And the last thing that has to be said plainly: the 'a file or a folder per task' pattern is not mainstream right now. A study of almost three thousand real repositories shows that static context files dominate. Individual leaders push the practice — but it is not an accepted norm yet."

54 · One file per task is enough as long as the task is linear and one agent and one developer are running it — otherwise moving to a folder is justified

"If at least half of these points are false, moving to a folder per task is justified. That is already beyond today's session — it is covered in detail in Lecture 3.

Let us draw a line under the memory section. Three cases — three different answers to the question 'what happens when a session ends': a decision stays for good in a flat log; the decisions become too many and grow a structure around themselves; and the progress of one task lives exactly as long as the task does, in a separate file. The first and the third we set up right away, on day 0, because it is cheap. The second is a reactive step, because you cannot know in advance whether your particular decision log will grow to the volume that requires it.

But all three are text that the agent reads and can take into account. 'Can', precisely — not 'will'. Later in this session we will look at what happens when reading stops being enough."

55 · The six decision points of today's session split into two types: specific to the project (wait for a signal) and universal hygiene (set it up right away, on day 0)

"Let us look back over today. Six decision points, three in each of the two sections. In the section about the instruction file — what not to write in advance, how to write down the 'done' gate and the improvement loop, whether to split the file across subfolders. In the section about memory — a flat decisions file, structured wiki memory, the task memory of a single session.

In the right-hand column — two logics. Three decision points — 1.1, 1.3, 2.2 — are specific to the particular repository: there was no way to say in advance whether a general description of the structure, nested files or complex memory would be needed, so we honestly waited for a signal. The other three — 1.2, 2.1, 2.3 — are universal hygiene: the readiness gate, the decision log, task memory. Those we set up right away, without waiting for the pain, because the cost of doing it on day 0 is small while the cost of not doing it is unpredictable and arrives quietly.

And one more thing that has not gone anywhere: the gate line in CLAUDE.md and the decision log are both still text that the agent reads and can take into account — not a guarantee. That is a separate conclusion, and it holds regardless of whether we set them up right away or on a signal — and we will come back to it, for exactly a minute, in the next step."

56 · Two of the harness's five slots are closed today; the other three plus both mechanical layers (hook, process) — Seminar 5

"And here is the honest figure, which is worth naming directly rather than leaving it implied. The harness from Lecture 3 has five slots. Today we worked through two of them in detail: instruction-rules and memory, each across three real decision points. The other three — skills, subagents, access to the outside — did not come up at all today, not in a single line.

And there is something else we never touched today: both of the practical layers that mechanically reinforce the text layer — the hook and the process. This is not forgetfulness and not a last-minute shortage of time. It is a direct answer to the question that has just come up: what do you do when a request in text is not enough, even if it was set up in time, on day 0? The answer is in the next session, Seminar 5. There — the remaining three slots and both mechanical layers at once, because they are exactly what turns a request into a barrier the agent cannot get around, even if it wants to."

57 · Transfer today's six decisions to your own repository — and check which of them is really a day-0 baseline and which one you deferred without a reason, or set up too early without a signal

The format is deliberately light and collects no answers: a pause of 5-7 seconds, no calling for voices from the room, no breakdown of "typical answers" — unlike all six previous open questions of the session, this one has no breakdown, because it does not test knowledge but initiates a transfer onto each person's own context.

The closing line after the pause: "There will be no answer aloud. If at least one decision turned out to be 'set up too early, with no signal' — or, the other way around, 'long overdue and still not there' — that is where to start. This is not a question of resources. It is a question of the day."

---

Comments