Your CLAUDE.md Works on the Rules You Did Not Need

Does Claude Code follow CLAUDE.md?

Partly, and the part it drops is the part you care about. Anthropic is direct about this in its own documentation. The memory page states that CLAUDE.md files are "loaded at the start of every conversation" and that "Claude treats them as context, not enforced configuration". Troubleshooting goes further: "Claude reads it and tries to follow it, but there's no guarantee of strict compliance, especially for vague or conflicting instructions."

What Anthropic does not publish, as of 13 September 2026, is a number. There is no measured compliance rate in the documentation, which is reasonable for a vendor and unhelpful for a team deciding whether the file is a control or a suggestion.

The absence has not stopped the question. A search of the anthropics/claude-code issue tracker for issues whose title contains both CLAUDE.md and a form of "ignore" returns 199 results, 9 still open on 13 September 2026. Not all are about rule compliance, but the recurring title is unmistakable: CLAUDE.md Mandatory Rules Consistently Ignored Across Multiple Repositories, open since June 2025 with 45 reactions. The plural is the tell. People are not reporting a bug, they are reporting a property.

What does the published research already say?

That the structural knobs everybody turns do nothing, and that compliance decays as a session gets longer. This is not our finding, and we want to be precise about that before presenting ours.

In May 2026, Damon McMillan published Instruction Adherence in Coding Agent Configuration Files: A Factorial Study of Four File-Structure Variables (arXiv 2605.10039, submitted 11 May 2026). It is by some distance the best-powered work on the question: 1,650 Claude Code CLI sessions, 16,050 function-level observations, two TypeScript codebases, five coding tasks, Sonnet 4.6 with Opus 4.6 as a cross-model check. Four variables were manipulated factorially: file size, instruction position, file architecture, and contradictions in adjacent files.

The result, in the paper's words: "None of the four structural variables or three two-way interactions produces a detectable contrast after multiple-testing correction." The size and conflict nulls carry affirmative-null Bayes factors (BF10 between 0.05 and 0.10), which is stronger than a failure to reject. The one thing that moved: "each additional function the agent generates is associated with approximately 5.6% lower odds of compliance per step (OR = 0.944)", described as non-monotonic.

That second point is the hinge. A trivial target annotation is a marker the agent has no reason to emit unless the file asks for it, so the file's job is to take compliance from roughly nothing to something, and it does that job. We asked the commercially awkward version: what happens when the agent's instinct already produces something plausible, and the rule says the plausible thing is wrong?

What did we measure that the big studies did not?

The case where the agent already has an answer. Our controlled study, published 24 August 2026, ran 30 developer tickets against a mid-sized Express and SQLite codebase in triplicate: three autonomous agents, same tickets, same model (Claude Opus 5 at effort high), 90 runs and 93 independent security scans. The manipulated variable was how the agent could know the platform's 49 rules: nothing, a rules section in the repository's CLAUDE.md, or injection at the moment of the edit.

The rules span eleven families, from refunds and loyalty arithmetic to GDPR minimisation, the Omnibus directive's reference-price obligation, cash caps and audit trails. Each carries a deliberately arbitrary specific: 422 REFUND_EXCEEDS_CAPTURED, a 150.00 euro manager threshold, erased-{id}@removed.invalid, ORD-{store}-{YYMMDD}-{seq}. The file arm's section was written the way real teams write them: 41 lines covering about half the corpus, paraphrased, two stale values, whole families absent. What it said about each rule was sealed before any run, as was the corpus. Tickets were written as incidents in developer voice and machine-checked for leaked literals, and three blind auditors graded every diff without knowing the arm.

13%

rule specifics exact with a realistic CLAUDE.md rules section

13%

rule specifics exact with no rules file at all

87%

rule specifics exact with the rule delivered at the edit

Seven of 55 in both control arms. Not close, identical. Over the nineteen tasks of phase 1, having a realistic rules file in the repository changed nothing measurable about the code produced. It did change the shape of the errors, which is the file's one demonstrated contribution: the bare arm violated more (18 against 12), the file arm more often produced the right behaviour with the wrong specific (28 against 20). A paraphrased document moves errors from "violated" to "nearly". It does not convert them into "exact".

What happens when the rule is in the file, with the right value?

It gets implemented exactly 31% of the time. Because the file's coverage of each rule was sealed before the runs, we can split phase 1 by what the file actually said. On 13 rules it carried the right value in black and white, in a document read at every session start.

Read the top row twice. Availability of the information was not the variable. The moment it arrived was.

The clearest case is task M06. The ticket asks the agent to accept a gift card smaller than the basket, with a bank card for the rest. The rule says a loyalty point is earned per full euro paid in money, and the gift-card share earns nothing. The file arm had that rule, exact. It rewrote the payment function, computed the eligible share to order the tender types, and left untouched the floor(total / 100) line that mints points on the total. In its own new flow, gift-card euros earn loyalty.

“

The work of a careful engineer who has never seen the rules document.

It had seen it, at session start, in a file it loaded before the first turn. The arm that received the same rule in its loop, as it edited the payment file, wrote earned = floor(remainderCents / 100) with two dedicated tests.

Were the rules just unguessable by design?

Some of them, yes, and that is the objection to answer rather than bury. Guessability was graded rule by rule and sealed before the first run: 35 unguessable, 13 weakly guessable, 1 guessable. The arbitrary specifics are deliberate, because they are the only way to separate knowing a rule from guessing a reasonable one.

Task M21 is the separation. All three arms capped cash payments at 1,000 euros, because that is a public legal fact the model knows and does not need to be told. Only the arm that received the rule answered with the platform's own error code, 422 CASH_LIMIT, boundary tested to the cent on the cash share of a mixed payment. The other two returned a 422 with a prose message.

To an API client those are not the same thing. One is handled, the other is guessed at by whoever writes the integration. What a company pays for is not what the model knows about the world, it is the arbitrary contract that makes its own systems talk to each other. A reader who finds our corpus too arbitrary has to answer M21's question: are your error codes and formats any less arbitrary?

Why do two differently configured agents make the same mistake?

Because agent errors are deterministic, not varied. This is the result we did not predict and the one with the largest operational consequence. On 16 of the 30 tasks, the two control arms, run independently in separate trees, converged on the same wrong implementation. Not similar. The same.

• The same absolute-write SQL at the yearly recount, quantity = excluded.quantity, where the rule requires a typed movement in a ledger.
• The same forbidden design for overlapping promotions, "overlap is a feature, the strongest wins", which is the engineer's instinct and exactly what the pricing rule forbids.
• The same 4,000-character cap on retained diagnostics, and the same forgotten iban field in the redaction list.

If you assume a bad agent output is a roll of the dice that review catches on the second or third occurrence, drop that assumption. A company that does not put its contract in front of the agent does not get a spread of interpretations for a reviewer to choose between. It gets the same reasonable, wrong interpretation everywhere, defended by the same tests.

That last clause matters. After 30 tickets the control arms left 57 and 52 non-conformant rules behind, of which 21 and 12 were outright violations: personal data in a carrier export and in the logs, an illegal struck-through price, loyalty points on gift cards, stock overwritten without a trace. At least a third were locked in by passing tests the agents wrote themselves. The debt is not just silent, it is defended by CI. More on that in business logic flaws in AI-generated code.

Does a perfect CLAUDE.md fix it?

It gets you two thirds of the way, and we tested that rather than assuming it. After task 22 the gap raised a fair objection: had the file arm ever had a real chance? So we pre-registered an amendment and replaced the drifted section with the complete 49-rule corpus verbatim, machine literals included, presented as freshly synced by engineering. That is the strongest version of "put the rules in the repo" that can exist.

Sixty-four percent is a real improvement and we will not pretend otherwise. It is also the ceiling, it costs the same in tokens, and the four failures are instructive. The file arm succeeded on the rules the ticket pointed at, and failed on two other kinds.

First, peripheral tokens inside a rule it otherwise got right: with 403 ADJUSTMENT_LIMIT beyond plus or minus 30 units spelled out verbatim in its file, it set the exact threshold of 30 and answered with a 403 in prose, without the literal. Dilution acts inside a single rule, not only across a document.

Second, cross-cutting rules the ticket does not name. The rule forbidding direct writes to stock quantities was in its file, verbatim, and lost three times out of three to the habit of the code in front of it. Existing code is an implicit instruction stronger than any document: it shows the agent how things are done here, it is under its eyes as it writes, and it compiles. A rule that contradicts a code habit only wins if it arrives at the same moment as the habit.

What do security vendors say about rules files?

Both things, often in the same product, and almost never with a measurement attached.

The optimistic case was made early and made well. In June 2025, Wiz's Rami McCarthy published Rules Files for Safer Vibe Coding and open-sourced baseline security rules for Cursor, Windsurf, Copilot, Codex AGENTS.md and Claude's CLAUDE.md, arguing that "rules files offer an ideal method to centralize and standardize these security-focused prompting improvements", with the post's own caveat that "the best security rules file is one custom to your organization". On the class it aims at, generic secure-coding posture, we have no evidence against it.

The sceptical case is usually about who can edit the file rather than whether the model follows it. Kodem's guide to securing AI code editors, published 31 October 2025 by Mahesh Babu, argues that protective features in these tools "are frequently disabled in practice", leaving "an environment where untrusted code or poisoned rules files can execute without scrutiny". A real problem, and a different one. A rule can fail because someone turned it off; it can also fail because the model diluted it, with the file perfectly intact.

Then the quietly revealing pattern: vendors shipping a file and a context-injection product at once. Sonar publishes an AGENTS.md for its agent workflow, telling you to "drop this file in your repo root as your agent's instructions file", and sells Vortex, which "injects the right project context and constraints so agents start with clarity". Endor Labs reads your CLAUDE.md as a statement of intent for its scanner, noting on 7 September 2026 that "a CLAUDE.md that says 'all API routes are authenticated through our SSO middleware' tells the scanner something no amount of pattern matching would surface".

None of them published a compliance rate, and neither, as far as we can find, has Anthropic. The position has rested on assertion and on GitHub threads. It now rests on one large factorial study of file structure and one small study of the hard case. Still thin. Less thin than it was.

What should you change, and what is not worth changing?

Start with what the evidence rules out, because that is where most teams spend their effort. McMillan tested, factorially and at scale, the four things every guide to writing a good CLAUDE.md recommends.

Two caveats. Anthropic's documentation does recommend targeting "under 200 lines per CLAUDE.md file", on context-budget grounds, and trimming a bloated file is good hygiene. But if you are betting a compliance outcome on trimming 300 lines to 150, the only direct experiment on record says you will not measure the difference. And "lost in the middle" is cited constantly to justify moving rules to the top. McMillan compared a rule at line 2 against the same rule at line 250. No effect.

What has support is the thing neither study set out to measure: distance from the write. His within-session decay and our served-versus-not-served split are the same phenomenon from two ends. Of the 58 graded rules our layer served at the moment of the edit, 55 were exact (95%). Of the 6 it failed to serve, 4 were not exact. Same agent, same session, same intelligence, different distance. So sort your rules by whether the agent could guess them.

Inventory the rulesCould the model guess it?Yes, leave it in CLAUDE.mdNo, move it to the editMust never happen? Hook

Leave in the file what the file is good at: build commands, project layout, conventions a competent engineer would pick anyway, and anything the agent would never produce on its own. That last category is what the well-powered study measured, and there the file works.

Move your arbitrary specifics to the moment of the edit. Error codes, file-name formats, thresholds, the shape of your audit line, the columns your export may contain. These are the rules a file handles worst, because the agent already has a plausible competing answer and the file is arguing with it from thirty thousand tokens away. They are also the rules with a regulator or a customer behind them.

Let rules become structure. When a rule was served at the edit, the agent built the mechanism it implies: a movements ledger, an idempotency key, an audit event. Later tasks reused it, sometimes without the rule being served again. When the audit-trail ticket arrived at task 14, the arm that had been emitting audit events all along finished the trail in 334 lines; the two that never wrote one built it from scratch in 1,107 and 1,620. Compliance laid down along the way cost five times less code than compliance caught up.

For anything that must never happen, stop using prose. Anthropic's answer is unambiguous: "To block an action regardless of what Claude decides, use a PreToolUse hook instead", and "CLAUDE.md instructions shape Claude's behavior but are not a hard enforcement layer." A rule you cannot afford to lose is a hook or a permission, not a bullet point. And review the file itself like code, because anyone who can commit to it writes your agent's policy: see instruction file injection.

How much should you trust these numbers?

Less than you should trust McMillan's, and we would rather say so than have you work it out. We ran one run per arm per task, on one codebase, with one model, across 25 rule-bearing tasks and roughly 65 graded rules per arm. He ran 1,650 sessions and 16,050 observations. The main gaps (13% against 87%, 64% against 100%) are too large to be counting accidents, but the secondary distributions should be read with that sample in mind. We publish no p-value, deliberately.

We also wrote the corpus, and we sell a tool that delivers rules at the edit. Three safeguards sit against that bias: corpus and rubrics sealed before any run, guessability graded in advance, tickets machine-checked for leaked literals. The blind auditors are models, not humans. A replication with human reviewers is the obvious next step and we have not done it. The study also reports where our own arm lost.

Four more, briefly. Two over-applications, right corrections made at the wrong moment. Four rules never retrieved, each from a family the ticket did not aim at, explaining four of our arm's seven deviations. Six tasks of degraded delivery from an authentication failure and a channel outage, counted against us rather than excluded. Thirteen false positives on the command guards.

His is the better experiment, ours is the harder case. He isolated structure cleanly with a target that had no competing instinct; we chose targets with nothing but competing instinct, and gave up statistical power to do it. Together they say something simple: the structure of the file is not what is failing, and the rules a file fails on are the ones you wrote it for.

What we sell, stated plainly

Agent-time injection is our product, this study measures it, and the study also says what it cannot do.

VibeDefend installs in the agent's environment rather than in your repository. A hook fires before each file edit and puts the rules relevant to that file and that intent into the model's context immediately before the write. Across the 30 tasks that came to 669 business-rule injections and 1,078 security-rule injections, for about a dollar a ticket.

What it does not do is enforce. Task M24 is in this article because it is the clearest boundary of the thing we sell. The deterministic part of the layer is the guard on the action, not the rule in the context, and we would rather name which of the two is a control than imply both are. The broader model is in AI coding agent security.

Frequently asked questions

Does Claude Code follow CLAUDE.md?

It reads the file at every session start and follows it inconsistently, with the inconsistency concentrated on specifics. Anthropic's documentation states that Claude "treats them as context, not enforced configuration" and that "there's no guarantee of strict compliance". In our study of 90 autonomous runs, a realistic rules section produced the same number of exactly-implemented rule specifics as no rules file at all: 7 of 55. On the 13 rules where the file carried the right value verbatim, the agent implemented it exactly 4 times.

What are the best practices for a CLAUDE.md file?

Sort your rules by whether the model could guess them, and stop tuning the file's structure. The only direct factorial test, across 1,650 Claude Code sessions, found no detectable effect of file size, instruction position, file architecture or contradictions with adjacent files, and an instruction at line 2 did no better than the same instruction at line 250. Keep in the file what a competent engineer would apply anyway, move your arbitrary specifics to the moment of the edit, and turn anything that must never happen into a hook.

How long should a CLAUDE.md be?

Short enough for your context budget, but length is not what decides whether a rule is followed. Anthropic recommends targeting under 200 lines per file, and trimming a bloated file is good hygiene. The factorial study found no detectable effect of file size, supported by an affirmative-null Bayes factor, so do not expect a shorter file to change whether a given rule survives to the write.

What should a CLAUDE.md contain?

Build commands, project layout, house conventions, and anything the agent would never produce on its own. That last class is what the well-powered study measured, and there the file does its job. What does not belong is your arbitrary specifics, error codes, file-name formats, thresholds: on the 13 rules where our file carried the exact value in black and white, the agent implemented it exactly 4 times.

Does Claude Code read AGENTS.md, or only CLAUDE.md?

Only CLAUDE.md, unless you wire AGENTS.md up yourself. Anthropic's documentation is explicit, "Claude Code reads CLAUDE.md, not AGENTS.md", with an @AGENTS.md import or a symlink as the supported bridge. Neither file is better than the other, because it is the same mechanism either way, a document loaded into context at session start, so the failure mode is identical too.

Why is Claude Code not following my rules?

Usually dilution rather than ignoring. The file is loaded once, and by the thirtieth edit it is an old fragment buried under tens of thousands of tokens of code, command output and reasoning. Damon McMillan's factorial study measured the same thing from the other side: each additional function the agent generates is associated with roughly 5.6% lower odds of compliance. The second cause is your own codebase, which shows the agent how things are done here and compiles.

Is a CLAUDE.md useful at all?

Yes for one class of rule, no for the class most teams write it for. McMillan's study measured a trivial target annotation, something the agent has no reason to emit unless the file asks for it, and there the file takes compliance from roughly nothing to something. Our study took the opposite case, rules for which the agent already has a plausible competing answer, and a realistic rules section scored exactly what no file at all scored: 7 rule specifics out of 55.

Should CLAUDE.md be committed to the repository?

Yes, and then reviewed like code, because anyone who can commit to it writes your agent's policy. Committing it is not the same as enforcing it, though. The rules section in our study sat in the repository and was loaded at every session start, and the code it produced was indistinguishable from the arm that had no rules file.

Did Andrej Karpathy prove a CLAUDE.md cuts errors from 40% to 11%?

We could not find a primary source for that figure. It circulates in social threads and secondhand write-ups, in more than one form, 40% or 41% down to 11%, with no published method and no sample size we could check. The measurements that do have a paper or a protocol behind them are less flattering: McMillan found no detectable effect of the four file-structure variables across 1,650 sessions, and our 90 runs put a realistic rules file level with no rules file at all.

How do I actually enforce a rule in Claude Code?

Not with prose. Anthropic's guidance is to use a PreToolUse hook "to block an action regardless of what Claude decides", and managed settings for permissions, since "settings rules are enforced by the client regardless of what Claude decides to do". A rule you cannot afford to lose belongs in a hook or a permission. A rule you want the agent to reason with belongs in its context at the moment it writes.