This fifth article in the AI Contradictions series moves from measuring AI use to controlling AI authority. It argues that greater coding-agent autonomy should depend on whether the engineering constraints that matter are discoverable or enforceable and not on how much context the agent can consume.
| Audience: | CTO đźž„ Enterprise Architect đźž„ Director of IT Strategy |
| Decision Horizon: | Next 90–180 days; before materially expanding AI coding autonomy |
| Sector Scope: | Cross-industry |
Executive Summary
AI coding agents need repository context, but the emerging evidence does not support a simple rule that more (or less) persistent context produces better results. Studies now report negative, null, and positive effects depending on the task, implementation, metric, and way guidance is produced.1,2,3,4
Decision posture: Keep AI access broad where appropriate, but expand cross-file, multi-step, or unattended agent authority only where material engineering constraints can be reliably discovered when relevant or independently enforced against the resulting change. Satisfying that condition removes one reason to restrict the agent; it does not prove productivity, quality, maintainability, or ROI.
Our Analysis
The strongest AI coding teams may look less disciplined because less code is typed by hand. Underneath, the opposite is happening. Useful autonomy increasingly depends on engineering intent being explicit enough to retrieve, constrain, or test, without assuming that any particular context-file convention will reliably produce better results.
The Narrative vs The Reality
The market is moving from coding assistance toward agents that explore repositories, plan changes, edit multiple files, and test their own work. That encourages a tempting response: put more architecture notes, conventions, and instructions into the agent's context and let improved models handle more implementation.5
The evidence complicates that approach:
- Repository instructions are not a readiness certificate. Gloaguen et al. found no improvement in task success from repository-level context files across the agents and models tested, while inference cost increased by more than 20%. The agents followed the supplied instructions, but unnecessary requirements encouraged additional exploration and could make tasks harder.1
- But persistent guidance is not uniformly harmful either. In its evaluated with/without-guidance conditions, Lulla et al. reported 28.64% lower median runtime and 16.58% lower output-token use across 124 pull requests from ten repositories, with comparable task-completion behavior.2 Shepard and Albrecht separately reported higher SWE-bench Verified resolution after iteratively refining repository guidance than with either no guidance or their initial static knowledge base, with gains concentrated in task coverage and file localization.3 These studies do not establish a universal benefit; they show why the literature should be treated as contradictory rather than simply negative.
- Observed results around instruction files are highly heterogeneous. Across 15,549 agentic pull requests in 148 projects, Arabat and Sayagh found that 27.7% of projects increased merge rate by at least 20% after instruction files appeared, while 26.35% decreased it by at least 20%. Because the study compares projects before and after the files appeared rather than using a controlled intervention, the result does not establish that the instructions caused either change; it does show that their presence is not a consistently positive readiness signal.4
- How context is obtained matters. A 2026 FSE study found no single context-engineering method dominated. Static analysis offered the strongest effectiveness-efficiency tradeoff, while navigation-based approaches reached a higher performance ceiling at roughly 10–20 times the computational cost.6
- Some agent failures are not context failures at all. In a narrow controlled ablation across two coding agents, 17 tasks from three repositories, and 288 evaluated runs, Khatri found no measurable correctness effect from persistent-context strategy within the study's bounds. Failure analysis instead identified problems such as feature design, pattern selection, and implementation wiring.7 This is useful as a boundary condition, not as evidence for a universal governance rule.
- Good repository familiarity still does not make AI productivity automatic. METR's randomized study found experienced developers working in repositories they knew well took 19% longer with early-2025 AI tools, despite believing AI made them faster.8
Practitioner reporting supplies useful texture without resolving the empirical uncertainty. Developers interviewed by 404 Media described both project-specific rules intended to improve AI output and complex multi-system work where supplying enough context remained difficult.9
The Signal in the Noise
The evidence is converging on a narrower discipline: agents need access to the constraints that matter, and those constraints must be supplied or enforced in a way appropriate to the work, not merely placed in a repository file.
What Changes the Decision
Treat constraint availability as a minimum admission condition for broader agent authority instead of seeing repository documentation as evidence that broader authority has been earned.
If a material constraint cannot be reliably surfaced to the agent when relevant or independently enforced through tests, schemas, static analysis, policy, interfaces, or other control, restrict autonomous change at that boundary. An enforcement mechanism counts only when it can demonstrably detect or prevent the material violation being relied upon. The mere existence of a test, rule, or policy does not satisfy the gate.
The cited studies do not directly test independent enforcement as a substitute for agent-visible guidance. This is Tactive's operating inference from their inconsistent results. Material constraints should not depend solely on an agent receiving, interpreting, and following persistent instructions when the organization can make the relevant violation independently observable or preventable.
Why This Matters Now
As agents take on broader repository-level work, context acquisition becomes part of the production path rather than a prompting convenience. A July 2026 benchmark found logged agent trajectories missed every relevant “gold” file in 27–35% of samples, while retrieval-derived initial context improved file identification over irrelevant context.10
These findings remain early. Several relevant studies are recent preprints, workshop papers, or first-generation benchmarks rather than settled engineering evidence. The literature has already produced positive, negative, and null findings within months, and that balance could shift again before the next budget or renewal cycle. CIO policy should therefore be robust to which context technique currently leads the research and not depend on today's study count.
This is a cross-industry software-engineering constraint. What changes the decision is repository coupling, tacit knowledge, validation coverage, dependency complexity, and consequence of change, not the organization's industry label.
What to Watch for Next
Expect coding platforms to invest heavily in repository memory, retrieval, and persistent instructions. The useful signal will not be how much context they can store, but whether important constraints can be surfaced or enforced without systematically steering the agent toward behavior that is wrong for the task.
Recommended Actions
Do This
- Make constraint availability the autonomy gate. Before granting cross-component, multi-step, or unattended edit authority, require the service owner to identify the material constraints for that class of change. Each must either have a reliable discovery path for the agent or an independent enforcement mechanism. Enterprise Architecture defines what counts as material. An enforcement mechanism qualifies only when the owner can demonstrate that it catches or prevents the failure mode that makes the constraint material. If one unacceptable failure path remains uncovered, that constraint still blocks expanded autonomy for the change class.
- Divert enablement spend when critical constraints still live only in people's heads. At the next agent-license expansion or platform renewal, identify repositories where material architectural, dependency, validation, or operating constraints are neither reliably discoverable nor adequately enforced. Defer additional autonomy spend for those repositories and fund the missing tests, dependency mapping, architecture controls, or targeted retrieval first. This removes a known blocker; but it does not create a presumption of AI ROI.
- Make scope stability, not brevity, the test for standing instructions. Keep a rule in persistent agent context only when violating it would matter across the tasks in its stated scope and the prescribed behavior remains correct across those tasks. The instruction must also have a named owner and update trigger. Task-dependent guidance should stay at its authoritative source and be retrieved or supplied when relevant rather than made globally persistent. A short instruction can distort behavior just as readily as a long one if it is wrong for the task.
Avoid This
- Building an AI-readiness dossier. Current studies report positive, negative, and null effects from persistent repository guidance depending on the experimental setting.1,2,3,7 The presence (or absence) of
AGENTS.md,CLAUDE.md, or another context pack therefore provides no defensible readiness verdict on its own. - Letting a context artifact award autonomy. The presence of instructions proves only that instructions exist. Use missing, unstable, or inadequately enforced material constraints to deny broader authority. Use normal engineering evidence to decide whether broader authority should actually be granted.
Bottom Line
AI coding does not become ready for more autonomy simply because the agent knows more about the repository. Broader authority should stop where material engineering constraints are neither reliably discoverable nor demonstrably enforced. The control is not how much context the agent receives. It is whether the rules it cannot afford to miss can still get past it.
Evidence and Sources
- Gloaguen, Thibaud, Niels MĂĽndler, Mark Niklas MĂĽller, Veselin Raychev, and Martin Vechev. 2026. Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents? ICLR 2026 Workshop on Memory for LLM-Based Agentic Systems. The study found no general task-success improvement and more than 20% higher inference cost from context files across the evaluated settings.
- Lulla, Jai Lal, et al. 2026. The Impact of AGENTS.md on AI Coding Agents. Preprint, 2026. Across ten repositories and 124 pull requests, the study reports 28.64% lower median runtime and 16.58% lower output-token use in its evaluated guidance condition, with comparable task-completion behavior. The result should be read within the study's specific repositories, agents, tasks, and experimental setup rather than as a general estimate of the benefit of persistent guidance.
- Shepard, Noah, and Joshua Albrecht. 2026. Probe-and-Refine: Automatically Improving Repository-Level Guidance for Coding Agents. Preprint, June 2026. The study reports a 33.0% mean SWE-bench Verified resolve rate using iteratively refined guidance, compared with 25.5% unguided and 28.3% using the initial static knowledge base; the observed gain was associated particularly with task coverage and file localization. The result is specific to the evaluated benchmark, model configuration, trials, and refinement procedure and should be treated as evidence that tuned guidance can help under some conditions, not as a general estimate of benefit.
- Arabat, Ali, and Mohammed Sayagh. 2026. Toward Instructions-as-Code: Understanding the Impact of Instruction Files on Agentic Pull Requests. 23rd International Conference on Mining Software Repositories. The study compares project outcomes before and after instruction-file introduction; it supports heterogeneous association rather than clean causal attribution.
- Heaven, Will Douglas. 2026. Anthropic's Code with Claude Showed Off Coding's Future—Whether You Like It or Not. MIT Technology Review, May 21, 2026. Reports the shift toward agents performing larger coding workflows while adding mechanisms for retaining task knowledge and context.
- Li, Yichen, Qiye Lin, Yun Peng, Zhihan Jiang, Jinyang Liu, Chaozheng Wang, Yintong Huo, and Cuiyun Gao. 2026. One Size Does Not Fit All: Revisiting Code Context Engineering for Repository-Level Code Generation. ACM International Conference on the Foundations of Software Engineering. The paper compares seven context-engineering methods across three paradigms and eight LLMs, finding substantial effectiveness-efficiency tradeoffs rather than a dominant universal method.
- Khatri, Prakhar. 2026. Do Context Files Help Coding Agents? A Two-Agent Ablation Study on Real Repositories. Preprint, July 28, 2026. The study covers two agents, 17 tasks from three repositories, and 288 evaluated runs; its results should not be generalized beyond those experimental conditions.
- Becker, Joel, Nate Rush, Elizabeth Barnes, and David Rein. 2025. Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. METR, July 2025. The authors characterize the 19% slowdown as a measurement of the tools and settings studied, not a permanent estimate of AI coding productivity.
- Maiberg, Emanuel. 2026. Software Developers Say AI Is Rotting Their Brains.404 Media, May 13, 2026. Practitioner accounts include an engineering manager writing project-based rules for Cursor and a developer identifying provision of sufficient context as difficult in complex distributed applications.
- Qin, Bowen, and Yi Xie. 2026. Agent Retrieval Bench: Evaluating Repository Context Retrieval for Coding Agents. Preprint, July 2026. The benchmark covers 427 samples across 25 repositories; no retrieval family dominated, and logged agent trajectories missed all relevant gold files in 27–35% of samples.