Agentic Software Engineering: How AI Agents Change Delivery

The first version of this article, written in 2023, listed autocomplete tools and asked whether AI would kill programming. Three years later the question is different. Coding agents plan a task, edit files across a repository, run the tests, read the failures and open a pull request. Generating code is no longer the constraint. Verifying it is.

This is the anchor article for what we call agentic software engineering: the discipline of shipping trustworthy software when code generation is cheap and review capacity is not. It covers what changed, the verification tax that follows, and the four practices that decide whether an engineering organization gets faster or just busier: context engineering, verification pipelines, human checkpoints and team-wide standards. It ends with the metrics we use to prove the difference.

From autocomplete to agents: what changed

AI-assisted development has moved through three stages.

Completion (2021 to 2023). Tools suggested the next line or function inside the editor. The developer stayed in control of every keystroke. GitHub Copilot and Amazon CodeWhisperer (since folded into Amazon Q Developer) defined this stage. The unit of work was a line.

Chat (2023 to 2024). Developers pasted code into a chat window, asked for a fix or an explanation, and pasted the answer back. Useful, but the model saw only what it was shown. The unit of work was a snippet.

Agents (2024 onward). The model gets a task, a repository and a set of tools. It reads the code it needs, plans the change, edits multiple files, runs the build and the tests, iterates on failures and hands back a pull request. It runs in the terminal, in the IDE or in the background against a ticket. The unit of work is a task.

The tools that matter in 2026 fall into three groups:

• Frontier models. Claude (Anthropic), the OpenAI model family and Gemini (Google) sit behind most coding tools. Teams increasingly run more than one: Menlo Ventures found 37% of enterprises run five or more models in production, and buyers now expect an abstraction layer so a model can be swapped when pricing or performance changes.
• Agentic coding tools. Claude Code runs as an agent in the terminal, the IDE and CI. GitHub Copilot has grown from completion into a coding agent that takes an issue and opens a pull request. Cursor is an agentic IDE. OpenAI Codex and Google’s Gemini tooling occupy the same space. They differ in interface and in how much autonomy they take by default. They share the same shape: task in, verified change out.
• Orchestration frameworks. LangChain and LangGraph are the common choices when a team builds its own agents, for example a migration agent that runs across 300 repositories or a review agent tuned to house conventions. LangGraph adds state, retries, timeouts and budgets, which matter as soon as an agent runs unattended.

What did not change is where projects fail. MIT’s NANDA initiative found 95% of enterprise GenAI pilots show no measurable P&L return, and attributed the gap to learning and workflow problems, not model quality. The same holds inside engineering teams. The model is rarely the bottleneck. The workflow around it is.

The verification tax

Every line an agent writes has to be read by someone who is accountable for it. When generation was slow, review capacity and generation capacity were roughly matched. Now generation is close to free and review is not. The gap shows up as a tax on senior engineers: hours spent checking, correcting and sometimes rewriting AI-suggested code.

The tax has a specific cause. Agents produce code that is valid but wrong: it compiles, passes the tests that exist, and violates a convention nobody wrote down. The naming scheme, the layering rule, the reason the team never calls that service directly. A human reviewer catches it on the third read. The agent had no way to know.

Three symptoms tell you the tax is high:

• Pull request volume rises, merged volume does not, and review queues grow.
• Senior engineers report that AI-assisted work takes longer to review than it would have taken to write.
• Rework rate rises: changes that pass review come back within a sprint.

You cannot pay this tax down by reviewing harder. You reduce it by moving verification earlier and making it automatic, which is what the rest of this article is about.

Context engineering: making the codebase legible to agents

An agent inherits your architecture. If the repository is modular, typed and documented, the agent works safely inside it. If the knowledge lives in people’s heads, the agent guesses. Context engineering is the work of making the codebase legible to an agent before you scale generation.

We start every agentic engagement with a context audit. It covers three things.

Semantic dependencies. Which modules actually depend on which, as opposed to what the import graph says. Where the hidden coupling is: shared database tables, environment variables, message formats. An agent that does not see a dependency will break it.

Documentation gaps. Whether the architecture, the APIs and the conventions are written down and current. Stale READMEs are worse than none, because an agent trusts them.

Machine-readable conventions. Turning tribal rules into artifacts an agent reads on every task: a repository instructions file, linters and formatters that enforce style, CODEOWNERS that route changes, type coverage that makes contracts explicit, and a small set of reference implementations the agent is told to imitate.

The practical stack, in order of leverage:

1. A repository-level instructions file that states the architecture, the conventions and the things an agent must never do. Kept short. Reviewed like code.
2. Typed interfaces at module boundaries, so an agent gets a compile error instead of a runtime surprise.
3. Architecture and decision records shipped in the repository, not in a wiki the agent cannot read.
4. Task templates that give the agent the acceptance criteria, the files it may touch and the tests that must pass.
5. Scoped tool access: read the repository, run the tests, no production credentials.

Spec-driven development follows from this. The engineer writes the specification and the acceptance criteria; the agent implements; the tests decide. The spec is the contract, and writing it well becomes the senior skill.

Verification pipelines

A verification pipeline is the set of automated checks a change passes before a person looks at it. For agentic development it is the primary safety net, and it has to hold under a multiple of the pull request volume you have today.

We build it in layers, cheapest first:

1. Static checks. Formatting, linting, type checking, dependency and secret scanning. They run in seconds and reject most convention violations without a human.
2. Tests. Unit and integration suites, run on every agent iteration, not only on push. For agents the test suite is the main validation layer for generated code. If it is thin or flaky, harden it before you scale generation, because an agent will optimize for whatever the suite rewards.
3. Agent self-review. A second pass, with a fresh context, that reads the diff against the specification and the conventions file and lists what it would reject. It catches a large share of the valid-but-wrong class before a human sees it.
4. Model-graded checks. An LLM judge scores the change against a rubric: does it follow the layering rule, does it add tests, does it touch files outside scope. Calibrated against human decisions on past pull requests, and treated as a signal, not a verdict.
5. Security gates. Static application security testing and policy checks, with hard fails on new secrets, new network calls and permission changes.

Only after these does a human read the change. The reviewer’s job shifts from finding defects to judging design, which is the part that needed a senior engineer in the first place.

Evals for coding agents

Teams that build LLM products know to keep an evaluation set. Teams that adopt coding agents usually do not, and then cannot say whether a new model, a new instructions file or a new tool made things better.

An eval for agentic development is a set of real tasks from your own backlog, with a known-good outcome for each: the tests that must pass, the files that should change, the ones that must not. Fifty tasks is enough to start. Run every configuration against it and record:

• Task success rate: the share of tasks that reach green without human edits.
• First-pass yield: the share whose first pull request would have been merged as is.
• Tokens and wall-clock time per task.
• Scope violations: files touched outside the task boundary.

Re-run the eval on every model upgrade and every change to the instructions file. It turns “the new model feels better” into a number, and it tells you which workflows to promote to the whole team.

Human checkpoints

Autonomy is not a switch. It is a policy about where a person must be in the loop. Mature setups define checkpoint boundaries and let agents run autonomously between them. Immature setups leave every developer to decide alone, which means some babysit every step and others merge unread.

The boundaries we recommend as a default:

• Specification. A human writes or approves the task, the acceptance criteria and the files in scope.
• Pull request. A human reviews the design after the pipeline has passed. Not before.
• Deployment. Existing release gates apply. An agent does not get a shortcut to production.

Some changes always require a person, regardless of pipeline results: schema migrations, authentication and authorization, payments, data deletion, and changes to external API contracts. Write the list down and enforce it in the pipeline, so an agent stops and asks instead of proceeding.

Team-wide standards instead of hero setups

In most engineering organizations the best agent workflows live in one person’s IDE. The instructions file, the prompts, the tool configuration and the habits that make it work are not versioned, not shared and not known to the CTO. When that person changes teams, the capability leaves with them.

Standardization is the step from scattered wins to compounding ones:

• Instructions files, agent configurations, reusable commands and tool integrations live in a repository, are owned, reviewed and versioned.
• Workflows that pass the eval are promoted as the team standard; onboarding covers agent use from day one.
• Adoption is instrumented by team, task type and outcome, so leadership can name the pockets of excellence and scale them.
• Model access, budgets and data-handling rules are set centrally, so an engineer does not have to work out alone what may be sent to which model.

This is also where the platform team earns its keep. A shared agent environment with the pipeline, the evals and the tool access already in place is cheaper than 40 personal ones.

Measuring impact

License counts measure spend. Delivery metrics measure value. We re-baseline a team on the DORA metrics and add four that are specific to agentic work.

Metric	Definition	What it tells you
Cycle time	Ticket opened to change merged	Whether the whole flow got faster, not just the typing
First-pass yield	Share of pull requests merged without a rework round	Whether generated code is trustworthy on arrival
Rework rate	Share of merged changes revisited within 14 days	Whether speed is borrowed from next sprint
Review time per PR	Reviewer hours per merged change	The verification tax, directly
Change failure rate	Share of deployments causing an incident or rollback	Whether the pipeline holds under volume
Agent task success	Share of eval tasks reaching green unaided	Whether a configuration change helped
Cost per task	Tokens and compute per merged change	Whether autonomy is priced sensibly

Measure weekly, per team, and compare with the baseline taken before the rollout. Expect cycle time to fall and rework to rise first. The second number is what the verification pipeline is for.

How to start: a 90-day path

1. Weeks 1 to 2: audit and baseline. Context audit of the main repositories. DORA baseline plus the four agentic metrics. An honest inventory of how AI is actually used today, by depth rather than license count. The AI readiness assessment gives a structured starting point for the CTO conversation in 3 minutes.
2. Weeks 3 to 6: foundations and a first workflow. Instructions files, typed boundaries, test hardening, the verification pipeline. One workflow, for example bug fixes with a reproducing test, run by one team with defined checkpoints.
3. Weeks 7 to 12: standardize and scale. Eval set from the backlog, shared configuration repository, adoption instrumentation, a second and third workflow. Metrics reviewed against the baseline at week 12.

A team that starts with licenses and skips the first six weeks pays the verification tax in full and concludes that agents do not work. The order matters.

Where Erbis fits

Erbis has built software since 2012 and production AI since 2017, with 200 engineers and $110M+ in documented client savings. Our agentic AI engineering practice converts existing engineering organizations to agentic development: context engineering, verification pipelines and team-wide standards, at a predictable cost, on tooling you already license. You own the configurations, the evals and the pipeline from day one.

We apply the same discipline to the agents we ship for clients. Two AI agents that automate identity management cut account setup from 15 minutes to under 1 minute and, through script caching, cut LLM API spend by 90%. A custom-trained LLM with a verification controller generates contracts for a law firm 10x faster, saving 28,800 hours and €3.5M a year. In both cases the verification layer, not the model, is what made production possible.

FAQ

Which AI coding tools should a team standardize on?

Pick one agentic tool as the team default (Claude Code, GitHub Copilot’s coding agent or Cursor are the common choices), keep model access behind an abstraction so you can switch between Claude, OpenAI and Gemini, and put the configuration in a shared repository. The choice of tool matters less than the instructions file, the verification pipeline and the eval set, which transfer between tools.

Will AI agents replace software engineers?

No. They change what engineers spend time on. Specification, architecture, review of design and ownership of outcomes move up; typing moves down. Teams that treat agents as a headcount replacement without a verification pipeline see rework rise and conclude the tools failed.

How do we keep AI-generated code from lowering quality?

Make quality automatic: typed boundaries, a trusted test suite, static and security checks, agent self-review, and a human checkpoint at the pull request. Measure first-pass yield and rework rate weekly. Quality drops when verification is left to the reviewer alone.

How do we measure whether agentic development pays off?

Baseline cycle time, first-pass yield, rework rate, review time per pull request and change failure rate before rollout. Re-measure weekly per team. A real gain shows as shorter cycle time with flat or falling rework. A false gain shows as shorter typing time and a longer review queue.

Where should humans stay in the loop?

At specification, at pull request review and at deployment, plus any change to schemas, authentication, payments, data deletion or external contracts. Between those checkpoints, agents should run unattended, with the pipeline as the gate.

If your team has licenses but no baseline, or agents in a few IDEs and none in the pipeline, start with the AI readiness assessment. Or book a discovery call and we will tell you directly whether an agentic engineering engagement is the right move.