How I take a software company from engineers using AI assistants to an organisation built around coding agents: the route, what I measure, what I set up first, and how sure I am of each part.
Version 0.1 · October 2026
Individuals get much faster. The company as a whole mostly does not, unless the way of working changes too.
Agents can work unsupervised only as far as tests, CI and review can prove their output right. Much of the job is building those checks.
Without a baseline, nobody can show later that anything improved. Most of the data already sits in Git, CI and the ticket system.
Adoption without a leader who uses agents personally tends to fade within a quarter.
Once people are fast, review queues, planning cycles and handoffs set the pace. That is when you change team shape, not before.
Each phase ends at a gate: a go, hold or stop decision with the sponsor, based on written criteria. Infrastructure and people work run side by side.
Every quarter: re-score against the Baseline. Durations are indicative for a 50–200 engineer organisation.
Is this company ready, and is the engagement set up to succeed?
Where are you now, in numbers you can compare against later?
Build what lets agents do more work safely.
Four levels of AI in engineering. Each level needs the one before it. Most companies sit at level I. Level II pays back on its own and is often skipped.
Bring the people along, group by group.
Change the organisation once people are fast.
Keep pace with monthly change, then hand over.
Mostly data the company already has. The one thing to start on day one is marking which commits and pull requests were written by AI; every later quality number depends on it.
| Measure | What it tells you | Where the data comes from | Watch out |
|---|---|---|---|
| Input: are people using it? | |||
| Usage per engineer | Who uses which tool, how often, and whether as editor help or as autonomous agents. | Admin consoles; Claude Code telemetry; Cursor and Copilot exports. | Personal data. Team level until privacy and works council are cleared. |
| Spend per engineer | The spread, not the average. A few heavy users and a long tail is normal. | Invoices, API billing. | Spend is an input, not a result. |
| Output: is the organisation faster? | |||
| Delivery flow (DORA) | Lead time for changes, deploy frequency, change failure rate, time to restore. | Git, CI/CD, incident log. | Fix the definitions before the first measurement, not after. |
| Time per stage | Where work waits: refinement, build, review, test, release. Shows where speed gets stuck. | Ticket system linked to Git (ticket ids in branch names). | Often needs a naming rule first. |
| Review load | Pull request size and waiting time for review. Agents move the bottleneck here. | Git platform. | Rising PR size is an early warning. |
| Quality: is the work any good? | |||
| Agent PR quality | Share merged after first review; share reverted within two weeks. Agent versus human. | Git, once AI-written commits are marked. | Start marking on day one. |
| Incidents and defects | Whether speed costs stability. | Incident log, bug tracker. | Linking incidents to code is often missing. |
| Readiness and risk | |||
| Agent readiness per repo | Can a fresh container build, test and run it using only the README? Test coverage, CI time. | A test run per repo. | Predicts where agents will succeed and fail. |
| Adoption groups | How many heavy users, light users, sceptics and refusers. | Usage data plus interviews. | Sceptics are sometimes right about the code. |
| Security posture | Where code and data go, retention, what agents may access. | Provider settings, pipeline, access lists. | "No training" is not the same as "not stored". |
In order. Most of these cost days, not weeks.
Is a leader using agents hands-on? Who owns the outcome? Agree 3–5 outcomes in writing.
Per-person usage data is personal data. Data protection impact assessment; in the Netherlands, works council consent for systems that monitor performance (WOR art. 27).
A commit trailer or PR label. Cheap, and every later quality number depends on it.
Read access to Git, CI, tickets and AI admin consoles. Pull three months of AI invoices.
The last 90 days are already in Git and the ticket system. No need to wait.
Fresh-container test on the five most important repos.
Leaders, the heaviest users, and the most sceptical senior engineers.
Data controls per provider (no training, retention settings). AI review on every pull request.
Name the internal AI Ops owner. Present the baseline to the sponsor: gate G1.
The full route has 45 steps. I rate each on how concrete it is today.
Standard tools or documents answer it. Baseline measurement, security controls, AI review, handover.
Clear idea, but thresholds and definitions must be set per company. Adoption groups, flow definitions, team shapes.
No proven method yet: productivity weighted by complexity, peer benchmarks, fully agent-run loops, new senior review roles, compliance for regulated software.
Built on Eshel & Fisher, The Agentic Awakening (Bessemer Venture Partners, June 2026), DX and DORA research, with my own additions on EU privacy, works councils and regulated software.
