Development & Automation

How to Build a Software Factory for AI Coding Agents.

Eight engineering practices for teams using AI coding agents, from code organization and type safety to lint, end-to-end tests, fast CI, and cloud agents.

· Huuush

How to Build a Software Factory for AI Coding Agents

Key takeaways

  1. 1Keep feature code and tests together, and enforce repository contracts with types, lint, and end-to-end checks.
  2. 2Allow small duplication across features while defining clear contracts for shared primitives such as auth, permissions, money, and dates.
  3. 3Use bugbots to complement deterministic checks, and keep cloud-agent work reviewable through scoped context, pull requests, and fast CI.
An AI coding agent software factory moving code through type checks, lint, tests, and CI
Repositories and CI need to enforce quality at the pace AI agents produce code.

AI coding agents can increase the volume of code a team produces. Without a repository that makes its rules clear and checks the result, though, review work and quality variance can rise along with that output. Sarah Fim, who works with teams of 30 or more, argues from experience that repository design and CI enforcement matter more than changing the agent model or harness.

Her article lays out eight practices: keep related code together, express contracts with types, allow small amounts of duplication, encode rules in lint, test real user flows, use bugbots for complex issues deterministic checks miss, shorten CI feedback, and give cloud agents access to relevant work context. Read Sarah's original X Article and her introductory video post.

Loading X post.

Unable to load the X post. Please try again.

Source: X sarahfim

When changing a feature, its implementation, styles, tests, and supporting code should be easy to find within the same boundary. Next.js App Router lets you colocate components, styles, and tests inside route folders; only files such as page and route become public routes. This gives an agent a clearer path to the context it needs and reduces the chance of an unrelated shared file becoming the default place to edit.

Distinguish folders that create URL segments from route groups used only for organization. Next.js colocation docs

2. Type safety: enforce contracts at boundaries

Types make the expected inputs and outputs of functions and components visible in code. Editors and builds can catch a change that breaks those contracts early. Use React's TypeScript support; in TypeScript services, tRPC can infer client types from a server router. For public REST APIs, an OpenAPI schema can serve as a shared contract and generate types.

Types do not replace runtime validation. Validate request data that you do not control. React and TypeScript · tRPC · OpenAPI TypeScript

3. Duplication is OK: allow small repetitions

Moving every similar snippet into a shared abstraction can couple features whose needs will change independently. A little duplication across distinct parts of a product can be easier to read and maintain than a central module that collects unrelated behavior.

Shared global primitives are the exception to examine carefully. Authentication, authorization, money, and date handling can carry invariants across many features. Make ownership and contracts explicit before deciding whether those behaviors should be shared or duplicated. This distinction is Sarah's design guidance.

4. Lint rules: turn team conventions into executable checks

Written instructions alone do not guarantee that an agent will make the same choice every time. Encode recurring mistakes and repository conventions as lint rules, then run the same checks locally and in CI. Define both the prohibited pattern and any legitimate exceptions.

Sarah describes repository-specific rules such as banning generic folders like lib/utils and restricting useState and useEffect. These are not universal React rules; they preserve a team's chosen structure and keep an agent from changing it by default. Implementing policy as ESLint rules makes it repeatable. typescript-eslint custom rules

5. End-to-end tests: exercise real user flows

Unit tests check individual functions; end-to-end tests check whether a user can still move through the product. Running important user journeys alongside agent changes can catch a broken screen even when the code compiles.

Meticulous records browser interactions and replays them to compare UI behavior across changes. MCPJam focuses on evaluating MCP server tools and compatibility across hosts, with CI evaluation gates. They support different testing areas, so choose based on the product flow you need to cover. Meticulous docs · MCPJam CI/CD eval gates · Cross-client testing

6. Bugbots: use them as a second look at complex bugs

Start with deterministic checks for formatting, types, lint, and reproducible tests. A bugbot has a narrower supporting role: it can read a change and its context to flag complex issues that are hard to express as fixed rules.

Cursor Bugbot reviews pull request changes for potential bugs and security issues, and can use repository guidance in BUGBOT.md. AI review can miss issues or raise false alarms, so keep its findings reviewable and do not make it a substitute for CI gates or human responsibility. Cursor Bugbot docs

7. Fast CI: shorten the feedback loop

When CI is slow, developers and agents wait longer to learn whether a change failed, and each correction takes longer to verify. Measure job wait and run times and cache hits first; then address bottlenecks with parallelism, caching, or a suitable runner.

Sarah says her team cut CI time by 50% and cost by more than 70% with Blacksmith. Those are results she reports for her team. Blacksmith also says CPU-heavy jobs can run 40–60% faster than GitHub-hosted runners and that its per-minute runner cost is half, which can add up to roughly 70% lower total CI spend. These are Sarah's and the vendor's claims under particular workload and pricing conditions, not a result guaranteed for every repository. Blacksmith on reducing GitHub Actions spend

8. Cloud agents: connect the work context they need

Cloud agents can take on work that depends on context outside the repository when they can read issues, pull requests, and relevant business tools. Composio groups a user identity, available tools, and connected credentials into a session so an agent can discover and run the integrations it needs. Keep access scoped to the user and task, and route changes through pull requests and CI.

Sarah says an internal agent called Zen, using Composio dashboard context access, created more than 1,300 PRs since February and handled 245 in September. These figures are the team example Sarah shared. Composio sessions · Composio agent setup

Build a repository agents can follow

These eight practices are not a prescription for one prompt or agent harness. They describe an operating environment: keep implementation and tests close, enforce repository rules with types, lint, and E2E checks, and provide fast CI plus the work context an agent needs. Start with your team's current failures and bottlenecks, and keep the rules alongside the code agents change.

Read next: OpenAI Opens the Codex Harness So You Can Embed Agents in Internal Tools · Same Model, 39% Lower Agent Costs: What Unreal Labs Changed Was the Harness · Pi Coding Agent Adds MCP Support: Codemode and Jev Turn Tool Calls Into Code

Frequently asked questions

Where should a team start to improve AI coding agent output?
Keep feature code and tests close together, then run type checks and lint in CI so repository conventions are enforced consistently.
Should teams remove all duplication in AI-generated code?
Small duplication across distinct features can be fine. Shared primitives such as authentication, permissions, money, and dates need clear contracts and invariants.
Do bugbots replace tests or code review?
No. Use a bugbot as a supporting review for complex issues deterministic checks may miss, and have people assess its findings.
Will Blacksmith reduce every team's CI costs by 70%?
That is a conditional claim from Sarah's team example and Blacksmith's materials. Results depend on workload and runner pricing.
What context should a cloud coding agent receive?
Give it only task-relevant issue, pull request, or business-tool context within scoped permissions, and review its changes through pull requests and CI.

Related articles