What We Learned Running 6 AI Agents Autonomously for 4 Months

Four months ago, I started an experiment: what if an AI system could run a software team without human micromanagement? Not a chatbot that answers questions, but real agents coordinating with each other, making decisions, and escalating only when they genuinely needed a human.

That experiment is now our production reality. A PM agent, an Architect, an Engineer, a Tester, a Marketing lead, and a Community manager — all AI — are shipping features, reviewing code, and managing pipelines with minimal founder involvement.

Here’s what we learned.

What We Built

The system is structured around six specialized agents:

• PM Agent — triages incoming work, manages pipeline state, coordinates handoffs, escalates blockers
• Architect Agent — reviews designs, provides architectural guidance, gates readiness
• Engineer Agent — implements features, writes code, creates PRs, handles CI failures
• Tester Agent — QA testing, gates launches, reports results back to pipeline
• Marketing Agent — content strategy, launches, communications, growth
• Community Agent — support channels, user feedback, public-facing responses

Each agent runs in its own session and communicates through a simple file-based protocol: trigger files (incoming work), response files (reporting results), and shared context (activity logs, pipeline state).

The founder (me) sits at the approval gate. Agents propose decisions; I approve or veto. In the ideal case, my only job is reviewing pull requests and saying “yes” or “no.”

What “Autonomous” Actually Meant (And Didn’t)

Here’s the first shock: autonomous doesn’t mean independent. It means structured delegation.

In the early weeks, I thought autonomy meant the agents would solve problems I haven’t thought about yet. Instead, what it actually meant was:

Event-driven, not polled. When an issue is created, the PM agent doesn’t wait for me to tell it to look. It receives a trigger immediately. It triages, assigns labels, and decides routing. The Engineer picks up the same trigger (via label), pulls the issue, and starts coding.

Label-driven workflow, not conversation-driven. An issue’s status is encoded in GitHub labels (status:ready-for-dev, status:in-progress, status:needs-testing), not in chat. Agents read labels, update them, and move work through the pipeline by changing them. The founder reads the same labels the agents do.

Escalation-only founder involvement. Most decisions don’t surface to me at all. Agents handle triage, routing, standard bug fixes, and QA passes entirely. I only see a decision when something is genuinely ambiguous (new product direction, pricing strategy, design disagreement) or when a blocker can’t be resolved by the agent team.

This is radically different from “AI assistant.” It’s closer to hiring a team of junior specialists who are smart enough to handle 90% of work, and smart enough to know when to ask the boss.

The Numbers

Over the last 4 months:

Metric	Total	Notes
Issues created	2,063	Total filed; 2,057 closed, ~6 still open
PRs opened	2,021	1,962 merged (~97% merge rate)
Agents active	6	All agents consistently in use
Multi-founder usage	2	Both alprimak and obprimak committing

The PR merge rate of ~97% shows high-quality issue triage and execution — almost every PR that’s created actually makes it to production.

The pipeline has consistently handled scaling across specialized agents without degrading. What started as an experiment in async delegation has become the operational standard.

Failure Modes (And How We Fixed Them)

Running autonomous systems in production means hitting failure modes that just don’t show up in demos.

1. The “Silent Deadlock” Failure

What happened: The Architect would propose a design, the Engineer would start coding it, but the Architect would change their mind mid-implementation. The Engineer wouldn’t notice the new comment and would keep coding the old design. By the time the PR was ready, the code didn’t match the current guidance.

The fix: We formalized a gate. The Architect’s approval comment updates a GitHub label (architecture:approved). The Engineer reads that label before starting, and the CI pipeline blocks merges if the label isn’t present. Decisions are signaled through state changes, not through prose comments.

2. The “Label Storm” Problem

What happened: Three agents would update labels on the same issue simultaneously (PM routing it, Architect approving it, Engineer starting work). GitHub’s API would race, labels would get dropped or duplicated, and the issue would end up in an inconsistent state.

The fix: We added a locking mechanism to the MCP layer. Only one agent can mutate labels on an issue at a time. Agents queue if they collide. It’s simple and slow, but it works — consistency won over throughput.

3. The “Context Evaporation” Problem

What happened: The Engineer would start a 2-hour task, run out of context (token limit), and exit mid-work. On the next session, there was no recovery mechanism. The work would be lost or duplicated.

The fix: We built a checkpoint protocol. When an agent hits a context limit, it writes a structured recovery record (current branch, phase, next step, files touched). On the next session, that recovery record is loaded and the agent resumes from where it left off. This reduced lost work from ~30% of long tasks to near zero.

4. The “Wrong Agent” Routing Problem

What happened: The PM agent would route a marketing issue to the Engineer. The Engineer would receive it, start reading, and realize it’s not a code task. Time wasted. Frustration mounting.

The fix: We tightened the routing rules. Each agent declares what labels it accepts. The PM checks those declarations before routing. An issue labeled content won’t be routed to the Engineer, period. This seems obvious now, but it took a few cycles to systematize.

5. The “Approval Limbo” Failure

What happened: An Engineer’s PR would pass tests and be ready for founder review, but I wouldn’t notice for 12+ hours. The PR would sit open, blocking the next task. Other agents would start new work, get confused about sequencing, and create merge conflicts.

The fix: The PM agent now escalates PRs that are review-ready and unreviewed for >2 hours. It pings me directly. I get to them faster. This cut average PR dwell time from 14 hours to 4 hours.

What Actually Worked

Beyond the numbers, a few patterns emerged as genuinely useful:

Clear scope boundaries. Agents don’t reason about trade-offs outside their domain. The Marketing agent doesn’t argue about architecture. The Architect doesn’t worry about launch timing. Each agent has one clear job and executes it well. No turf wars.

Async-first communication. All coordination happens through files and GitHub, never through synchronous hand-off. An agent finishes work, writes a response file, and exits. The next agent picks up the response asynchronously. This scales better than agent-to-agent chat.

Explicit escalation rules. Rather than agents trying to solve ambiguous problems, we hardcoded: “If you can’t decide, escalate to the PM agent. If the PM can’t decide, escalate to the founder.” Most decisions stop at the PM level. Only the genuinely ambiguous stuff reaches me.

Cross-session continuity. Agents remember prior sessions. When the Engineer resumes a multi-session task, it knows exactly what state things are in because the prior session left recovery notes. This removed a huge category of bugs (duplicated work, lost progress).

The Real Lesson

If you asked me four months ago what autonomous agents would be good at, I would have guessed: “writing code, solving novel problems, adapting to unforeseen scenarios.”

Four months in, I’d say the actual killer app is: removing the human from the execution loop while keeping the human at the decision gate.

The agents are good at applying known rules to new instances. Are all tests passing? Approve the PR. Does the issue have all required fields? Route it to the team. Is this the same bug we fixed last month? Apply the same fix.

I’m good at deciding which rules to apply. Should we prioritize performance or features? Should we take technical debt or refactor? Should we launch this now or wait?

The system works because neither side is trying to do the other’s job.

What’s Changed for Me as a Founder

My job used to be: identify a problem, solve it, implement it, review the implementation, ship it.

Now it’s: identify a problem, write one paragraph of requirements, review the solution the agents built, approve shipping.

The throughput is higher because I’m not bottlenecked on implementation time. But I’m also more responsible for the problems we solve, because if the requirements are bad, no amount of agent intelligence fixes the outcome.

It’s less “how do I write more code” and more “am I pointing the team at the right problems?”

Getting Started with Multi-Agent Orchestration

If you’re thinking about building or running autonomous agents, here’s what actually matters:

1. Start with clear roles. Don’t build a general-purpose agent. Build specialized agents that know their domain deeply.

2. Use structured handoffs. Conversation-based coordination is fragile. Use state machines, labels, and explicit gates. Make decisions visible.

3. Build recovery into the system. Long-running agents will fail. Plan for checkpoints and resumption, not for perfect uptime.

4. Keep the human in the loop at decision gates, not in execution. The bottleneck should be the quality of decisions, not the speed of implementation.

5. Instrument everything. You can’t debug a system you can’t see. Logging, activity feeds, and state snapshots are non-negotiable.

Try It Yourself

We built Operum to solve the multi-agent coordination problem at scale. It’s the orchestration platform we use to run these six agents, complete with the file-based IPC, the checkpoint protocol, the routing gates, and the team-log logging.

If you’re interested in running your own autonomous agent team — for internal productivity, for a side project, or just to explore what’s possible — visit operum.ai to learn more.

The future of work isn’t “AI replaces humans.” It’s “humans and AI split the job differently, and it works better for both.”

We’ve been living that future for four months. The numbers show it.

---

Operum Marketing · operum.ai