I Run a Fleet of AI Agents Across Three Machines. Here's What Broke. - Kyle Jaejun Lee, KRAFTON

AI Engineer · 9 min · 165 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

  1. 00:00Hello, I'm Kyle.
  2. 00:01I run a fleet of AI coding agents across three machines every day, and this is the story of what broke, starting with me.
  3. 00:08This is my fleet, three machines I run every single day.
  4. 00:12The MacBook is where I do my heavy coding and my personal projects, and it sleeps because, well, it's a laptop.
  5. 00:18Then two Linux boxes, both headless and always on.
  6. 00:21Linux A takes the long-running coding tasks.
  7. 00:24Linux B runs my short-lived personal side projects.
  8. 00:28Two operating systems, one control plane tying it all together.
  9. 00:31And to be clear, this isn't a demo I spun up for the talk.
  10. 00:35This is what I actually use all day, every day.
  11. 00:39And here it is live, the whole thing running on one screen.
  12. 00:42Real agents, real work.
  13. 00:43This is a normal Tuesday for me.
  14. 00:45So, let's go back to the beginning because before any machine broke, I broke.
  15. 00:49When I started, it was just me and a terminal.
  16. 00:53A few tmux panes, a few agents, then a few more.
  17. 00:56And really fast, I'm sitting in front of four, five, six live contexts at the same time.
  18. 01:02And here's what nobody warns you about.
  19. 01:04At that point, I'm not running agents anymore.
  20. 01:07I've become the scheduler, deciding who does what.
  21. 01:09I'm the memory, holding what every one of them is doing, and I'm the reviewer, checking all of it.
  22. 01:14One human, three roles, six contexts.
  23. 01:17It does not scale.
  24. 01:19This This is the mess that broke me.
  25. 01:22I just couldn't hold what six agents were doing at once.
  26. 01:25My own attention was the bottleneck.
  27. 01:28So, I asked myself kind of a strange question.
  28. 01:30How does a handful of executives run a company of thousands of people?
  29. 01:34They don't hold all of it in their heads.
  30. 01:36They separate context.
  31. 01:37Each person only ever sees their own slice.
  32. 01:40And that was the unlock.
  33. 01:41What if my agents weren't a flat pile?
  34. 01:43What if they were an organization?
  35. 01:46So, that's what I built, a hierarchy, CEO, VP, manager, worker.
  36. 01:50And please hear me on this.
  37. 01:51These are real entity types in the system.
  38. 01:54It is not a cute metaphor.
  39. 01:55Each one is its own agent with its own scoped context and its own approval boundary.
  40. 02:00Context flows down.
  41. 02:02Each layer only gets the slice it needs.
  42. 02:04Results flow back up and I only review what reaches the very top.
  43. 02:08So, instead of holding six contexts in my head, I hold exactly one.
  44. 02:12Next problem, where does an agent's state actually live?
  45. 02:16Normally, it lives inside the model's context window and that window fills up.
  46. 02:20So, I moved it out.
  47. 02:21Every entity gets its own workspace on disk.
  48. 02:24Shared context that everyone has to honor lives in shared.
  49. 02:28Machine-bound state lives under machines.
  50. 02:30And inside each workspace, the mission, the current status, and a handoff folder, the actual work product that gets passed along.
  51. 02:38The state lives in files.
  52. 02:40It is not trapped inside one model.
  53. 02:42And this is the single most practical thing I learned all year.
  54. 02:46When that window fills up, the built-in move is to compact, summarize the history, make room.
  55. 02:52I stopped doing it.
  56. 02:53It's slow.
  57. 02:54I can't choose what survives.
  58. 02:55And whatever it throws away is just gone.
  59. 02:57So, instead, I don't compact, I reset.
  60. 03:00And by reset, I mean right inside Claude, I clear the context completely.
  61. 03:05And then it just reads back the handoff and the history files it wrote for itself and picks up exactly where it left off.
  62. 03:13The context can get wiped, the machine can even crash, and the work still survives because it was never only in the model.
  63. 03:20But there was still one problem, alignment.
  64. 03:22Plans flow down the hierarchy, but they drift.
  65. 03:26And I'd only catch it by walking into each pane by hand.
  66. 03:29So, I built a review gateway.
  67. 03:31Any layer that wants to act submits its plan and then it blocks.
  68. 03:34It waits.
  69. 03:35Nothing runs until I approve.
  70. 03:37And the second I approve, a hook fires the work off automatically.
  71. 03:40One web inbox, one control point, I never walk into the work windows anymore.
  72. 03:45Here's one full cycle.
  73. 03:47A plan comes in, I approve, the work resumes.
  74. 03:50That's the whole loop.
  75. 03:51And honestly, I didn't build this gateway by hand.
  76. 03:54An infra team inside the fleet built it.
  77. 03:57Agents building the tools that run the agents.
  78. 03:59I think that's kind of the whole point.
  79. 04:02So on one machine, all of this works beautifully.
  80. 04:05And then it didn't.
  81. 04:06Then one machine wasn't enough.
  82. 04:08Five things broke.
  83. 04:10Failure.
  84. 04:10One, my agents kept doing the work themselves instead of dispatching it down to a worker.
  85. 04:15The orchestrator is supposed to delegate.
  86. 04:18Instead, it just rolls up its sleeves and does the task itself.
  87. 04:21Wrong.
  88. 04:22So I forced its hand.
  89. 04:24A CLI harness with skills that call those CLIs.
  90. 04:27So dispatching becomes the only path it can take.
  91. 04:30Failure two, and this one's almost funny.
  92. 04:33As I kept throwing more tasks at my managers, they kept spinning up more and more worker panes inside a single window until it was so crowded the panes were too small to read.
  93. 04:44Even tmux capture pane, the thing I used to read a pane programmatically, couldn't pull anything meaningful out of them anymore.
  94. 04:51I'd literally run out of room to see what my own fleet was doing.
  95. 04:55Failure three, out of memory.
  96. 04:56Look at this.
  97. 04:57That's my activity monitor, completely buried under cloud code and MCP processes.
  98. 05:02Swap almost full.
  99. 05:03Basically no free memory left.
  100. 05:05The sessions just stacked up until the machine couldn't breathe.
  101. 05:09Failure four, get credentials.
  102. 05:11The expectation is clean.
  103. 05:13Credential A to workspace A, credential B to workspace B, one to one.
  104. 05:18The reality, they collided, crossed over, bound to the wrong workspaces.
  105. 05:23The fix was a clean, fully separated environment for each workspace.
  106. 05:27And failure five, the one that really hurt.
  107. 05:31The MacBook is a laptop, so the moment it loses power or drops off the network, every in progress job dies with it.
  108. 05:38And one time I'd pushed so many workers onto it that the whole machine just gave out.
  109. 05:43I came back later, opened the lid, and it had already restarted itself.
  110. 05:48Everything in flight just gone.
  111. 05:50So, the very first thing I did was build a boot command, one overlord boot, and the whole fleet comes straight back up because all the state was sitting in files.
  112. 05:59But a machine that can just vanish on you like that, that's when I knew one machine wasn't going to cut it.
  113. 06:04Every single one of these pointed at the same answer.
  114. 06:07So, I added machines.
  115. 06:09Here's the split.
  116. 06:10The MacBook was carrying everything, heavy coding and my personal projects.
  117. 06:14So, I offloaded it.
  118. 06:15Long-running coding tasks went to Linux A, short-lived personal projects went to Linux B, both always on.
  119. 06:21More resources, the long-running work finally off the laptop, and still one control plane over all of it.
  120. 06:27But now I've got context on one machine that I need on another.
  121. 06:31How do I move it?
  122. 06:32Git.
  123. 06:32I commit the context files and push.
  124. 06:34Then I poke the other machine with tmux send-keys over SSH and tell it to pull.
  125. 06:39It pulls, the agent there reads the files, and picks up exactly where the first one left off.
  126. 06:44Of course, it wasn't that clean.
  127. 06:45When two machines point at the same directory and both change it, you get conflicts.
  128. 06:50The same context quietly diverged, so I separated it.
  129. 06:53Per-machine directories for machine-specific state, and the shared stuff only changes through a pull request.
  130. 07:00It's boring, but boring is what stops the two machines from silently disagreeing.
  131. 07:04Then the gateway came back to bite me.
  132. 07:07I now had one on every machine, so I was checking multiple inboxes again.
  133. 07:12So, I collapsed them.
  134. 07:13Every machine sends its review requests over SSH into one main gateway, and that main one lives on an always-on Linux box because, remember, the Mac sleeps.
  135. 07:23Your one point of control can't be a thing that falls asleep.
  136. 07:27And then, my absolute favorite failure of the whole project.
  137. 07:30I was developing across all of these, bouncing back and forth, and at one point I went looking for a feature I'd built, and I genuinely could not remember which machine I'd built it on.
  138. 07:40And that's when it clicked.
  139. 07:42I need one place to root from.
  140. 07:44So, Discord became my single router.
  141. 07:46One bot per machine.
  142. 07:47Mac, Linux A, Linux B. My phone is now the remote control for the entire fleet.
  143. 07:53So, where am I now?
  144. 07:54Honestly, with a pile of things I have not solved.
  145. 07:56Four of them.
  146. 07:57Consistency across machines, abstracting away the local only tools that are still stuck on my Mac, the MCP servers, the browser, secure credential handoff between instances, and resource management.
  147. 08:08I really want to stop being the one who decides what runs where.
  148. 08:12And when I lined those four up, I realized these aren't new questions at all.
  149. 08:16An agent should just declare what it needs, not where it runs.
  150. 08:20Above it sits the orchestrator, the review gate, the logical hierarchy.
  151. 08:24Below it, compute, secrets, tools.
  152. 08:27A scheduler places it, and the machines just disappear underneath.
  153. 08:31These are the exact questions Kubernetes already answers.
  154. 08:35So, that's where I'm headed.
  155. 08:36I'm not going to reinvent compute, secrets, and tools.
  156. 08:39Kubernetes already nailed those.
  157. 08:41I'm stacking them underneath, and building my orchestration manager on top.
  158. 08:45Task orchestration, review flow, context management.
  159. 08:48Reuse what exists, build the new part on tops.
  160. 08:52And that's the honest state of it.
  161. 08:54One machine, I solved.
  162. 08:56Across machines, still rough, still building.
  163. 08:58If you're running agents at any kind of scale, I'd genuinely love to compare notes, because this is the part nobody has figured out yet.
  164. 09:05I'm on LinkedIn and on X. Links are in the description.
  165. 09:08Thank you so much.