{
  "video": {
    "id": "rAUyRDOFyHU",
    "title": "Agents in Orbit: Workflows, Not Autopilot — Eduardo Simon & Javier Casal, Flywire",
    "duration": 1133,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": ""
  },
  "analysis": {
    "video_id": "rAUyRDOFyHU",
    "title": "Agents in Orbit: Workflows, Not Autopilot — Eduardo Simon & Javier Casal, Flywire",
    "one_liner": "Flywire's Orbit platform treats agents as YAML-defined workflows rather than autopilot: deterministic code does the predictable steps, and a model is called only for the steps that need reasoning, with tightly scoped permissions and strict JSON output. Across its workflows it ran over 11,000 executions in 30 days with a 99% success rate.",
    "summary": "Flywire handles payments under PCI DSS and SOC 2, so it built Orbit, an in-house 'toil killer', and runs it on Victoria, its internal PaaS on GitLab and AWS, so sensitive data never leaves its control. Each agent is a single YAML file reviewed through a merge request. Deterministic activities are chained with a few LLM steps, and each LLM step runs the Claude Agent SDK with restricted network, tools, skills and MCP servers and must return a strict JSON schema. The speakers walk through the code review agent, the infrastructure (Fargate, then AWS Lambda microVMs), the guardrails, an evaluation approach that tests individual LLM steps, and production numbers. Their advice is to start small, use the model only where it adds value, and keep guardrails and humans in the loop.",
    "key_points": [
      "Orbit pairs deterministic code with small LLM steps. Repetitive procedural logic becomes code, and a model is called only when reasoning is needed. Agents are single YAML files that go through the same merge request review as any other code.",
      "Code review agent flow: a GitLab webhook fires when an engineer mentions the bot. Deterministic steps fetch the MR metadata, classify its risk tier, load prior findings and clone the repo. Then a read-only LLM step answers in a strict output schema, and a final deterministic step posts the findings back to the MR.",
      "They built rather than bought because frontier models make custom feature development cheap. Keeping execution on Victoria keeps PCI, PII and PHI data inside their control and avoids long vendor security reviews.",
      "Workflows start from the internal API or CLI, from webhooks (GitLab, Jira, Sentry, AWS) or on a schedule (translation syncs, feature flag cleanups, knowledge base refreshes). An Orbit MCP server is being built so Claude in Slack can trigger workflows.",
      "Architecture: Orbit began as a single ECS Fargate container and then split orchestration, agent runtime and session state into separate components. It now also uses AWS Lambda microVMs, which can pause and resume, with a runtime chosen by project type. Full sandboxing (Docker-in-Docker or multiple microVMs) is still in progress. The baseline container deliberately leaves out package managers such as Bundler, Maven and Composer.",
      "Guardrails: configurable timeouts, rate limits that are aggressive by default (a broken integration could spawn thousands of agents), and control-plane locks that drop duplicate runs. Each step declares in YAML which network, tools, skills and MCP servers (down to individual tools) it may use. If output fails JSON schema validation, Orbit runs a targeted retry or fallback.",
      "Evaluation: a new agent reaches production after a check on a few real cases. Every run is traced end to end with OpenTelemetry. Real traces go into golden datasets versioned next to the code, and the harness tests individual LLM steps by injecting exact historical state. There are no online evals yet.",
      "Last 30 days: code review ran 9,700+ times at under 3 minutes per MR. The audit fixer opened 159 MRs for vulnerable dependencies, the AI readiness fixer improved about 80 repos, the Victoria linter fixer fixed 413 projects and the merge conflict resolver ran 921 times. In total there were 11,000+ executions with a 99% success rate. The team had made close to 1,000 commits since forming in April."
    ],
    "takeaways": [
      "Start with a skill: it is cheap to write and easy to iterate on. Once a task is proven worth automating, turn it into a workflow where the predictable parts are deterministic code and the model handles only the reasoning.",
      "Give every LLM step least-privilege access to network, tools, skills and MCP tools, declare that access in reviewable config, and enforce strict JSON schemas so control flow never depends on parsing free text.",
      "Evaluate individual LLM steps with replayed historical state and golden datasets built from production traces, instead of noisy, expensive end-to-end benchmarks.",
      "Control cost by choosing a model per step (planning and implementation can use different models), and add rate limits, timeouts and deduplication locks before scaling agents out.",
      "Keep the merge request as the unit of work, so humans still review and approve what agents produce in a regulated environment."
    ],
    "topics": [
      "agents",
      "workflow-automation",
      "guardrails",
      "evals",
      "platform-engineering",
      "compliance",
      "code-review",
      "sandboxing"
    ],
    "tools": [
      "Orbit (Flywire internal)",
      "Victoria (Flywire internal PaaS)",
      "Claude Code",
      "Claude Agent SDK",
      "MCP",
      "GitLab",
      "AWS",
      "AWS ECS Fargate",
      "AWS Lambda microVMs",
      "Jira",
      "Sentry",
      "Slack",
      "OpenTelemetry",
      "Bundler",
      "Maven",
      "Composer",
      "YAML"
    ],
    "quotes": [
      {
        "text": "Instead of letting AI agents run freely in open-ended loops, Orbit pairs deterministic code with a small LLM steps.",
        "at": "03:17",
        "url": "https://www.youtube.com/watch?v=rAUyRDOFyHU&t=197s"
      },
      {
        "text": "This is what we mean by workflows, not autopilot.",
        "at": "05:04",
        "url": "https://www.youtube.com/watch?v=rAUyRDOFyHU&t=304s"
      },
      {
        "text": "Start with the skill. Skill is cheap to write, easy to iterate on, and useful from day one.",
        "at": "12:35",
        "url": "https://www.youtube.com/watch?v=rAUyRDOFyHU&t=755s"
      },
      {
        "text": "We also learned that cost control starts with model selection.",
        "at": "16:35",
        "url": "https://www.youtube.com/watch?v=rAUyRDOFyHU&t=995s"
      }
    ],
    "words": 3189
  },
  "summary_url": "/#rAUyRDOFyHU",
  "transcript": {
    "html": "/transcripts/rAUyRDOFyHU.html",
    "txt": "/transcripts/rAUyRDOFyHU.txt",
    "vtt": "/transcripts/rAUyRDOFyHU.vtt"
  }
}