{
  "video": {
    "id": "JJGbw4ggaFs",
    "title": "From Blind Spots to Merged PRs: Continuous Agentic Performance Optimization - May Walter, Hud",
    "duration": 1366,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": "Online"
  },
  "analysis": {
    "video_id": "JJGbw4ggaFs",
    "title": "From Blind Spots to Merged PRs: Continuous Agentic Performance Optimization - May Walter, Hud",
    "one_liner": "Hud's CTO shows how they turned the black-box 'investigate why prod is slow' phase into a weekly autonomous agent run — GitHub Actions + Claude Code + an MCP into function-level runtime data — that scores opportunities by ROI, verifies its own fix in runtime, and hands the human one convincing report instead of 80 PRs.",
    "summary": "The talk argues that performance work never gets prioritized because the fix is estimable but the investigation isn't — 'somewhere between an hour and a week' — so teams ignore issues until they become a crisis, then go straight back to ignoring them: a leaky bucket by definition. Hud automated that missing investigation phase as a weekly agentic workflow that reads real production traces, scores high-ROI/low-risk opportunities, applies the fix, reruns tests and confirms the impact before a human ever looks. The speaker is candid that it did not work out of the box: the failures were 'plausible unverified' fixes, unreliable ClickHouse query generation, and the 'lazy fix' (catch the exception instead of asking why it was thrown), all addressed with function-level production context ('prod-to-code'), a query language, and skills. The closing argument is that autonomous agentic engineering needs 80–90% trust, which is dramatically different from an agent in your IDE where 80% is fine because you're there to steer.",
    "key_points": [
      "The prioritization problem is asymmetric: teams can estimate a specific optimization but never the research phase, which 'could take an hour or weeks' — so nobody proactively looks, even though 'miraculously, every time we look, we actually find things that can be done.'",
      "Google's 2026 DORA metrics are used as evidence: AI's biggest impact is individual effectiveness, the second is software delivery instability, and throughput is 'not impacted as much as we expected' — more effective individually, same team throughput, more breakage.",
      "The stack: GitHub agent workflows as the vendor-neutral harness (neutral on compute, harness and model), GitHub Actions on a weekly schedule, Claude Code as the agent, runtime intelligence pulled in over MCP, report delivered to Slack — explicitly swappable for Cursor/Copilot/Teams/email.",
      "Three concrete failure modes found in practice: the 'plausible unverified' fix that sounds right but doesn't work when checked; complex ClickHouse queries (columnar, unlike classic SQL patterns) producing high variance in their evals; and the 'lazy fix' where the agent wraps the exception in a catch instead of finding why it was thrown.",
      "'Prod-to-code': service/endpoint/CPU/P90 metrics don't speak the agent's language because agent context lives at function and file level, so Hud captures function-level context linked back to the endpoint, event consumer or cron job that started the task, plus forensic evidence captured only for requests past P99 or a defined threshold.",
      "Layered architecture: HUD query language (ClickHouse over functions/endpoints/forensics), then skills that encode methodology (for an HTTP 500 find where the error came from; for a memory spike compare what ran on those pods at that time against a baseline), then automations — auto-fix, dead-code detection for code not run in 60 days, and the performance workflow.",
      "The agent hunts specific known-shape wins: artificial delays (timeouts and sleeps), N+1 queries, missing indexes, sequential async calls — 'much easier to find when you actually look for them', and plentiful in a 20-year-old codebase with hundreds of contributors.",
      "Opening PRs automatically failed on human grounds: 'no one wants to wake up to a rain of 80 pull requests.' They switched to one human-friendly report at a time — e.g. an endpoint usually at 200ms that occasionally takes 45 seconds because it uses distinct instead of Mongo's search index — scored on hot path, business impact (payments, sign-up) and risk (migrations), optimizing for ROI rather than raw impact.",
      "The loop is closed before the human sees it: detect, root-cause, argue business relevance (a flow running 7,000 times a week), apply the fix, rerun the tests, measure the impact — the human is a review gate, not an idea-triage queue."
    ],
    "takeaways": [
      "Automate the investigation phase, not just the fix — the value here isn't doing existing work faster, it's running a phase that 'just did not happen in the day-to-day lives of engineers': nobody stops weekly to look for low-hanging fruit.",
      "Don't ship an unverified suggestion to a human. Have the agent apply the fix, rerun the tests and show the measured effect on the specific flow, so the handoff is 'here's something that works', not 'here's an idea'.",
      "Score and gate before surfacing: hot-path frequency, business criticality, and change risk together. Ask yourself whether you could convince the product manager to prioritize it; surface one item at a time to build appetite rather than flooding the review queue.",
      "Give agents context at the altitude they reason at — function and file level, tied to the invoking endpoint/consumer/cron — rather than service-level CPU, memory and P90s. 'Context over cleverness works almost every time'; the models are already good enough, the steering is on you.",
      "Wrap raw query access in skills that encode how a senior engineer investigates (error provenance, spike-vs-baseline diffs). Letting the agent freestyle queries against a columnar store produced high eval variance.",
      "Hold autonomous automations to a much higher bar than IDE agents: 80% is fine when you're in the loop steering, but an unattended workflow needs 80–90% confidence or people stop trusting its output entirely."
    ],
    "topics": [
      "agents",
      "observability",
      "performance",
      "agentic-workflows",
      "production-context",
      "evals",
      "devex",
      "mcp"
    ],
    "tools": [
      "Hud",
      "Thundra",
      "GitHub agent workflows",
      "GitHub Actions",
      "Claude Code",
      "MCP",
      "Slack",
      "Microsoft Teams",
      "Cursor",
      "GitHub Copilot",
      "ClickHouse",
      "MongoDB",
      "Google DORA metrics (2026)",
      "HUD query language"
    ],
    "quotes": [
      {
        "text": "we would know how much time it takes to do a specific optimization, but we never know how long it's going to take to investigate it and to find it. And miraculously, every time we look, we actually find things that can be done around performance and stability, but we never stop to proactively look for them because it doesn't make sense.",
        "at": "00:56",
        "url": "https://www.youtube.com/watch?v=JJGbw4ggaFs&t=56s"
      },
      {
        "text": "we basically feel more effective, we're more effective individually, but as a team, our throughput is kind of the same and our software breaks more often, which is not exactly what we were hoping for with this AI revolution just yet.",
        "at": "02:49",
        "url": "https://www.youtube.com/watch?v=JJGbw4ggaFs&t=169s"
      },
      {
        "text": "that's not exactly how that worked because people are still people and no one wants to wake up for, you know, a rain of 80 pull requests, as small as small as they can be. That's just not how people operate.",
        "at": "15:50",
        "url": "https://www.youtube.com/watch?v=JJGbw4ggaFs&t=950s"
      },
      {
        "text": "agentic engineering is not like coding with an agent. If something works 80% of the time and you're using it with your cursor in your IDE, that's fine because you're there, you're in context, and you can help fix and steer.",
        "at": "21:02",
        "url": "https://www.youtube.com/watch?v=JJGbw4ggaFs&t=1262s"
      }
    ],
    "words": 4328
  },
  "summary_url": "/#JJGbw4ggaFs",
  "transcript": {
    "html": "/transcripts/JJGbw4ggaFs.html",
    "txt": "/transcripts/JJGbw4ggaFs.txt",
    "vtt": "/transcripts/JJGbw4ggaFs.vtt"
  }
}