{
  "video": {
    "id": "EL123UNokkI",
    "title": "Building uReview, Uber’s Multi-Agent Code Review Engine — Will Bond & Ameya Ketkar, Uber",
    "duration": 907,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer"
  },
  "analysis": {
    "video_id": "EL123UNokkI",
    "title": "Building uReview, Uber’s Multi-Agent Code Review Engine — Will Bond & Ameya Ketkar, Uber",
    "one_liner": "Uber built uReview, an in-house multi-agent code review engine, after first-time-to-review ballooned from 3 hours in 2024 to 9 hours in 2026 — it now posts ~25,000 comments a week with a 67% addressal rate, at 60% lower cost and ~70% higher accuracy than a naive implementation.",
    "summary": "Will Bond and Ameya Ketkar describe why Uber built its own automated code review system rather than buying one: it still runs Phabricator (unsupported by most vendors), needs the same review rules applied in the agent inner loop as in human PRs, and must distribute customization across hundreds of teams via the existing ownership model. The talk walks through uReview's architecture — multiple review generators tuned for different cost/performance points, plus post-processing that rates, categorizes, filters and deduplicates comments — and argues that the real unlock was observability: sentiment classification of developer replies, addressal rate, and agent trajectories, which let them tune quality-to-cost. They close on the inner-vs-outer-loop question, arguing that rather than killing the outer loop, automated review expands it: humans move up a layer to architecture, domain expertise and product thinking.",
    "key_points": [
      "Uber has thousands of engineers, hundreds of teams, 12 sites, and six language-specific monorepos; first time to review went from 3 hours in 2024 to 9 hours in 2026, making code review the bottleneck.",
      "They built in-house rather than buying because most vendors don't support Phabricator (they're mid-migration to GitHub), and because they want the identical review experience in the agent inner loop as in the human outer loop.",
      "Architecture: three review surfaces (GitHub, Phabricator, the agent loop) feed a uReview service that routes to multiple generators tuned for different cost/performance profiles — including plug-ins for third-party review systems so they can benchmark themselves — then post-processes to rate, categorize, filter and deduplicate so engineers see only high-confidence, actionable comments.",
      "Observability evolved from surface-level cost + NPS/Google Forms/Slack support (quality-to-cost 'all over the place') to classifying developer reply sentiment into positive/negative categories, tracking address rate, and profiling agent trajectories (tool calls and thinking) to tune runtime.",
      "Ketkar's biggest learning: 'the model doesn't know that it's wrong' — it is always confidently 100% sure — so it needs team style guides and anti-patterns baked in, plus guardrails telling the agent what not to waste turns doing, since review has to finish in a bounded time span.",
      "The customization stack is a ladder: per-file general-purpose logic-bug reviewers, deep multi-file agent review carrying each monorepo's style guides/anti-patterns, few-shot 'AI linters' that deterministically gather context then run rules for mechanical issues, and fully custom team agents linked to a knowledge base and past PRs.",
      "Customizations piggyback on Uber's existing ownership model, are co-located next to the code they govern, and are dispatched by smart deterministic routing that picks review type, model and generators per team.",
      "Writing a review skill turned out to be easy — teams asked Claude to read their past PR reviews and write one — but running skills at scale with consistent quality and low cost was the hard part, requiring iteration from both the uReview team and each adopting team.",
      "Results: ~25,000 comments/week, 10% get any feedback, only 4% of PRs get negative feedback, ~67% overall addressal rate, roughly three quarters of high-severity issues addressed; versus a naive implementation, cost down 60% and quality/accuracy up ~70%.",
      "Inner-loop reviews need *higher* accuracy than human-facing ones, or you get 'cavitation' — an agent fixing something, getting another low-quality comment, and fixing backwards; agents will also happily fix 100 nits that would infuriate a human engineer."
    ],
    "takeaways": [
      "Instrument the review loop before tuning it: classify the sentiment of developer replies, track whether comments are actually addressed, and capture agent trajectories (tool calls + reasoning) — that's what moved Uber's quality-to-cost ratio, not better prompts alone.",
      "Post-process aggressively. Multiple generators produce duplicate and low-confidence comments, so rate, categorize, filter and deduplicate before anything reaches an engineer.",
      "Give the agent explicit guardrails about what *not* to spend turns on; a review is time-bounded, and a wandering agent produces a worse review.",
      "Push customization down to teams via the ownership model you already have, co-located with their code — but feed the observability (addressal rate, sentiment, trajectories) back to those teams so they can see which of their own rules developers dislike and fix them.",
      "Raise the accuracy bar for reviews served into the agent inner loop above what you'd accept for humans, and plan for humans reviewing at a higher altitude — architecture, domain expertise, product thinking — rather than removing them."
    ],
    "topics": [
      "agents",
      "code-review",
      "evals",
      "observability",
      "developer-productivity",
      "multi-agent",
      "monorepo",
      "agentic-sdlc"
    ],
    "tools": [
      "Uber",
      "uReview",
      "Phabricator",
      "GitHub",
      "Claude",
      "Google Forms",
      "Slack"
    ],
    "quotes": [
      {
        "text": "Back in 2024, we were seeing that engineers would get their first review within 3 hours. Now in 2026, that has grown to 9 hours",
        "at": "01:07",
        "url": "https://www.youtube.com/watch?v=EL123UNokkI&t=67s"
      },
      {
        "text": "One of the biggest learnings in this process was like the model doesn't know that it's wrong. It always confidently says 100% sure that yeah, this is the review for your code. Go ahead.",
        "at": "06:35",
        "url": "https://www.youtube.com/watch?v=EL123UNokkI&t=395s"
      },
      {
        "text": "Actually writing the skill was very easy. Like teams just very quickly wrote a skill by asking Claude to write one, go over my previous PR reviews and write me a skill. But the hard part was how to run these skills at scale with consistent quality and low cost.",
        "at": "09:44",
        "url": "https://www.youtube.com/watch?v=EL123UNokkI&t=584s"
      },
      {
        "text": "Rather than killing the outer loop, I think that we believe and the industry has just started to really kind of coalesce on this idea that we're really expanding the outer loop. Rather than removing humans from the code review process, we are moving their responsibilities up a layer.",
        "at": "13:45",
        "url": "https://www.youtube.com/watch?v=EL123UNokkI&t=825s"
      }
    ],
    "words": 2872
  },
  "summary_url": "/#EL123UNokkI",
  "transcript": {
    "html": "/transcripts/EL123UNokkI.html",
    "txt": "/transcripts/EL123UNokkI.txt",
    "vtt": "/transcripts/EL123UNokkI.vtt"
  }
}