{
  "video": {
    "id": "TJPInBjhE4Q",
    "title": "ReviewDebt: a practical framework for scoring every pull request — Sachin Gupta, Ebay",
    "duration": 1500,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": "Online"
  },
  "analysis": {
    "video_id": "TJPInBjhE4Q",
    "title": "ReviewDebt: a practical framework for scoring every pull request — Sachin Gupta, Ebay",
    "one_liner": "eBay's Sachin Gupta argues coding agents are accruing \"review debt\" — the gap between code agents produce and code humans have actually reviewed — and gives a deterministic 0–100 scorer built from five signal families, demoed against 524 PRs across three public repos.",
    "summary": "Gupta opens on a measurement gap: GitHub's October 2025 data shows commits up 25% year over year while comments on commits fell 27%, and Faros AI's 2026 benchmark shows median PR review time up 441.5% and 31% more PRs merged with no review at all. He defines review debt as the compounding gap between agent-produced code and code humans have reviewed, trusted and understood — compounding because unreviewed code grounds tomorrow's suggestions, reviewers cede architectural calls, and leadership resets velocity expectations without hiring reviewers. He then presents a deliberately LLM-free scorer: five signal families, ten deterministic checks, one 0–100 number with four bands, walked through three scored PRs and a 524-PR cross-repo scan. The scan's punchline is that AI authorship stayed flat at 5–20% while review burden climbed — volume, not authorship, drives the debt.",
    "key_points": [
      "The headline numbers are real but are vanity metrics: PRs per developer up 16% (Faros AI 'acceleration whiplash', April 2026, 22,000 developers / 4,000 teams), median PR size up 63% from 44 to 72 lines (DX 2026, 400 orgs, 16 months), PR throughput up ~8% against ~65% AI-usage growth — 'PR count goes up when one PR splits into seven PRs.'",
      "Review debt compounds through three feedback loops: agents ground on code that was never deeply reviewed; reviewer attention contracts to syntax and obvious bugs when most of a PR was generated, moving big-picture decisions 'from review time to never'; and once leadership sees the new throughput you don't get to hire reviewers in proportion.",
      "Five signal families, ten checks, all deterministic and computable from the PR plus its repo: diff size and coupling, test evidence gap (test lines added / production lines added), directory and ownership spread (distinct CODEOWNERS teams in the diff), AI authorship indicators, and evidence/rationale gaps.",
      "LLM-as-judge is explicitly rejected: the score would become a moving target that changes when the model changes, and it would stop being defensible in an engineering review or in front of leadership.",
      "AI authorship detection uses three modes — co-authored footers (e.g. 'Co-authored-by: Copilot', the strongest signal), branch prefixes (codex/copilot/cursor), and 'generated by'/'assisted by' phrases in PR bodies and commit messages. Across three repos it fired 0% on repo C, which appears to strip those markers.",
      "Three scored PRs demoed: a clean PR at 0/100 and 6 estimated review minutes with no checks fired and empty advice lists; a high-debt PR at 60/100 'needs evidence' with 86 estimated minutes, where the AI indicator contributed only 5 of 60 points (~8%) and the other 55 came from diff size, claim mismatch and missing tests; and a well-shaped AI-authored PR at 7/100 and 14 minutes because tests shipped, CI was green and the risky path was called out.",
      "The 524-PR / three-repo, 27–90-day scan: 228 cumulative senior reviewer hours, one repo alone accumulating 186 in 27 days while another took 43; 9 PRs/day sustained merge rate in the high-velocity repo; a single PR estimated at 5,036 minutes (84 hours) scoring 73; only 4 of 524 PRs landed in the needs-evidence or high bands, and all four were structural — large migrations, SDK rewrites, multi-team refactors.",
      "The scored output is not just a number — it ships a 'why fired' list, a reviewer-focus list and an author-next-action list, and the report itself states that AI indicators are 'information only, not a definitive claim, and not a penalty on its own.'"
    ],
    "takeaways": [
      "Backfill before you build: run the scorer over your last 200 merged PRs and calibrate the weights against your team's actual reviewer experience — or, if you want a cheaper start, hand-score 20 PRs from last week against the five signals.",
      "Adopt in five steps — backfill, set a threshold (default: score ≥50 requires a comment from the author), surface the score as a non-blocking PR comment on every PR, aggregate weekly per team, and bring the number to retros and roadmap reviews.",
      "Watch the slope, not the level: the weekly trend of review debt per team is the leading indicator an engineering manager should be tracking.",
      "Five habits, no new tooling required: one logical change per PR, tests ship with the change and the human author confirms they assert what the code should do, stay inside one owner's territory (split cross-cutting work per team), the author writes the 'why' rather than the agent, and hold AI PRs to the same review standard as human PRs.",
      "Kill the anti-patterns: approve-with-comment merges, 'the AI did the boring part', 'we'll catch it in QA', 'PRs are smaller now' — and stop writing LGTM."
    ],
    "topics": [
      "coding agents",
      "code review",
      "developer productivity",
      "engineering metrics",
      "technical debt",
      "ai governance",
      "testing"
    ],
    "tools": [
      "GitHub",
      "GitHub Copilot",
      "Codex",
      "Cursor",
      "Faros AI",
      "DX",
      "eBay",
      "CODEOWNERS"
    ],
    "quotes": [
      {
        "text": "It actually accrues interest, but this interest is not money. This interest is paid in human attention.",
        "at": "05:05",
        "url": "https://www.youtube.com/watch?v=TJPInBjhE4Q&t=305s"
      },
      {
        "text": "These things tell you the speed of production. They do not tell you the speed of trust.",
        "at": "03:27",
        "url": "https://www.youtube.com/watch?v=TJPInBjhE4Q&t=207s"
      },
      {
        "text": "Code that was not deeply reviewed yesterday grounds tomorrow's PR.",
        "at": "05:22",
        "url": "https://www.youtube.com/watch?v=TJPInBjhE4Q&t=322s"
      },
      {
        "text": "The agent did not cause the score, but the shape of the PR that is created by the agent did this.",
        "at": "16:06",
        "url": "https://www.youtube.com/watch?v=TJPInBjhE4Q&t=966s"
      }
    ],
    "words": 4507
  },
  "summary_url": "/#TJPInBjhE4Q",
  "transcript": {
    "html": "/transcripts/TJPInBjhE4Q.html",
    "txt": "/transcripts/TJPInBjhE4Q.txt",
    "vtt": "/transcripts/TJPInBjhE4Q.vtt"
  }
}