{
  "video": {
    "id": "474j-n1Ltxc",
    "title": "AI-Generated Code Is Already Competing With Human Code — Daksh Gupta, Greptile",
    "duration": 761,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": ""
  },
  "analysis": {
    "video_id": "474j-n1Ltxc",
    "title": "AI-Generated Code Is Already Competing With Human Code — Daksh Gupta, Greptile",
    "one_liner": "Greptile's data from reviewing more than a million pull requests a month shows that about a quarter of PRs are now fully or largely AI-generated, and that these PRs match human PRs on revert rate, severity of bugs found and review rounds to merge, though each agent's typical failures differ from human ones.",
    "summary": "Daksh Gupta, co-founder of Greptile (an AI code-review company), expected fully autonomous coding agents to be mostly Twitter hype. He tested that against Greptile's review data from thousands of companies, most of them enterprises. He detected AI-authored PRs from author fields, 'co-authored by' footers and branch-name prefixes, then compared their quality with human PRs on three measures: revert rate, how many severe bugs Greptile flagged, and review rounds to merge. He found no meaningful quality gap, but each agent had its own failure pattern. He argues that when top users open hundreds or thousands of PRs a month, validation has to be rethought from first principles around three questions, using agents that run code in a sandbox.",
    "key_points": [
      "Detecting AI-generated PRs was 'surprisingly difficult'. The GitHub author field showed Codex, Claude or Cursor on fewer than 1% of PRs. Adding 'co-authored by Claude/Cursor' PR-description footers and agent branch-name prefixes (for example Codex's) raised the estimate to about a quarter of all PRs Greptile reviews each month being completely or largely AI-generated.",
      "Looking back 12 months, fewer than 1% of PRs showed evidence of being fully AI-generated early last year. The growth looks continuous, and you can't see new model releases on the chart; it is 'diffusing into the economy' steadily.",
      "Revert rates, spotted through GitHub's 'revert-<PR number>' branch naming: Codex about 1 per 1,000 PRs, Devin about 3.5 per 1,000, humans in between at about 2.5. PR size showed little correlation with reverts, which undercuts the idea that humans simply take the harder work.",
      "Greptile's own P0/P1/P2 bug findings: three of the four agents tested produced fewer P0s than humans, with a similar picture for P1s and P2s. Overall, human and agent PRs were 'about equal in quality'.",
      "Review rounds from open to merge (usually an agent addressing Greptile's comments with a new commit): Devin 2.1, Codex about 2.45, humans in between. There was little if any statistical difference.",
      "Failure types do differ. Scanning several million Greptile comments (about 4 per PR) for phrases like 'SQL injection' and 'N+1 query', relative to a human baseline of 1x, showed Claude is 1.5x as likely as humans to produce a SQL injection issue, while Devin is about half as likely to produce an auth bypass issue.",
      "Among Greptile users, the median writes 50 PRs a month (2+ per workday), P90 writes 500 a month and P99 writes thousands. People at the margins are opening PRs as fast as they have ideas, a pace that manual review, testing and QA firms can't match.",
      "Greptile's first-principles framing of validation comes down to three questions: does the change violate the user contract, does it increase the chance of a future violation, and does it do what the author intended? Almost a fifth of PRs Greptile reviews are now merged with no human review or human testing."
    ],
    "takeaways": [
      "Don't assume agent-written PRs are worse by default. Measure them in your own codebase with revert rate, severity-weighted review findings and review rounds to merge.",
      "Profile failure modes per agent, not just overall quality. For example, look harder for SQL injection in Claude-generated changes, because each agent's error pattern differs from humans'.",
      "Make AI authorship detectable: keep 'co-authored by' footers and agent branch prefixes so you can attribute and analyse AI-generated PRs later.",
      "Plan validation for hundreds of PRs per engineer per month. Use agents that run the code in a sandbox, install dependencies, mock inputs and drive a browser to check the user contract and the author's intent, rather than scaling manual review."
    ],
    "topics": [
      "coding agents",
      "code review",
      "ai-generated code",
      "code quality metrics",
      "security",
      "developer productivity",
      "sandboxed testing"
    ],
    "tools": [
      "Greptile",
      "GPT-3.5",
      "Cursor",
      "GitHub Copilot",
      "Codex",
      "Claude",
      "Devin",
      "GitHub",
      "Nvidia",
      "Coinbase",
      "Scale",
      "Datadog",
      "American Express"
    ],
    "quotes": [
      {
        "text": "What's also interesting is that you can't tell when new models came out on this chart.",
        "at": "05:00",
        "url": "https://www.youtube.com/watch?v=474j-n1Ltxc&t=300s"
      },
      {
        "text": "Broadly speaking, human generated PRs were about equal in quality to agent generated PRs based on this data.",
        "at": "07:29",
        "url": "https://www.youtube.com/watch?v=474j-n1Ltxc&t=449s"
      },
      {
        "text": "For instance, Claude is one and a half times more likely to produce a SQL injection error than humans.",
        "at": "09:16",
        "url": "https://www.youtube.com/watch?v=474j-n1Ltxc&t=556s"
      },
      {
        "text": "The 90th percentile writes 500 poll requests per month.",
        "at": "10:19",
        "url": "https://www.youtube.com/watch?v=474j-n1Ltxc&t=619s"
      }
    ],
    "words": 2826
  },
  "summary_url": "/#474j-n1Ltxc",
  "transcript": {
    "html": "/transcripts/474j-n1Ltxc.html",
    "txt": "/transcripts/474j-n1Ltxc.txt",
    "vtt": "/transcripts/474j-n1Ltxc.vtt"
  }
}