{
  "video": {
    "id": "ZFxh7sqbUZo",
    "title": "Teaching AIs to Hack — Prof. David Brumley, Bugcrowd",
    "duration": 1637,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer"
  },
  "analysis": {
    "video_id": "ZFxh7sqbUZo",
    "title": "Teaching AIs to Hack — Prof. David Brumley, Bugcrowd",
    "one_liner": "CMU professor and Bugcrowd chief AI/science officer David Brumley argues that most cybersecurity benchmarks are broken because they reward finding one easy crash, and shows that on 41 real Chrome V8 vulnerabilities frontier models achieved full arbitrary code execution up to 73% of the time — on par with an elite human researcher.",
    "summary": "Brumley frames teaching LLMs to hack the same way he teaches high schoolers: a ladder along two axes — target difficulty (toy → CTF → hardened targets) and exploitation difficulty (find bug → crash → arbitrary read/write → full code execution) — with deterministic graders, never LLM-as-judge. He argues existing benchmarks (Cybench, CyberGym, BountyBench, AIxCC) are structurally flawed because real programs contain multiple vulnerabilities, so models reward-hack by repeatedly finding the easiest bug, and because pointing the model at a backtrace stunts its reasoning. His fix is the 'audit task': ask for all vulnerabilities with proofs, uniquify them by stack backtrace, and score multiplicative precision and recall so unknown bugs count and spam doesn't. He then presents ExploitBench, running 41 hand-verified V8 exploits across models to measure how far up the exploitation ladder each one climbs.",
    "key_points": [
      "Two-axis design for RL cyber tasks: target difficulty (toy programs → CTF/synthetic → hardened real targets) and exploitation difficulty (locate bug → trigger crash → arbitrary read/write → full arbitrary code execution). 'Hacking is really a ladder,' which is why it maps so well to RL — a graduated task list plus a good oracle.",
      "LLM-as-judge fails in security: 'The LLMs will always say they were successful hacking.' The gym must use a deterministic grading oracle, exposed with the vulnerable app inside a container, via MCP functions (setup, read/write in sandbox, grade), and the prompt must ask the model to *exploit*, not just find, so a real exploit witness distinguishes hallucination from a bug.",
      "The single-vulnerability assumption behind current benchmarks is empirically false. DARPA spent $60M on the Cyber Grand Challenge and 50% of its hand-curated challenges contained unknown vulnerabilities that were actually exploited; in AIxCC at DEF CON (Brumley designed the scoring algorithm), 18 of the bugs found were unintended.",
      "Catch-22 in existing evals: benchmarks like Cybench hand the model a backtrace identifying the vulnerable function, so it can fit that function in context and doesn't have to reason — but omit it and, with multiple bugs present, the model just reward-hacks the easiest crash every time.",
      "The 'audit task' fix: ask for all vulnerabilities discovered, run every submitted proof-of-vulnerability through the deterministic grader, uniquify crashes by stack backtrace (the same method Microsoft/Apple crash reporting uses), normalize the known-vuln set to include newly discovered bugs, and score multiplicative precision × recall — recall pushes toward unknown bugs, precision blocks spamming non-vulns.",
      "Hard-target result: V8 (which powers Chrome, Edge, Node.js, and Cloudflare Edge Workers) was tested on 41 hand-verified exploitable vulnerabilities, validated by Chrome security lead Sung Hin Lee, against a 16-capability ladder from 'triggers the vulnerable line' through in-sandbox arbitrary read/write to out-of-sandbox primitives and full ACE. In-sandbox crashes are expected behavior and worth nothing; only out-of-sandbox escapes make V8 a $10K–$100K bounty (millions on the black market).",
      "Crash-triggering doesn't distinguish models: GPT, GPT 5.5, and 'Mythos' all hit 95% (39/41), and weaker models (Gemini, Kimi, MiniMax, GLM) still hit ~50%. Full arbitrary code execution does distinguish: Mythos 73% (30/41), GPT 68%, Gemini and Kimi 0%.",
      "Evidence against memorization: on CVE-2023-670x, Mythos reversed JavaScript's math.random to forge a pointer for a return-oriented program out of the Uber cage — a route experts thought too hard in practice. On CVE-2024-7965 it found a new WASM path past where all public work stopped and exploited it on x86, which Brumley's own internal expert didn't think was possible; on CVE-2024-0519 there was a public vuln but no public exploit. 'The work was on par with a human elite researcher.'",
      "ExploitBench is downloadable at exploitbench.ai as Docker images pulled from GitHub with an MCP interface — all data and transcripts released except Mythos's, withheld both under NDA and because it produced weaponized non-public exploits, an unresolved open-science problem Brumley explicitly says he has no answer to.",
      "Bugcrowd runs a vulnerability-mining machine (built on a decade of DARPA work) to find zero-days in open-source software specifically so RL environments can't be memorized, and supplies partner companies up to 10,000 RL environments per month."
    ],
    "takeaways": [
      "Never use LLM-as-judge for security grading — build a deterministic oracle per rung of the ladder (crash → arbitrary read/write → control-flow hijack, e.g. 'launch a calculator or get a reverse shell'), and always require an exploit rather than a claim.",
      "Stop building single-bug synthetic benchmarks. Ask 'find all vulnerabilities' against real open-source targets, add backtrace-based crash uniquification to your grader, and score multiplicative precision and recall so the model can be credited for bugs you didn't know about without being able to spam.",
      "Don't hint the vulnerability location (no backtraces, no 'the bug is in this function') — it lets the whole function fit in context and stunts the model's reasoning. Also avoid guaranteeing a bug exists: telling the model 'go find a bug' is itself leaked information that biases it toward finding exactly one.",
      "Measure weaponization, not crashes. If your eval stops at 'did it crash,' you'll report that a weak model 'hacks 50% of the time' when it can't escape a sandbox at all — build a multi-rung capability ladder (Brumley used 16 buckets) so hard targets still yield signal when the model fails.",
      "Audit your transcripts by hand with an actual security expert: check whether the model memorized the answer, whether it reward-hacked the grader, and how you'll handle vulnerabilities it found that you didn't know about."
    ],
    "topics": [
      "reinforcement-learning",
      "cybersecurity",
      "evals",
      "benchmarks",
      "reward-hacking",
      "offensive-security",
      "agents",
      "grading-oracles"
    ],
    "tools": [
      "picoCTF",
      "Pwn2Own",
      "Bugcrowd",
      "Carnegie Mellon University",
      "DARPA Cyber Grand Challenge",
      "AIxCC",
      "DEF CON",
      "Cybench",
      "CyberGym",
      "BountyBench",
      "ExploitBench (exploitbench.ai)",
      "Chrome",
      "V8",
      "Node.js",
      "Microsoft Edge",
      "Cloudflare Edge Workers",
      "MCP",
      "Docker",
      "GitHub",
      "OpenAI",
      "Anthropic",
      "Claude",
      "GPT / GPT 5.5",
      "Mythos",
      "Gemini",
      "Kimi",
      "MiniMax",
      "GLM",
      "Tesla"
    ],
    "quotes": [
      {
        "text": "We want to take control of that program. That's the beautiful thing about hacking. It's bending computers to our will. It's what makes it unique in the sciences.",
        "at": "04:12",
        "url": "https://www.youtube.com/watch?v=ZFxh7sqbUZo&t=252s"
      },
      {
        "text": "One of the things I think the previous talk was talking about was LLM as a judge is a reasonable thing. What we found in cybersecurity is that is flawed. The LLMs will always say they were successful hacking.",
        "at": "07:27",
        "url": "https://www.youtube.com/watch?v=ZFxh7sqbUZo&t=447s"
      },
      {
        "text": "All they really checked is whether the AI could crash the program. Crashing a program is different than hacking it. You can't go steal someone's IP by simply crashing a program.",
        "at": "18:06",
        "url": "https://www.youtube.com/watch?v=ZFxh7sqbUZo&t=1086s"
      },
      {
        "text": "If you could give Chrome to an LLM and it could come up with a zero-day, you would essentially be able to hack nation-states at that point.",
        "at": "19:53",
        "url": "https://www.youtube.com/watch?v=ZFxh7sqbUZo&t=1193s"
      }
    ],
    "words": 5806
  },
  "summary_url": "/#ZFxh7sqbUZo",
  "transcript": {
    "html": "/transcripts/ZFxh7sqbUZo.html",
    "txt": "/transcripts/ZFxh7sqbUZo.txt",
    "vtt": "/transcripts/ZFxh7sqbUZo.vtt"
  }
}