{
  "video": {
    "id": "Ki980nV0__0",
    "title": "Computer-use models will agentify the web, not APIs — Dhruv Batra, Yutori",
    "duration": 1260,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": "In person"
  },
  "analysis": {
    "video_id": "Ki980nV0__0",
    "title": "Computer-use models will agentify the web, not APIs — Dhruv Batra, Yutori",
    "one_liner": "Dhruv Batra of Yutori argues the web will be agentified by vision-based computer-use models clicking through pixels, not by a long tail of websites suddenly exposing APIs and MCP servers — and shows benchmark, latency and cost numbers to say that trend is already arriving.",
    "summary": "Batra accepts the common claim that AI agents will become the primary drivers of action on the web, but rejects its usual mechanism: that the web will hand agents APIs and protocols (MCP, web MCP, the 20-odd competing payment protocols). Using restaurant menus buried in JPEG galleries and school-district RFPs that require a Freedom of Information Act email answered via scanned PDFs on Google Drive, he shows the long tail of ~200 million active sites will never ship an endpoint, and that reading HTML fails too because modern pages fetch JSON asynchronously and render state (greyed-out = sold out) rather than writing it as text. His conclusion is a bitter lesson for web agents: the web was built for human eyes, so pixels are the source of truth and the general solution is screenshot-in, clicks-and-code-out — demonstrated with Yutori's Navigator model, 97% human eval on Mind2Web and 80¢ per task versus $2.30 for frontier models.",
    "key_points": [
      "The flight-booking demo is the wrong poster child: aggregators already take a variable and return JSON, and LLMs can tool-call, so clicking buttons on flights.google.com is pointless — the real cases are the long tail where no endpoint exists.",
      "Restaurant menus escalate from easy (German text page with prices) to medium (menu button opens a PDF) to hard (a Spanish site where 'Nuestra carta' leads to a gallery of pixelated JPEGs of the menu that ChatGPT struggles to OCR) — none of these will expose a gluten-free query parameter.",
      "For the 15,000–20,000 US school districts, answering 'is this district procuring a laptop right now?' means navigating Finance → Purchasing → Solicitations to scanned PDFs, and at boss level filing a Freedom of Information Act request whose answer arrives as a scan of your own email plus PDFs attached on a Google Drive: 'These are the people you're telling me will give you an MCP server?'",
      "Scale and inertia kill the API story: ~200 million active websites and roughly a billion total, infrastructure that changes very slowly, institutions still faxing each other, and no unfettered access for a coding agent to generate endpoints even if the technology arrived.",
      "Throwing coding agents at HTML also fails — nba.com renders an empty placeholder and fetches the 125–109 score by async JSON hundreds of milliseconds to seconds later, so the answer is simply not in the loaded HTML.",
      "On a product page, HTML's option selector for the 25-mm osmium cube says neither 'sold out' nor 'available'; the browser fetches a JSON product count whose `quantity` field (10, or 0) drives a rendering script that greys out and disables zero-stock options — the information is calculated and rendered, not written.",
      "The bitter lesson for web agents: scaffolds built around existing sites don't generalize to the long tail; the browser is a rendering engine like a game engine, and reading source to predict pixels is an exact inversion of that process, so the general solution is pixels in.",
      "Yutori's Navigator shipped its first version in November last year as pure screenshot-in, clicks-and-scrolls-out — able to validate a natural-language discount-code constraint by finding the product, adding to cart, applying the code and checking the 22%-off claim over 20–40 steps.",
      "The next Navigator version writes JavaScript on demand (a Chrome extension action 'execute JS, value default, text select' filled multiple form fields at once); the screenshot acts as a built-in formal verification system, so it clicks when it must, writes code when it can, and confirms through pixels.",
      "Navigator N1.5 sits at 97% human eval on Mind2Web — 8 of 300 trajectories incorrect over 30–50 interaction steps — which Batra calls saturated and says should be retired for something harder; against Opus 4.7 and GPT 5.5 its accuracy gain is within statistical noise but its smaller footprint gives lower latency per step and 80¢ versus $2.30 per task."
    ],
    "takeaways": [
      "Stop designing long-tail automation around a future API or MCP server — for restaurant menus, school-district procurement and similar institutional sites, assume the endpoint will never exist and plan for a vision-based agent instead.",
      "Where an aggregator or real API already exists (flights being the canonical case), use tool calling — don't burn a computer-use trajectory on something a JSON endpoint answers.",
      "Don't assume loaded HTML contains the answer: check whether the value arrives via an async JSON fetch or is produced by a rendering script (greyed-out and unclickable rather than the words 'sold out'), because scraping the markup will silently miss it.",
      "Build agents that mix modalities — click when necessary, write and execute JavaScript when that's faster, and always verify the outcome against the screenshot as the source of truth.",
      "Evaluate computer-use models on latency per step and cost per task, not just accuracy; a smaller model matching frontier accuracy within noise at 80¢ versus $2.30 per task is what makes parallel, orchestrator-launched fleets of sandboxed browsers viable.",
      "Retire benchmarks once they saturate — at 97% human eval with 8 bad trajectories out of 300, Mind2Web is no longer measuring anything, and harder, longer-horizon tasks are the next target."
    ],
    "topics": [
      "computer-use agents",
      "web agents",
      "mcp",
      "browser automation",
      "benchmarks",
      "vision models",
      "agent cost",
      "multi-agent systems"
    ],
    "tools": [
      "Yutori",
      "Navigator (N1.5)",
      "MCP",
      "web MCP",
      "ChatGPT",
      "Mind2Web",
      "Opus 4.7",
      "GPT 5.5",
      "Chrome extension",
      "nba.com",
      "flights.google.com",
      "Google Drive"
    ],
    "quotes": [
      {
        "text": "This last bit that suddenly the web will provide you APIs for accessing things is is just delusional.",
        "at": "01:25",
        "url": "https://www.youtube.com/watch?v=Ki980nV0__0&t=85s"
      },
      {
        "text": "These are the people you're telling me will give you an MCP server?",
        "at": "07:19",
        "url": "https://www.youtube.com/watch?v=Ki980nV0__0&t=439s"
      },
      {
        "text": "Fundamentally, the web was built for human eyes. Pixels are the source of the truth because the consumers of the websites are humans.",
        "at": "12:08",
        "url": "https://www.youtube.com/watch?v=Ki980nV0__0&t=728s"
      },
      {
        "text": "We will just pile on another layer of mess on top of the mess that the web is.",
        "at": "20:04",
        "url": "https://www.youtube.com/watch?v=Ki980nV0__0&t=1204s"
      },
      {
        "text": "And at some point, you will say, \"Yeah, that's an API. Like, why do I care?\"",
        "at": "20:33",
        "url": "https://www.youtube.com/watch?v=Ki980nV0__0&t=1233s"
      }
    ],
    "words": 4029
  },
  "summary_url": "/#Ki980nV0__0",
  "transcript": {
    "html": "/transcripts/Ki980nV0__0.html",
    "txt": "/transcripts/Ki980nV0__0.txt",
    "vtt": "/transcripts/Ki980nV0__0.vtt"
  }
}