{
  "video": {
    "id": "vS2IA9pq0Q4",
    "title": "Version-Controlled Search: The Web as It Was — Ryan Ahern, Exa",
    "duration": 1153,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": ""
  },
  "analysis": {
    "video_id": "vS2IA9pq0Q4",
    "title": "Version-Controlled Search: The Web as It Was — Ryan Ahern, Exa",
    "one_liner": "Exa Snapshot is a set of 400B+ dated snapshots of the web. It lets you run a search as it would have returned at a past date, so you can backtest web-search-driven financial models and agents without future information leaking into the results.",
    "summary": "Ryan Ahern, a forward deployed engineer at Exa, starts by explaining why agentic web search matters. Agents now handle most searches for people, and they run many long, detailed queries per task, so the quality, cost and latency of the search tool decide whether the agent succeeds. He then presents Exa Snapshot. Exa's crawler checks pages for freshness, and the version history from those checks lets you fetch a URL as it was on or before a date, or run a search limited to what existed at that point. The main use he shows is backtesting financial models: a quick OpenAI run-rate example, a company-acquisition prediction example, and three archetypes ranging from traditional models to fully agentic predictors.",
    "key_points": [
      "Agents search differently from humans. They write verbose, detailed queries and run many searches per task before answering, while humans type a few keywords and give up after a couple of tries.",
      "Exa runs its own crawling infrastructure, database, embedding model and retrieval architecture. Customers include Cursor, which searches for the latest docs, Gamma (for example, pulling Apple's 2018 SEC data), hedge funds, asset managers, sovereign wealth funds, insurers and VCs.",
      "Exa's three tenets are quality, cost and latency. Fewer tool calls mean cheaper inference and a better chance the agent's trajectory succeeds. Dynamic highlights pick the right snippets across all search results, giving higher-quality results with 95% fewer tokens than full page text. Exa also released a new benchmark that separates search-tool performance from the model calling it. Exa Agent Ultra handles long-running deep research.",
      "Exa Snapshot holds 400B+ snapshots of the web, captured during Exa's freshness checks. You can use it in two ways: fetch a URL as it was at or before a given date, or run a version search that returns only results that would have been served at that point.",
      "OpenAI example: a snapshot search as of October 1st surfaces Axios reporting of an annualized run rate of about $70B. A live search shows later reporting of about $50B, because cloud-provider revenue should be counted as a vendor pass-through, so net annualized revenue is the better figure. Snapshot lets you ask what information, such as an SEC filing footnote or a less reputable outlet, existed before the market caught on.",
      "Acquisition-prediction example: a live search simply finds the deal announcement. With Snapshot you query 1 day, 7 days or a month before the announcement, build a set of such companies, and test whether the model can predict the deal. You can also build about 50 sample sets and check them against a holdout to show the approach generalizes.",
      "Three backtesting archetypes: (1) a traditional ML model or decision tree fed with structured data extracted from the versioned web, such as software prices, sentiment or terms and conditions; (2) adding agentic features, such as an adverse media scan or equity research, to an existing model, then backtesting them; (3) a fully agent-driven predictor where Snapshot stops future information leaking through a web tool call.",
      "In a workshop with about 100 people, one high-scoring feature set included 'investor publicly called for a sale', 'share price at least 60% below its 5-year high', and 'is the company named Dominion?'. Ahern uses the last one as an example of overfitting that Snapshot-based evals can catch and test against for generalization."
    ],
    "takeaways": [
      "When backtesting any model or agent that uses web search, use point-in-time search such as Exa Snapshot so future information, like a deal announcement, cannot leak in and inflate accuracy.",
      "Backtest new agentic features, such as an adverse media scan or equity research, against your existing process before adopting them. For example, check what the scan would have said when a company entered your portfolio months ago, so you don't have to take on faith that search is better.",
      "Build evals with many sample sets and a holdout to check that agent-discovered features generalize rather than overfit, like the 'is the company named Dominion?' feature. Doing this against the live web would mean waiting months.",
      "Measure the cost of search tools by the agent's total task cost, not the price of one tool call. Return targeted snippets, such as dynamic highlights, instead of full pages to cut input tokens."
    ],
    "topics": [
      "web search",
      "agents",
      "backtesting",
      "finance",
      "evals",
      "data leakage",
      "point-in-time data"
    ],
    "tools": [
      "Exa",
      "Exa Snapshot",
      "Exa Agent Ultra",
      "Exa dynamic highlights",
      "Google",
      "ChatGPT",
      "Claude",
      "Cursor",
      "Gamma",
      "OpenAI",
      "Axios"
    ],
    "quotes": [
      {
        "text": "Agents will be very verbose. Um they they tend to describe what they want to search in great detail and will execute many searches in order to understand the space more fully before they respond.",
        "at": "02:00",
        "url": "https://www.youtube.com/watch?v=vS2IA9pq0Q4&t=120s"
      },
      {
        "text": "you can actually get higher quality results at 95% fewer tokens than you would at full page text.",
        "at": "05:49",
        "url": "https://www.youtube.com/watch?v=vS2IA9pq0Q4&t=349s"
      },
      {
        "text": "I want to make sure that I can control that feature so I know that future information is not just going to leak directly into my model.",
        "at": "15:35",
        "url": "https://www.youtube.com/watch?v=vS2IA9pq0Q4&t=935s"
      },
      {
        "text": "Hey, is the company named Dominion? Now this is a little bit of overfitting for people that know like backtesting.",
        "at": "16:57",
        "url": "https://www.youtube.com/watch?v=vS2IA9pq0Q4&t=1017s"
      }
    ],
    "words": 4712
  },
  "summary_url": "/#vS2IA9pq0Q4",
  "transcript": {
    "html": "/transcripts/vS2IA9pq0Q4.html",
    "txt": "/transcripts/vS2IA9pq0Q4.txt",
    "vtt": "/transcripts/vS2IA9pq0Q4.vtt"
  }
}