{ "video": { "id": "59AA5kIoqjA", "title": "The Search Engine for the Agentic Web — Will Bryk, Exa", "duration": 1069, "upload_date": null, "channel": "AI Engineer", "source": "AI Engineer", "event_format": "In person" }, "analysis": { "video_id": "59AA5kIoqjA", "title": "The Search Engine for the Agentic Web — Will Bryk, Exa", "one_liner": "Exa's founder argues that AI systems will overtake humans in web searches this year and search a thousand times more within a few years, so the search stack must be rebuilt as a near-perfect database of the world's information for agents rather than a human-facing recommendation engine.", "summary": "Will Bryk frames Google as a recommendation engine — it returns striped shirts when you search 'shirts without stripes' — rather than a database that can answer complex queries like 'everyone in Singapore who works on AI search and any blog post or research paper they've written.' Exa's founding thought experiment was running GPT-3 over every query-document pair to judge relevance, which would give near-perfect search at roughly $10M per query, making the whole company an exercise in reducing that cost by a billion or trillion times via embeddings plus keywords. The talk traces Exa from a 2021 start (as Metaphor), through the accidental discovery of the API business when strangers DM'd asking for access, to today's 5,000 companies and 400,000 developers, and closes on 'perfect information' — a year of research in a second — as critical civic infrastructure ahead of the 2028 US election.", "key_points": [ "Web searches per day from AI systems are expected to exceed searches from humans in 2026, and to be roughly 1,000x human volume within a few years — a number Bryk says you can't even fit on the graph.", "Exa serves over 5,000 companies and over 400,000 developers, including Cursor (which calls Exa under the hood when its agent looks up technical docs or news), HubSpot's go-to-market agents, and a wave of New York financial agents.", "The problem framing is 'misinformed anarchy': the web is ~a trillion pages, Google behaves like a recommendation engine akin to social media, and nobody ends up with a complete understanding of anything.", "Founding thought experiment: run GPT-3 over a query and a document to judge match, then do that over a trillion documents per search — near-perfect results, but about $10 million per search, turning search into a billion-to-trillion-x cost-reduction optimization problem.", "The technical bet was embeddings — pre-processing every document into a structure that captures the neural net's intelligence — combined with keywords, plus 'stack more layers' bitter-lesson conviction; they raised a couple million dollars and spent half of it on a GPU cluster.", "The API business was discovered by accident: they launched as Metaphor in 2022 (two weeks before ChatGPT), then a Twitter DM asked for API access, then more requests including from Bryk's own roommate downstairs. The insight: even GPT-5 is tiny compared to the internet, so models always need a retrieval engine.", "Product surface today spans slow-but-exhaustive complex queries (e.g. every YC-funded startup working on AI with batch and status, taking minutes), a 200-millisecond search endpoint Bryk calls the fastest in the world (built for voice agents, not humans), token extraction that returns only the ~100 most important tokens from 10 documents to cut downstream LLM cost, and structured output.", "Exon lets data providers partner with Exa so developers' agents can buy private data alongside the public web — a free market where providers price their own content; the demo combined public web data with non-public Similarweb monthly-visitor numbers." ], "takeaways": [ "Stop writing agent queries in Google-shaped keyword form — Exa is built to answer arbitrarily complex list queries ('every X matching Y, with their Z'), and Bryk says most users don't realise they can push it that far.", "Pick the latency/quality point deliberately: use the 200ms endpoint for voice and interactive agents, and accept minutes-long searches when you need exhaustive, high-recall results.", "Cut LLM spend at the retrieval layer — ask for extracted tokens or structured output instead of piping full documents into your model.", "Treat search as per-application configuration, not a single engine: constrain by domain allowlist/blocklist, time window, and page type (e.g. exclude product pages) to get your own tuned search engine.", "If your agent needs data that isn't on the public web, check whether a provider sells it through Exa's marketplace rather than assuming public-web-only retrieval." ], "topics": [ "agents", "search", "rag", "embeddings", "apis", "retrieval", "llm-cost", "data-marketplace" ], "tools": [ "Exa", "Metaphor", "Exon", "Google", "Cursor", "HubSpot", "GPT-3", "ChatGPT", "GPT-5", "Similarweb", "Y Combinator", "Twitter" ], "quotes": [ { "text": "there are a lot of ways to say the problem but the short way is just misinformed anarchy", "at": "02:13", "url": "https://www.youtube.com/watch?v=59AA5kIoqjA&t=133s" }, { "text": "The problem is that would cost like $10 million per search. So it really becomes an interesting optimization problem.", "at": "06:24", "url": "https://www.youtube.com/watch?v=59AA5kIoqjA&t=384s" }, { "text": "It's way too fast for humans, right? But we're not serving humans, we're serving AI systems.", "at": "11:36", "url": "https://www.youtube.com/watch?v=59AA5kIoqjA&t=696s" }, { "text": "perfect search is not one thing. It's actually we have 5,000 search engines for each of our 5,000 customers", "at": "12:52", "url": "https://www.youtube.com/watch?v=59AA5kIoqjA&t=772s" } ], "words": 4699 }, "summary_url": "/#59AA5kIoqjA", "transcript": { "html": "/transcripts/59AA5kIoqjA.html", "txt": "/transcripts/59AA5kIoqjA.txt", "vtt": "/transcripts/59AA5kIoqjA.vtt" } }