{ "video": { "id": "3Ad73Onx3Aw", "title": "Better Search Took This AI Agent From 36% to 98% Accuracy - Aamir Shakir, Mixedbread", "duration": 990, "upload_date": null, "channel": "AI Engineer", "source": "AI Engineer", "event_format": "" }, "analysis": { "video_id": "3Ad73Onx3Aw", "title": "Better Search Took This AI Agent From 36% to 98% Accuracy - Aamir Shakir, Mixedbread", "one_liner": "Mixedbread argues that agents are limited by access to data rather than intelligence, citing a hedge fund with over 1 billion documents whose answer accuracy rose from 36% to 98%, with LLM spend down 3x, after replacing its OCR-chunking-embedding-vector-database stack with multimodal late-interaction search.", "summary": "Aamir Shakir argues that classic RAG never really worked because parsing, chunking, single-vector embedding and retrieval are each lossy, and that grep-style agent search over large document sets burns tokens because agents end up reading everything. He contends that single embeddings cannot separate near-identical documents, such as Nvidia earnings reports from different years, so quality degrades as a corpus scales and agents compensate with more queries and sub-agents. Mixedbread's answer is a vertically integrated product with its own models, database and ingestion pipeline, built around a multimodal late-interaction model that embeds what each page looks like. A live demo and a hedge fund case study are used to support the claim that better search improves accuracy and cuts LLM costs.", "key_points": [ "Grep-style lexical search, as used by Claude Code or Claude Cowork on locally ingested data, returns many hits with no relevancy signal, so the model reads every document and a basic question can cost $10 or $20 per query.", "Many PDFs are not really greppable because they are binary formats or contain scans and images, so multimodal data goes unexploited.", "The 'Frankenstein stack' of parsing/OCR, chunking, embedding, vector database and reranking loses information at every step, and each step needs its own vendor; Shakir says teams can spend six to eight months evaluating vendors and allocate four to five people to build something that still won't work.", "Single-vector embeddings struggle with complex documents: Nvidia's earnings reports from 1996 and 2026 would produce very similar points, so quality may be good at tens or hundreds of documents but degrades significantly at production scale.", "When retrieval is poor, agents query aggressively and spin up tens of sub-agents, which fills the context window and makes every query expensive; costs would skyrocket as agent traffic grows.", "Shakir presents retrieval history as BM25 first, then the first really good embedding models in 2022, then roughly linear improvement with no step-function change; he calls the late-interaction model Mixedbread is publishing next week, already live with many customers, that step change.", "The late-interaction model creates multiple contextualized embeddings per document and per query, for example separate embeddings for net income and net revenue that still know the filing is about Nvidia. Embeddings are created from a page screenshot rather than OCR text, while layout information and OCR remain available for non-multimodal LLMs and bounding boxes allow answers to be traced to their source.", "Mixedbread's own agentic search model does multiple queries and multi-hop reasoning; Shakir claims it is as good as Astra and Fable while being 40 times faster and cheaper, and normal search is said to usually take under 200 milliseconds.", "In the case study, a large hedge fund with over 1 billion documents was onboarded in about one and a half weeks; answer accuracy, graded against questions where human experts knew the answer, went from 36% to 98%, multimodality gave a significant bump, and LLM spend fell 3x." ], "takeaways": [ "Treat poor agent answers as a data-access problem first: Shakir's case study attributes the accuracy jump to better search, not a smarter model.", "Don't rely on grep alone across large PDF-heavy knowledge bases; it misses scans and images and pushes token costs up as agents read everything.", "Test RAG quality at production scale on many similar documents, not just tens or hundreds, because that is where single-vector approaches are said to degrade.", "Consider multi-vector late-interaction retrieval and visual page embeddings for tables, figures and financial filings, where OCR and chunking lose information.", "Give agents grep, normal search and agentic search as primitives and let them choose by query complexity, and keep bounding-box citations so decisions can be traced back to their source." ], "topics": [ "retrieval", "rag", "agents", "late-interaction", "multimodal-search", "embeddings", "agentic-search", "finance" ], "tools": [ "Mixedbread", "Claude Code", "Claude Cowork", "grep", "Pinecone", "Elasticsearch", "BM25", "ChatGPT", "Astra", "Fable", "Intel D270", "Nvidia" ], "quotes": [ { "text": "the real real bottleneck is actually access to your data and to your knowledge", "at": "08:35", "url": "https://www.youtube.com/watch?v=3Ad73Onx3Aw&t=515s" }, { "text": "we internally call it the Frankenstein stack", "at": "05:33", "url": "https://www.youtube.com/watch?v=3Ad73Onx3Aw&t=333s" }, { "text": "So it took the space 27 years to come something up for text search better than BM25.", "at": "09:25", "url": "https://www.youtube.com/watch?v=3Ad73Onx3Aw&t=565s" }, { "text": "the issue is not intelligence it's again getting the right data", "at": "15:17", "url": "https://www.youtube.com/watch?v=3Ad73Onx3Aw&t=917s" } ], "words": 3571 }, "summary_url": "/#3Ad73Onx3Aw", "transcript": { "html": "/transcripts/3Ad73Onx3Aw.html", "txt": "/transcripts/3Ad73Onx3Aw.txt", "vtt": "/transcripts/3Ad73Onx3Aw.vtt" } }