{ "video": { "id": "v0UkV5qS9E4", "title": "Why We Rebuilt Microsoft Office From Scratch for AI Agents - Yijia Liang, Compound AI", "duration": 945, "upload_date": null, "channel": "AI Engineer", "source": "AI Engineer", "event_format": "" }, "analysis": { "video_id": "v0UkV5qS9E4", "title": "Why We Rebuilt Microsoft Office From Scratch for AI Agents - Yijia Liang, Compound AI", "one_liner": "Compound AI concluded that desktop plugins, VM-based computer use and off-the-shelf engines like openpyxl can't run long, parallel finance agents. So it had agents rebuild Excel, PowerPoint and Word from scratch, using real Office as the oracle, and owning the engine gives it parallelism, pixel-perfect rendering and a write-render-check-fix loop.", "summary": "Yijia Liang, a former Coatue VP, argues that what made personal agents take off (reliable background tasks, proactivity, integrations) doesn't carry over to institutional finance. There, the work is going from a 15,000-file data room to IC models, slides and memos, under NDA and with managing directors who don't want rogue agents. Compound tackles this on three fronts: multiplayer agents that each have their own email inbox, a headless Office rebuilt from scratch, and context handling that uses agent swarms instead of vector DBs, plus engine-processed views of very large workbooks. Liang believes finance is at the same point coding was when Opus 4.5 launched, moving from copilot to autopilot.", "key_points": [ "Every Compound agent gets its own inbox because most finance workflows run through Outlook in threads of 20+ emails. Email agents need authentication so everyone on an NDA'd thread has read/write access to the agent. They also need human-like behaviour, such as adding latency so several quick emails don't each trigger a separate acknowledgement.", "Today's deployments of Claude and ChatGPT are single-player, so the work gets downloaded, emailed, posted to Teams and sent round for feedback. Compound wants every deal-team member in one thread talking to one agent that holds everyone's context.", "They evaluated Office desktop plugins, computer use on VMs, open-source packages like openpyxl and commercial engines. None was good enough, so they rebuilt Office from scratch, spending 1.5 trillion input/output tokens and producing over 8 million lines of code.", "Method: point agents at the objective 'recreate Office in full fidelity', use real Office as the oracle or judge, and benchmark on real files (a public corpus plus internally created ones). The Excel rebuild started from the open-source IronCalc and grew to 800,000 lines of Rust. Rust renderers keep rendering pixel-perfect, because firms 'only trust what they can download and what they can send'.", "Desktop Office is bound to one machine's CPU, while a headless engine runs tasks in parallel. They claim it recalculates 20 MB Excel files 4x faster.", "Owning the editor enables asynchronous comment-driven collaboration. The talk's own deck was built in Compound by dropping in formatting comments, and Liang claims a team of five can get through a 100-page deck in an afternoon instead of weeks.", "Owning the engine gives you the write, render, check and fix loop. With openpyxl there is no renderer to check formatting, so you're 'just guessing and hoping', and Microsoft's own interface provides 'admittedly very little'.", "For breadth of context, skip the vector DB except for specific use cases: a lead agent slices the file directory and swarms of coding agents read files (for example with grep) and report back, with every finding cited to its source. For depth, such as a 30 MB model with 2 million formulas, their engine lays the workbook out in a format the model can easily read and exposes richer APIs and tools. That makes it more accurate, faster and cheaper than an LLM writing openpyxl code." ], "takeaways": [ "If agents must work at scale in documents, consider a headless engine you own instead of desktop plugins or VM computer use, so you get parallelism and a render-based check loop.", "To rebuild a complex existing product with coding agents, give them a clear objective function, use the original product as the oracle or judge, and benchmark on real-world plus internally made files.", "For large unstructured corpora, try a lead agent plus swarms of grep-capable coding agents with citations back to the source before reaching for a vector DB.", "Bring agents into the channels teams already use (for example a per-agent email inbox), and design for auth/privacy and human-like pacing, not just connectivity.", "Instead of having the LLM read raw library output for big files, preprocess them through an engine into a compact format the model can process and expose richer tools; that cuts cost and latency." ], "topics": [ "agents", "finance", "multiplayer", "document engines", "long-running tasks", "context management", "agent swarms", "evals" ], "tools": [ "Compound AI", "Coatue", "Microsoft Excel", "Microsoft PowerPoint", "Microsoft Word", "Microsoft Outlook", "Microsoft Teams", "SharePoint", "Box", "Claude", "ChatGPT", "Opus 4.5", "openpyxl", "IronCalc", "Rust", "iMessage", "WhatsApp", "Cursor" ], "quotes": [ { "text": "we ran agents in many loops, spend 1.5 trillion input-output tokens and over 8 million lines of code to rewrite office from scratch.", "at": "06:15", "url": "https://www.youtube.com/watch?v=v0UkV5qS9E4&t=375s" }, { "text": "these firms only trust what they can download and they what they can send.", "at": "07:33", "url": "https://www.youtube.com/watch?v=v0UkV5qS9E4&t=453s" }, { "text": "So, with an own engine, you actually own the write, render, check, and fix loop.", "at": "09:44", "url": "https://www.youtube.com/watch?v=v0UkV5qS9E4&t=584s" }, { "text": "No longer do you have to index everything in a vector DB unless it's for it's for a very specific use case.", "at": "10:39", "url": "https://www.youtube.com/watch?v=v0UkV5qS9E4&t=639s" } ], "words": 3363 }, "summary_url": "/#v0UkV5qS9E4", "transcript": { "html": "/transcripts/v0UkV5qS9E4.html", "txt": "/transcripts/v0UkV5qS9E4.txt", "vtt": "/transcripts/v0UkV5qS9E4.vtt" } }