{ "video": { "id": "I2LL_wd89-A", "title": "How VS Code Went from Monthly to Weekly Releases with AI — Harald Kirschner", "duration": 1175, "upload_date": null, "channel": "AI Engineer", "source": "AI Engineer", "event_format": "" }, "analysis": { "video_id": "I2LL_wd89-A", "title": "How VS Code Went from Monthly to Weekly Releases with AI — Harald Kirschner", "one_liner": "VS Code moved from monthly to weekly releases by changing the whole shipping system around AI, not by having engineers use more AI: agent-ready codebases, 10x faster builds with TypeScript Go, agents that can run and screenshot the app, mandatory AI code review, automated triage and error-fixing, staged rollouts, and evals that are easy to extend.", "summary": "Harald Kirschner, who works on VS Code, describes how a very small team shipping to over 50 million users went from a monthly release cycle, used for over 10 years, to a weekly one. He argues the gain did not come from 'token maxing'. It came from removing one bottleneck after another in three stages: shipping code faster, keeping quality high at that speed, and learning faster so the team ships the right thing. Examples include a code survival metric, a component screenshot-diff browser, a Playwright-driven /launch skill, AI triage of issues and error stacks that opens fix PRs automatically, staged rollouts, and VSC bench evals.", "key_points": [ "VS Code tracks 'code survival': the percentage of agent-written code that actually gets committed. It was 55% with GPT-4.1 and is 86% today with Claude Opus 4.6, thanks to better models and harness improvements. He reads this as growing developer trust.", "AI raised the volume of everything: more issues (some better quality, more of them automated and low quality), more open PRs, and more merged community PRs. VS Code then moved from monthly to weekly releases after 10+ years of monthly ones.", "To make the codebase agent-ready they use lightweight AGENTS.md files as living maps of the code, plus skills that encode expert knowledge. An accessibility skill maintained by the accessibility area owner replaced pulling in one specific person. The litmus test: can a PM vibe-code effectively in the VS Code repo?", "Switching to TypeScript Go made builds 10x faster. That matters because with 10 or 20 agents running, any slow part of CI/CD compounds.", "Agents need to use the product to check UI. A component browser screenshots every VS Code component on each change and highlights the differences. A /launch skill uses Playwright to open VS Code, click through a scenario, gather logs, and confirm fixes before and after. For iOS he recommends Xcode MCP.", "Automated GitHub Copilot code review is now mandatory on every PR, with low, medium or high effort levels coming. Humans don't look at a PR until all review comments are resolved.", "Issue triage is automated: AI filters spam, enriches and translates issues, and assigns area owners. A Chrome extension lets humans correct duplicate detection and feed that back. Error telemetry (51 billion events per day) is filtered to stacks that have the full stack, fingerprinted into buckets, and turned into about 10 issues assigned to area owners, each with an auto-created PR that tries to fix it.", "Releases moved from 'yolo' 100% rollouts to staged rollouts with monitoring, because rollbacks on an installed app are expensive. VSC bench evals are extended by filing a GitHub issue, which an agent bootstraps into a scenario from a template. A hello-world eval showed the most expensive model (not the high-reasoning one) using 70x more tokens. Prototypes now drive daily feedback loops." ], "takeaways": [ "Look for bottlenecks across the whole delivery system (build speed, review, triage, rollout, evals) instead of pushing people to use more AI each day. Fix the next bottleneck, then repeat.", "Give agents a way to run and see your app (Playwright, a screenshot-diff component browser, Xcode MCP) so they can verify UI and fixes themselves.", "Write AGENTS.md files and skills that hold scarce expert knowledge (such as accessibility), keep them as living documents, and have the area owner maintain them.", "Put human feedback loops into automation, such as a tool for correcting duplicate-issue mistakes, and use staged rollouts with monitoring rather than shipping to 100% at once.", "Make evals trivially easy to extend from real issues, and study how each model behaves in your harness, including token cost on trivial tasks." ], "topics": [ "agents", "developer productivity", "release engineering", "code review", "evals", "ci/cd", "triage automation", "ui testing" ], "tools": [ "VS Code", "GitHub Copilot", "GitHub", "GPT-4.1", "Claude Opus 4.6", "TypeScript Go", "Playwright", "Electron", "Xcode MCP", "AGENTS.md", "Chrome extension", "VSC bench" ], "quotes": [ { "text": "If your agent cannot use your application, your product directly to get this feedback loop of is it all working, then that's a really big investment that pays off every time you work on UI.", "at": "08:48", "url": "https://www.youtube.com/watch?v=I2LL_wd89-A&t=528s" }, { "text": "you have 10 agents, 20 agents running and hitting the same bottleneck, these any slow part of your CI/CD will compound.", "at": "07:26", "url": "https://www.youtube.com/watch?v=I2LL_wd89-A&t=446s" }, { "text": "humans will not even look at PR until all the comments are resolved and and addressed.", "at": "11:57", "url": "https://www.youtube.com/watch?v=I2LL_wd89-A&t=717s" }, { "text": "The same five character file took the most expensive model 70x more tokens and that was not the high reasoning model.", "at": "17:18", "url": "https://www.youtube.com/watch?v=I2LL_wd89-A&t=1038s" } ], "words": 3731 }, "summary_url": "/#I2LL_wd89-A", "transcript": { "html": "/transcripts/I2LL_wd89-A.html", "txt": "/transcripts/I2LL_wd89-A.txt", "vtt": "/transcripts/I2LL_wd89-A.vtt" } }