{ "video": { "id": "ZTA0GwpAUak", "title": "Bringing Continual Learning into Enterprises — Samuel Denton, Applied Compute", "duration": 1143, "upload_date": null, "channel": "AI Engineer", "source": "AI Engineer", "event_format": "In person" }, "analysis": { "video_id": "ZTA0GwpAUak", "title": "Bringing Continual Learning into Enterprises — Samuel Denton, Applied Compute", "one_liner": "Applied Compute's Sam Denton lays out a 2×2 of continual learning — offline vs online production traces crossed with offline vs online hints — and shows real distillation results in both corners: SWE-bench submit-tool calls rising 22%→60% with no test-pass regression, and customer-specific hyperlink formatting going 15%→~80%.", "summary": "Denton frames continual learning as two spectrums: how online the production trace is (a one-time batch of traces → a daily batch → a unified inference-and-training engine serving traffic) and where the hint that makes the teacher smarter than the on-policy student comes from (static priors vs. dynamically constructed from the rollout). Applied Compute does research across all four resulting quadrants but concentrates on quadrant one (offline hint + offline trace), which lets an enterprise improve 'for free today' from a dump of production traces, and quadrant four (online hint + online trace), which raises the ceiling continuously. He insists this must work without any golden answer, and backs the framing with two production-style results plus two implementation tricks: per-step hinting chosen by a judge, and relevance-mask self-distillation.", "key_points": [ "The distillation spectrum runs from a single one-time batch of production traces (where most enterprises are today), through a daily batch, to the 'holy grail' — a unified engine where a model serves a request, creates a trace, learns from it, updates, and serves the next request.", "A second, orthogonal axis is hinting: offline hints are derived from static data (known rubrics, behavior priors like a customer support agent too quick to give refunds, loss reports), while online hints are constructed dynamically from what the on-policy model actually did in its rollout.", "Crossing the two axes gives four quadrants; Applied Compute researches all four but focuses on Q1 (offline hint + offline trace, needs no replayable environment) and Q4 (online hint + online trace, the most scalable and highest-ceiling option).", "Denton's stated frustration with the distillation literature: it usually assumes a golden answer or a beautifully golden rubric per task, which enterprises don't have — the goal is continual learning without one.", "SWE-bench experiment (Q1): a Qwen 3.5 thinking model was taking up to 80 turns to submit; an offline hint ('you are near your 40-turn limit... finalize and verify your fix and then call this tool') lifted the task-complete tool call rate from about 22% to 60% while the test pass rate stayed flat or rose slightly.", "The surprising mechanism: because the rollout is conditioned on an off-policy production trace that never called the tool, the teacher never changes the tool-call tokens — it only bends the reasoning path toward calling the tool.", "Rolling out just one on-policy step from an offline trace (the 'cheat' of quadrant three) raised SWE-bench pass rate more than the fully offline setup, because the teacher eventually encourages the actual tool token.", "Hyperlink-formatting experiment (Q4): a customer-specific link format was far out of distribution; a formatting reward and SFT on correctly-formatted traces both degraded overall coding-agent performance, but online hints referencing the model's own prior rollout took correct formatting from ~15% to ~80% — offline hinting applied uniformly to every rollout climbed far less.", "Two tricks that make it work: per-step hinting, where a judge picks where in the rollout to inject the hint and distillation happens only on that next step (or a few steps forward) because the KL learning signal decays with distance from the hint; and relevance-mask self-distillation, using an LLM judge to select which teacher tokens to learn from, since teachers push irrelevant connector-word preferences." ], "takeaways": [ "Start where the enterprise already is: a one-time dump of production traces plus an offline hint for a specific target behavior can move that behavior measurably without a replayable environment or a golden answer.", "When the behavior you want is far out of distribution, don't reach for a reward term or SFT on good traces first — both regressed base capability here; construct the hint against the model's own prior rollout instead.", "Don't inject the hint at the start of a rollout and distill over the whole thing. Use a judge to pick the moment, then distill on that next step or the few steps after it — the learning signal falls off the further you get from the hint.", "Mask teacher tokens by relevance (LLM judge) so you learn the targeted behavior instead of the teacher's connector-word style, which is how you avoid catastrophic degradation while teaching OOD behavior.", "Measure the targeted behavior and the base-task regression separately, then report their intersection — Denton's three metrics were task-complete rate, test pass rate, and combined SWE-bench pass rate." ], "topics": [ "continual-learning", "distillation", "self-distillation", "rl", "agents", "post-training", "evals", "enterprise-ai" ], "tools": [ "Applied Compute", "Qwen 3.5 (thinking)", "SWE-bench", "AI Engineer" ], "quotes": [ { "text": "this is sort of the holy grail of continual learning where I have a model that's serving production traffic, it does a rollout, it creates a trace, we figure out how to learn from that trace, we update the model, and then we serve the next production request.", "at": "02:24", "url": "https://www.youtube.com/watch?v=ZTA0GwpAUak&t=144s" }, { "text": "We can improve for free today, and we can raise all ceilings tomorrow.", "at": "09:26", "url": "https://www.youtube.com/watch?v=ZTA0GwpAUak&t=566s" }, { "text": "A lot of distillation work is done assuming you have some kind of golden answer that you can distill into the model. And this is often not the case.", "at": "10:02", "url": "https://www.youtube.com/watch?v=ZTA0GwpAUak&t=602s" }, { "text": "The teacher doesn't force the tool call. It just starts to force the reasoning path towards the tool call without ever actually changing the tool call.", "at": "13:26", "url": "https://www.youtube.com/watch?v=ZTA0GwpAUak&t=806s" } ], "words": 4148 }, "summary_url": "/#ZTA0GwpAUak", "transcript": { "html": "/transcripts/ZTA0GwpAUak.html", "txt": "/transcripts/ZTA0GwpAUak.txt", "vtt": "/transcripts/ZTA0GwpAUak.vtt" } }