{ "video": { "id": "OA-Mc60Rboo", "title": "Beating RL With Reflection: GEPA and Optimize Anything — Lakshya A. Agrawal, GEPA", "duration": 1288, "upload_date": null, "channel": "AI Engineer", "source": "AI Engineer", "event_format": "" }, "analysis": { "video_id": "OA-Mc60Rboo", "title": "Beating RL With Reflection: GEPA and Optimize Anything — Lakshya A. Agrawal, GEPA", "one_liner": "GEPA swaps RL's single scalar reward for LLM reflection on full rollout traces plus Pareto-pool candidate selection. With one reflection round on three examples it got twice the gains GRPO got after 25,000 rollouts, and its successor, optimize_anything, extends the same loop to any text artifact you can score.", "summary": "Lakshya Agrawal argues that most teams are limited by sample efficiency: they lack domain data, and agentic rollouts are slow or expensive. RL with verified rewards throws away almost everything in a trajectory (chains of thought, tool calls, error messages) and keeps only an O(1) score. GEPA instead has an LLM or agent reflect on whole traces and rewrite prompts, since one natural-language edit can change behavior as much as thousands of small gradient steps. It keeps a Pareto pool of candidates rather than only the best one. 'Optimize anything' generalizes this to code, agent harnesses, skills, scheduling policies and numeric problems, driven by an evaluator that returns a score plus 'actionable side information'.", "key_points": [ "GEPA vs GRPO on Qwen3 8B optimizing itself with no external teacher: one reflection round on 3 data points gave 2x the gains GRPO reached after 25,000 rollouts, and a few more steps doubled that gap again.", "GEPA's prompts are detailed problem specifications (how to read the input, the purpose of this pipeline stage, lessons from the data), not model tricks like 'my grandmother will be angry'. It automates weeks of manual prompt tweaking in about 30–60 minutes.", "On AMD's new NPU XDNA2, where the API has almost no public documentation, GEPA took an existing agent from 4.25% to 30.52% (7x) with no other changes. In one step it learned to 'avoid including ADF.h', AMD's own library that doesn't work on that hardware generation.", "The Pareto pool keeps every candidate that wins on even one training example. A plain LLM-in-a-loop keeps only the best and gets stuck in local optima. Across 4 benchmarks, more than half of GEPA's gains come from this, and it gets about 2x the gains of the model-in-a-loop.", "optimize_anything: supply problems plus an evaluator returning a score and an open-ended dictionary of side information (compiler errors, profiler output, expert feedback, docs). Modes are single problem, multitask search, and generalization.", "A 4-line chain-of-thought program grew into a 6-step ARC-AGI agent (rule induction, code synthesis, execution tracing, debugging) in 16 reflection rounds, taking Gemini Flash from 32.5% to 89.5%. A discovered 2-step agent raised GPT-4.1 nano on MATH-500 by 20%.", "GSkill learned repo skills on mini-SWE-agent with GPT-5 mini, raising Go-repo issue resolution from 24% to 93%. The same skills moved Claude Sonnet 4.5 to 100% resolution and cut resolution time about 50%.", "Other results: GPT-4.1 mini beat GPT-4.1 on math; cloud scheduling costs about 40% below expert heuristics; OCR error rates about 35% lower (externally validated); Databricks tuned GPT-OSS 120B to outperform Claude Opus at 90x lower cost." ], "takeaways": [ "Have your evaluator return rich, domain-specific 'actionable side information' (error messages, traces, profiler output, expert feedback) alongside the score.", "When data or rollouts are scarce or expensive, try reflective text-space optimization of prompts, harnesses or skills before RL or SFT.", "Keep a Pareto pool of candidates rather than looping an LLM on only the current best prompt, to avoid local optima.", "For subjective tasks: have a human give detailed annotations on about 50 production traces, use GEPA to optimize an LLM-as-judge prompt, then use that judge to optimize the agent, creating a data flywheel.", "Optimize skills and prompts cheaply on a small model, then transfer them to frontier models. Better models benefit more from precise instructions." ], "topics": [ "prompt-optimization", "reflective-optimization", "reinforcement-learning", "agents", "sample-efficiency", "agent-skills", "evals", "evolutionary-search" ], "tools": [ "GEPA", "optimize_anything", "GSkill", "GRPO", "Qwen3 8B", "GPT-4.1", "GPT-4.1 mini", "GPT-4.1 nano", "GPT-4", "GPT-5 mini", "Claude Opus 4.6", "Claude Sonnet 4.5", "Claude Code", "Codex", "Gemini Flash", "GPT-OSS 120B", "mini-SWE-agent", "AMD NPU XDNA2", "ADF.h", "Optuna", "ARC-AGI", "MATH-500", "Snorkel", "Databricks", "Dropbox", "Shopify", "OpenAI" ], "quotes": [ { "text": "we only learned an O of one score and propagated that via gradient descent", "at": "02:35", "url": "https://www.youtube.com/watch?v=OA-Mc60Rboo&t=155s" }, { "text": "So if you can write it as text and score it, JPA can optimize it.", "at": "11:00", "url": "https://www.youtube.com/watch?v=OA-Mc60Rboo&t=660s" }, { "text": "as models get better they will get better at instruction following and the more precise instruction about your task that you have to give to a very smart model the better that model will be", "at": "18:45", "url": "https://www.youtube.com/watch?v=OA-Mc60Rboo&t=1125s" }, { "text": "don't be afraid to optimize in the tech space and many problems can be framed as optimization", "at": "20:51", "url": "https://www.youtube.com/watch?v=OA-Mc60Rboo&t=1251s" } ], "words": 4114 }, "summary_url": "/#OA-Mc60Rboo", "transcript": { "html": "/transcripts/OA-Mc60Rboo.html", "txt": "/transcripts/OA-Mc60Rboo.txt", "vtt": "/transcripts/OA-Mc60Rboo.vtt" } }