{
  "video": {
    "id": "TyPpSRXGhbc",
    "title": "Small Models, Big Results: Training a Finance Agent for Under $500 — Charles Dickens, Snorkel AI",
    "duration": 1036,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": ""
  },
  "analysis": {
    "video_id": "TyPpSRXGhbc",
    "title": "Small Models, Big Results: Training a Finance Agent for Under $500 — Charles Dickens, Snorkel AI",
    "one_liner": "Snorkel AI and UC Berkeley's Sky Computing Lab used RL (GRPO with a binary LLM-judge reward) on expert-validated data built from 10-K filings to train Qwen3-4B. It reached almost 60% pass@1 on financial tasks against 51% for Qwen3-235B, for under $500.",
    "summary": "Charles argues that for enterprise AI workloads, reliability and specialization can matter more than raw parameter count, provided you have high-quality data that matches the target domain and task distribution. He walks through how they built FinQA: SQL tables extracted from SEC 10-K filings, question-answer pairs generated from an expert-designed taxonomy, and three layers of verification. He then describes RL training of a 4B Qwen model in the rLLM framework. Ablations showed that the simplest choices won: training only on single-table data and using a plain binary reward beat curricula, mixed data and expert-designed rubric rewards. The tool-use skills learned this way transferred to multi-table reasoning and to general tool calling on BFCL.",
    "key_points": [
      "RLLM FinQA 4B (Qwen3-4B base) reached almost 60% pass@1 against 51% for Qwen3-235B, on 290 expert-curated held-out samples. Training more than doubled the base model's accuracy.",
      "FinQA pipeline: 10-K reports pulled from SEC EDGAR, Qwen3-30B used to extract about 6,900 SQL tables, one QA pair generated per table using a question taxonomy built with financial experts, plus metadata for lineage. Verification has three layers: programmatic consistency checks against the metadata (table and column names), automated review by independent agents, and manual expert review.",
      "Splits: about 4,000 train, 500 validation and 290 benchmark, with no company appearing in more than one split. The FinQA reasoning variant (questions spanning 2-5 tables) has about 1,000 train, 120 validation and 80 benchmark.",
      "Recurring failure modes, even in frontier models: schema hallucination (assuming tables or columns exist, possibly from pre-training), context flooding (poorly planned queries such as SELECT * that can overwhelm the context window), and poor error recovery (the 235B model repeated failed strategies instead of adapting to error messages). The same failure modes showed up in insurance underwriting work presented at CAIS.",
      "Training setup: rLLM framework, a ReAct agent with tools over about 7,000 tables, a binary correctness reward from GPT-5 nano as a reference-based LLM judge, GRPO, about 1,000 concurrent environments and 8×H100. Total cost was under $500: roughly $420 compute and $40 for the judge API.",
      "Generalization: with no explicit multi-table training, the model still improved on multi-table FinQA reasoning. BFCL overall accuracy rose slightly, with minor gains on multi-turn and memory, so specialization did not erode general tool use.",
      "Ablations: training only on simple single-table data beat both mixed single+multi data and a single-to-multi curriculum. A binary reward beat richer signals, including an expert-designed rubric with multiple weighted components. In his words, 'The bottleneck was never reasoning depth, it was just tool use.'",
      "Proposed axes for evaluating agents: environment complexity, autonomy horizon and output complexity. He calls output complexity under-explored because most benchmarks focus on text artifacts."
    ],
    "takeaways": [
      "For narrow enterprise workflows, consider RL-tuning a small model on high-quality domain data before reaching for a larger frontier model; the deployment economics can be much better.",
      "Find out where agents actually fail (schema hallucination, context flooding, error recovery) and train tool-use discipline on simple tasks first; the speaker found that composed multi-step skills followed.",
      "Start with a simple binary correctness reward from a reference-based LLM judge before investing in rubric or intermediate-step reward engineering.",
      "Build training data with layered verification (programmatic checks against metadata, independent agent review, expert review), and split by entity (here, company) to avoid leakage into the benchmark.",
      "After specializing, check general capability on a broad benchmark such as BFCL to confirm nothing regressed."
    ],
    "topics": [
      "reinforcement learning",
      "small models",
      "agents",
      "tool use",
      "finance",
      "synthetic data",
      "evals",
      "reward design"
    ],
    "tools": [
      "Snorkel AI",
      "UC Berkeley Sky Computing Lab",
      "rLLM",
      "Qwen3-4B",
      "Qwen3-235B",
      "Qwen3-30B",
      "GPT-5 nano",
      "GRPO",
      "FinQA",
      "BFCL",
      "SEC EDGAR",
      "NVIDIA H100",
      "GitHub",
      "Hugging Face"
    ],
    "quotes": [
      {
        "text": "reliability and specialization uh can actually outweigh raw parameter scale.",
        "at": "00:41",
        "url": "https://www.youtube.com/watch?v=TyPpSRXGhbc&t=41s"
      },
      {
        "text": "You know, you wouldn't call Terence Tao for a tax audit",
        "at": "04:44",
        "url": "https://www.youtube.com/watch?v=TyPpSRXGhbc&t=284s"
      },
      {
        "text": "The bottleneck was never reasoning depth, it was just tool use",
        "at": "13:00",
        "url": "https://www.youtube.com/watch?v=TyPpSRXGhbc&t=780s"
      },
      {
        "text": "simpler was better. A single binary reward signal",
        "at": "13:55",
        "url": "https://www.youtube.com/watch?v=TyPpSRXGhbc&t=835s"
      }
    ],
    "words": 2855
  },
  "summary_url": "/#TyPpSRXGhbc",
  "transcript": {
    "html": "/transcripts/TyPpSRXGhbc.html",
    "txt": "/transcripts/TyPpSRXGhbc.txt",
    "vtt": "/transcripts/TyPpSRXGhbc.vtt"
  }
}