{
  "video": {
    "id": "Ti-BmGU1_mQ",
    "title": "The AI Bill Is Too Damn High: Tokenomics 101 — Eugene Cheah, Featherless AI",
    "duration": 3325,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": ""
  },
  "analysis": {
    "video_id": "Ti-BmGU1_mQ",
    "title": "The AI Bill Is Too Damn High: Tokenomics 101 — Eugene Cheah, Featherless AI",
    "one_liner": "Most AI token savings come from measuring first: give each department and use case its own API key, then right-size the model and the tokens-per-second speed to whether a person is actually waiting. Background agents and CI/CD should never run at 'ultra' speed.",
    "summary": "Eugene Cheah (Featherless AI, which sells inference) lays out four moves for cutting the token bill: measure, right-size, buy capacity, and check your processes. He aims this at organisations spending more than $10K a month and says smaller ones should focus on finding strong use cases instead. He splits workloads into two tracks with opposite economics: agentic work, which is long-context, cache-heavy and about 1% output, and process automation, which is short-context and a volume game. The fixes he describes are open or fine-tuned models, planner/implementer splits, one-shot classifiers, slower tokens-per-second targets and dedicated capacity sized to base load. He ends by warning that tokens are not work done, and that 'slop grenades' and review bottlenecks can wipe out the gains.",
    "key_points": [
      "Measure first: in 30-40% of the customers who contact Featherless, the whole organisation shares one API key. Split keys by department, team and use case. 'All your biggest ROI is just by measuring.'",
      "Case study: a team of 5 developers averaged 8 agents running continuously on Opus 4.8 via the API. That came to 116B tokens and $150K a month. They moved to a single-node GLM 5.3 instance for under $10K a month and saw no disruption.",
      "Process automation case: a telco cut inference from $250K to $7.6K a month. They moved off a closed model by splitting off traffic, finding open models that fit, and fine-tuning. AT&T fine-tuned 27B and 7B models for call-centre tiers, and Shopify fine-tuned for its proprietary storefront configuration language.",
      "Speed does not scale linearly with cost. His rule of thumb is 'every 2x of speed is 3x the cost and it compounds.' On one GLM 5.3 node, a single request gets 112 tok/s. Running 16 in parallel roughly halves per-request speed but gives about 10x the total output, which is about 9-10x cheaper. Most customers pick between 20-30 and 60 tok/s.",
      "Planner/implementer split: the best frontier model plans and reviews, while a cheap fast model (e.g. GLM 5.3 Flash, DeepSeek 4.1 Flash) does the token-heavy implementation. People won't switch models to save money, so make the cheaper model faster. Teams where both models ran at the same speed never switched while they had budget to spare.",
      "One-shot classifiers (JAF; Featherless's open-source Simple JAF) are commonly 10x cheaper and faster, but they can't give chat responses. A surprising share of process traffic is really classification.",
      "The open/closed gap is closing. Kimi K3 beat Opus on some UI benchmarks before Opus 5.5 came out. Step 1.5 got near-Opus quality on financial use cases despite being about 10x smaller. DeepSeek-V4/Flash is popular because it's easy to prompt. Today's best 27-32B models that run on a top-spec MacBook Pro already beat GPT-4.",
      "Biggest saving in capacity buying: a dedicated instance sized to your base load, with spikes spilling to public cloud. Featherless's largest customer runs 2 trillion tokens a day of an Opus-4.8-class private model on 512 AMD GPUs at about $0.03 per million tokens."
    ],
    "takeaways": [
      "Issue separate API keys per department, team and major use case before attempting any other optimisation. Background or CI/CD keys should be pinned to cheaper, slower tokens-per-second tiers.",
      "Give departments model labels such as 'fast', 'smart' or 'finance agent' rather than model names. You can then swap or A/B the underlying model without anyone noticing; he found teams still running Llama 2 because no one switched the default.",
      "For high-volume processes, build test cases and evals, search for the cheapest model that passes, then fine-tune (usually SFT). You can bootstrap the data from an expensive model where its terms allow it, and keep sending some shadow traffic to the frontier model to keep growing the dataset.",
      "If you use turn-based auto-routing, lock it to 2-3 models or cache hit rates collapse. He prefers task- or department-based routing.",
      "Prevent slop grenades: people own whatever they generate or hand over. Measure outcomes rather than usage, and fix review processes, for example with better testing and smaller modular merges, so agent output isn't thrown away and redone by hand."
    ],
    "topics": [
      "tokenomics",
      "inference cost",
      "open models",
      "fine-tuning",
      "model routing",
      "coding agents",
      "classifiers",
      "gpu capacity"
    ],
    "tools": [
      "Featherless AI",
      "Opus 4.8",
      "Opus 5.5",
      "GLM 5.3",
      "GLM 5.3 Flash",
      "DeepSeek-V4",
      "DeepSeek 4.1 Flash",
      "Kimi K3",
      "Step 1.5",
      "Gemini 5.3 Flash",
      "Fable",
      "Qwen 27B",
      "Llama 2",
      "GPT-4",
      "JAF",
      "Simple JAF",
      "Hugging Face",
      "Cerebras",
      "AMD MI215",
      "Google Gemini",
      "Claude Code",
      "Pi coding agent",
      "Devin",
      "Cloudflare",
      "AT&T",
      "Shopify",
      "Walmart",
      "Sunflower model",
      "Adaption AI"
    ],
    "quotes": [
      {
        "text": "don't pay for ultra on a background agent. That's just dumb.",
        "at": "04:44",
        "url": "https://www.youtube.com/watch?v=Ti-BmGU1_mQ&t=284s"
      },
      {
        "text": "nobody really switches their model to save the company money.",
        "at": "15:50",
        "url": "https://www.youtube.com/watch?v=Ti-BmGU1_mQ&t=950s"
      },
      {
        "text": "For every two x of speed, it's three x the cost and it compounds.",
        "at": "30:09",
        "url": "https://www.youtube.com/watch?v=Ti-BmGU1_mQ&t=1809s"
      },
      {
        "text": "Do not let slop grenades happen.",
        "at": "42:16",
        "url": "https://www.youtube.com/watch?v=Ti-BmGU1_mQ&t=2536s"
      }
    ],
    "words": 11430
  },
  "summary_url": "/#Ti-BmGU1_mQ",
  "transcript": {
    "html": "/transcripts/Ti-BmGU1_mQ.html",
    "txt": "/transcripts/Ti-BmGU1_mQ.txt",
    "vtt": "/transcripts/Ti-BmGU1_mQ.vtt"
  }
}