{
  "video": {
    "id": "Ok6QdZHCcI8",
    "title": "How Revolut Cut AI Coding Costs 90% With Self-Hosted Open Models - David Abitbol, Revolut",
    "duration": 707,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": ""
  },
  "analysis": {
    "video_id": "Ok6QdZHCcI8",
    "title": "How Revolut Cut AI Coding Costs 90% With Self-Hosted Open Models - David Abitbol, Revolut",
    "one_liner": "Revolut self-hosts GLM on 56 B200 GPUs for about 2,000 engineers, giving them unlimited coding tokens at a blended $0.30 per million. Its bill is 1/10 of what it paid Anthropic and OpenAI, and coding time improved 60% and lead time 70%.",
    "summary": "David Abitbol, who leads AI operations at Revolut, says frontier coding agents bought under strict budget limits had little measurable effect. Self-hosted open-weight models (GLM) changed that because engineers could use AI 'guilt-free' with unlimited tokens. He shares the serving metrics that matter, such as time to first token, throughput, GPU count and cost per token. He shows how tuning alone cut time to first token by 88% on a fixed fleet of 56 GPUs while demand rose 76%, and lists the serving parameters he tuned. He ends with the production setup, the agent harness and the compliance case for using Chinese open-weight models at an EU-regulated bank.",
    "key_points": [
      "Data from about 2,000 engineers: rolling out Gemini in January 2025 improved lead time by 10% and coding time by 20%. Claude Code ('Cloud Code' in the transcript), launched with strict budget limits because some engineers were spending $1k–$10k a month for little ROI, had little impact. Self-hosted GLM ('GLM 51' in the transcript) from April 2026 brought a 60% improvement in coding time and 70% in lead time.",
      "Lead time improved more than coding time, which suggests coding is not the main engineering bottleneck. With unlimited tokens, engineers also use AI for their inbox, Jira and Confluence instead of saving tokens for the hardest tasks or the end of the quarter.",
      "Serving GLM ('GLM-53' in the transcript, about 800B parameters) to 2,000 engineers takes about 56 B200s. The target is under 3 seconds to first token; above 5 seconds, complaints flood his Slack. About 200 tokens per second of interactivity is enough. After 5pm, when the GPUs are idle, engineers get 800 TPS.",
      "The blended cost is $0.30 per million tokens, and the invoice is 1/10 of what Revolut spent on Anthropic and OpenAI. Billing is flat, with no bill shock. Codex and Claude Code are still used, but only as exceptions.",
      "Tuning on the same 56 GPUs between April and August: requests per second rose from about 3 to over 7, and time to first token fell from over 10 seconds to about 1.6 seconds. That is an 88% performance improvement with 76% more demand. Simply loading weights onto GPUs gives 'terrible performance'.",
      "Parameters that matter: cap context at 256K even if 1M is supported; on Blackwell, use FP4 or lower, with NVFP4 ('VFP4' in the transcript) best (he says FP4 versus FP8 saves about 40% of GPU footprint with no precision loss); use TP8 instead of TP4 if TP4 leaves too little KV cache; keep cache hit above 90% for coding; and look at speculative decoding and sequential prefill.",
      "Production setup: GLM is text-only, so a local router sends image recognition to Gemini. A GLM vision 'flash' model ('Gnome 5G flash' in the transcript) is used by Revolut's UI designer. The endpoints are OpenAI-compatible with the Responses API and work with OpenCode and the Codex app.",
      "Compliance: using Chinese open weights on US-hosted GPUs means no data goes to China. Sandboxing with approval rules (for example, allow grep but not Gradle) shows the regulator there is a human in the loop rather than an autonomous agent. It also guards against rogue insiders and prompt injection. Other requirements are data residency in data-equivalency countries (US, EU, UK, Israel), zero data retention by default, and strong OpenTelemetry."
    ],
    "takeaways": [
      "Give engineers flat-cost, effectively unlimited tokens from a self-hosted open model instead of budget-capped frontier access. Keep frontier models such as Claude Code and Codex for exceptions.",
      "Measure lead time as well as coding time; the biggest gains may come from non-coding work like inbox, Jira and Confluence.",
      "Tune for time to first token (aim for under 3s; above 5s gets complaints) rather than raw throughput. Before adding GPUs, check context length (256K), quantization (NVFP4 on Blackwell), tensor parallelism versus KV cache, prefix cache hit rate (above 90%) and speculative decoding.",
      "If you choose a stronger text-only model, add a router that sends image tasks to a vision model. Expose an OpenAI-compatible Responses API endpoint so tools like OpenCode and Codex work unchanged.",
      "For regulators, run coding agents in a sandbox with explicit allow/deny command rules so a human stays in the loop. Pair that with zero data retention, data residency and OpenTelemetry logging for incident forensics."
    ],
    "topics": [
      "self-hosting",
      "open-weight models",
      "llm inference",
      "gpu optimization",
      "coding agents",
      "ai cost",
      "compliance",
      "developer productivity"
    ],
    "tools": [
      "Revolut",
      "GLM",
      "Gemini",
      "Claude Code",
      "Claude Sonnet 3.7",
      "Codex",
      "OpenCode",
      "Anthropic",
      "OpenAI",
      "NVIDIA B200",
      "NVIDIA Blackwell",
      "NVFP4",
      "OpenAI Responses API",
      "OpenTelemetry",
      "Jira",
      "Confluence",
      "Slack",
      "Stack Overflow",
      "Gradle"
    ],
    "quotes": [
      {
        "text": "what we find is the bottleneck for engineering is not necessarily coding.",
        "at": "03:37",
        "url": "https://www.youtube.com/watch?v=Ok6QdZHCcI8&t=217s"
      },
      {
        "text": "As soon as you exceed 5 seconds, my Slack inbox starts to blow up and so we know that's the limit.",
        "at": "04:50",
        "url": "https://www.youtube.com/watch?v=Ok6QdZHCcI8&t=290s"
      },
      {
        "text": "And so, our invoice is 1/10 of what we were spending on Anthropic and um OpenAI.",
        "at": "05:35",
        "url": "https://www.youtube.com/watch?v=Ok6QdZHCcI8&t=335s"
      },
      {
        "text": "long context is a scam.",
        "at": "07:15",
        "url": "https://www.youtube.com/watch?v=Ok6QdZHCcI8&t=435s"
      }
    ],
    "words": 2129
  },
  "summary_url": "/#Ok6QdZHCcI8",
  "transcript": {
    "html": "/transcripts/Ok6QdZHCcI8.html",
    "txt": "/transcripts/Ok6QdZHCcI8.txt",
    "vtt": "/transcripts/Ok6QdZHCcI8.vtt"
  }
}