{
  "video": {
    "id": "-tviRdpmHvs",
    "title": "Training Krea 2: What matters in generative model training — Sangwu Lee, Krea.ai",
    "duration": 1306,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": "In person"
  },
  "analysis": {
    "video_id": "-tviRdpmHvs",
    "title": "Training Krea 2: What matters in generative model training — Sangwu Lee, Krea.ai",
    "one_liner": "Krea's Sangwu Lee walks through how Krea 2 (whose medium variant they open-sourced) was trained, arguing that after you lock in a diffusion transformer architecture, data curation — 30–40 in-house classifiers, aggressive removal of AI-generated images, and captioning that doesn't drop details — is what actually determines model quality and stylistic diversity.",
    "summary": "Lee contrasts Krea's goal — fast generation with stylistic diversity so creative studios can explore visual ideas — against big-lab models like ChatGPT and Nano Banana Pro, which take a minute or two per image and buy reliability by mode-collapsing onto the most average output. He then details the training pipeline: latent diffusion with a DiT, a captioning pass (OCR → metadata → VLM detailed caption → rewrite), hash- then embedding-based dedup over 2–10 billion images, VLM-designed filters distilled into cheap SigLIP classifiers, sparse autoencoders used as an unsupervised tagging system, and Wikipedia PageRank to check world-knowledge coverage. The training itself is explicitly LLM-inspired: 256→1k progressive-resolution pre-training, mid-training, SFT, preference optimization, GRPO-style RL against reward servers, plus a small prompt-expander LM. He refuses distillation from other image models on both quality and pride grounds, and is next working on multi-expert on-policy distillation into a single student.",
    "key_points": [
      "Production models from big labs (ChatGPT, Nano Banana Pro) take up to a minute or two and are very reliable, but got there by significantly mode-collapsing: 'burning skull' returns consistent, flawless, near-identical outputs with barely any diversity. Krea instead optimizes for fast generation plus knobs, for users who don't yet know what they want.",
      "Krea 2 Medium is the open-sourced variant of their closed-source model; stylistic diversity was the explicit focus, which meant *not* over-sampling on standard aesthetic/image-quality scores — some people want the low-resolution CRT-video look.",
      "Bad-data taxonomy they filter on: duplicated samples and over-represented concepts (dedup + clustering-based rebalancing), images VLM captioners consistently mis-caption, text-dense images at low resolution (2000 characters on 256×256 is unlearnable), and AI-generated images.",
      "The canonical mis-caption example: a framed painting on a white wall. The captioner says 'a painting of X' but never mentions the frame or wall, so the model later renders every painting hanging on a white wall. That data got filtered or under-sampled rather than fixed.",
      "Captioning pipeline: run OCR first (text rendering matters), attach optional metadata such as a famous person's identity, then a second VLM pass for a detailed caption, then rewrite into JSON prompts or whatever format feeds the model.",
      "Dedup is staged for cost: pHash/MD5 across the full 2–10 billion image corpus first, then embedding methods (SSCD, SigLIP) for semantic/near-duplicate removal once the set is smaller. Filters follow the same pattern — prompt a large VLM to judge 'is this an AI image', then distill that into a SigLIP-sized classifier cheap enough to run over a billion images (the same trick EssentialWeb used for text, distilling to ~500M params).",
      "Sparse autoencoders trained on a vision model give an off-the-shelf unsupervised tagging system — features fire for 'horse', 'black and white', 'blurry' — which they used to filter signatures, watermarks and border artifacts.",
      "World knowledge coverage came from the original CLIP paper's trick: PageRank every Wikipedia concept, take the top 90th percentile as things the model should know, then plain-text or embedding search the dataset to confirm coverage. (Incidental find: there is a Barack Obama the horse.)",
      "Training is LLM-shaped end to end: 256→1k progressive resolution pre-training (semantics low-res, structure and detail high-res), mid-training and SFT on curated illustration/graphic design/photography/cinematic data to mold the distribution, preference optimization on pairs, GRPO-inspired RL with reward servers targeting text rendering and anatomy, and a small prompt-expander LM because long detailed prompts sit closer to the training distribution.",
      "Architectural observations: the stack has inverted into an autoregressive decoder (prompt expander) feeding a diffusion encoder, which he notes resembles DALL·E 2's conditioning pipeline. He wants to delete the VAE and text encoders and train a single clean transformer, and sees richer textual conditioning — bounding boxes (as Ideogram and Reve are exploring) and 2017-era scene graphs from Fei-Fei Li's lab — as the exciting direction now that VLMs are strong enough to produce them."
    ],
    "takeaways": [
      "Do not train on AI-generated images even though it is a shortcut to a good model — synthetic data is 'sticky', and you end up with a recognizably ChatGPT or Nano Banana aesthetic instead of your own.",
      "Filter on what your captioner reliably describes, not just on image quality. If a VLM consistently omits an attribute of a class of images, that whole class is bad data and should be dropped or under-sampled.",
      "Stage expensive filtering: cheap hashes over billions, embeddings over millions, and distill any large-VLM judgement into a small CLIP/SigLIP-sized classifier before running it over a pre-training corpus.",
      "Lock the architecture early and spend the effort on data — code changes easily, data is 'eternal' and stays valuable regardless of the next training paradigm.",
      "Steal from LLM research deliberately: the pre-train → mid-train → SFT → preference optimization → RL recipe transfers to diffusion, and so do the kernels and literature. Prefer methods with few hyperparameters to tune, and invest in infrastructure that lets you iterate fast."
    ],
    "topics": [
      "image-generation",
      "diffusion-models",
      "data-curation",
      "training-pipelines",
      "open-source-models",
      "captioning",
      "reinforcement-learning",
      "interpretability"
    ],
    "tools": [
      "Krea",
      "Krea 2",
      "Krea 2 Medium",
      "ChatGPT",
      "Nano Banana Pro",
      "CLIP",
      "SigLIP",
      "SSCD",
      "pHash",
      "MD5",
      "EssentialWeb",
      "Wikipedia",
      "Ideogram",
      "Reve",
      "DALL·E 2",
      "GRPO"
    ],
    "quotes": [
      {
        "text": "the easiest and most reliable way to like render a person is render the most boring average person that exists and then like put it in a center frame",
        "at": "02:28",
        "url": "https://www.youtube.com/watch?v=-tviRdpmHvs&t=148s"
      },
      {
        "text": "synthetic data is like so sticky to the model that once you like start training on AI image data sure your model is good but you kind of lose the point",
        "at": "07:43",
        "url": "https://www.youtube.com/watch?v=-tviRdpmHvs&t=463s"
      },
      {
        "text": "you know as a researcher it also slightly hurts my ego if all I'm doing is distillation",
        "at": "08:10",
        "url": "https://www.youtube.com/watch?v=-tviRdpmHvs&t=490s"
      },
      {
        "text": "data is like eternal like you can if you have good data that's going to be valuable no matter what the hot new training paradigm is",
        "at": "18:43",
        "url": "https://www.youtube.com/watch?v=-tviRdpmHvs&t=1123s"
      }
    ],
    "words": 4655
  },
  "summary_url": "/#-tviRdpmHvs",
  "transcript": {
    "html": "/transcripts/-tviRdpmHvs.html",
    "txt": "/transcripts/-tviRdpmHvs.txt",
    "vtt": "/transcripts/-tviRdpmHvs.vtt"
  }
}