{
  "video": {
    "id": "-561cZmir5Q",
    "title": "From Tokens to Cells: Foundation Models for Single-Cell Biology - Akram Baharlouei, Altos Labs",
    "duration": 1017,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": "Online"
  },
  "analysis": {
    "video_id": "-561cZmir5Q",
    "title": "From Tokens to Cells: Foundation Models for Single-Cell Biology - Akram Baharlouei, Altos Labs",
    "one_liner": "An Altos Labs ML engineer argues that single-cell foundation models are underperforming their compute budget — sometimes losing to simple linear models — because RNA-seq data is noisy, bursty snapshots of a movie, and that distribution-matching flow-matching models like PrimeFlow beat the compress-to-a-latent-vector transformer recipe.",
    "summary": "Akram Baharlouei frames single-cell biology as an engineering problem: modelling cells is the path to virtual tissue, virtual human and digital twins, and to fixing a drug pipeline that takes up to 10 years, costs billions and whose yearly output of new drugs is declining despite AI advances. The talk walks through the measurement modalities (genome, RNA-seq, proteomics, morphology), explains why RNA-seq dominates foundation-model training purely because PCR makes it scalable, and why that data is a poor training substrate — two identical cells don't read the same, changes happen in bursts, and each sample is a snapshot from a whole movie. Benchmarking work (two NeurIPS papers) found that expensive transformer models like scGPT and Geneformer don't match the gains seen in language and imaging, and that simple linear models are sometimes on par or better, because compressing a cell into a latent vector loses information. The conclusion: flow-matching models that predict the distribution — the team's PrimeFlow — do better, and progress needs data quality and measurement realism, not just more cells.",
    "key_points": [
      "Yamanaka's 2006 discovery of four transcription factors reprogrammed an old skin cell back to an embryonic-stem-cell-like state, won the Nobel in 2012, and in 2026 the first reprogramming medicine (OSK) is going into human testing — roughly 20 years end to end. Partial reprogramming aims to change only the cell's age, not its type.",
      "Drug development runs opposite to Moore's law: the number of drugs developed each year is declining, acceptance rate is ~5% or less, and the full pipeline takes up to 10 years and can cost billions. Speeding up only R&D (e.g. protein design) saves a few years but doesn't fix the whole pipeline.",
      "Modality trade-offs: DNA is nearly identical across cells; RNA-seq/transcriptomics gives a ~20,000-gene expression profile per cell; proteomics measures what arguably matters most (proteins do the functions) but is very low throughput; morphology/imaging adds shape and spatial position within tissue.",
      "RNA-seq dominates foundation-model training only because PCR makes it easy to scale — datasets run from tens of millions of cells to a reported 500 million, with a 1-billion-cell project underway.",
      "The data is a bad fit for naive scaling: two identical cells don't read the same, cell-cycle changes happen in bursts rather than continuously, sequencing captures a snapshot from a whole movie, and different labs and machines add technical batch variation.",
      "Transformer models (scGPT, Geneformer) treat a cell as a sentence and genes as tokens, BERT-style masking gene counts and predicting them with bidirectional attention; downstream tasks include cell-type prediction and perturbation response modelling.",
      "Two NeurIPS benchmarking papers last year — one on multimodal imaging + RNA-seq, one on perturbation response — found these expensive models don't perform as well as equivalents in language and imaging; simple linear models are sometimes on par or better, because compressing to a latent vector doesn't preserve the information.",
      "PrimeFlow (on arXiv) uses flow matching from Gaussian noise to match the data distribution rather than compress it; in the shown results it tracks the ground-truth distribution (green dots) while a CPA autoencoder-based baseline collapses to predicting the mean, with MMD scores reported."
    ],
    "takeaways": [
      "Benchmark single-cell foundation models against simple linear baselines before paying for large-scale transformer training — in published benchmarking the cheap baseline was sometimes on par or better.",
      "Prefer distribution-matching objectives (flow matching, as in PrimeFlow) over autoregressive/autoencoder approaches that compress cells into a latent vector and end up predicting the mean.",
      "Treat data quality and measurement realism as first-class work alongside scale — a billion noisy, burst-sampled, batch-varying snapshots won't produce generalization on its own.",
      "Don't assume RNA-seq is the right modality just because it scales; proteomics and morphology measure things RNA-seq can't, and understanding biology needs technological advances in those dimensions too.",
      "When evaluating AI's impact on drug development, look at the whole pipeline (preclinical, clinical, approval), not just the R&D stage where models already help."
    ],
    "topics": [
      "foundation models",
      "single-cell biology",
      "flow matching",
      "transformers",
      "benchmarking",
      "drug discovery",
      "multimodal",
      "data quality"
    ],
    "tools": [
      "Altos Labs",
      "scGPT",
      "Geneformer (transcribed as \"Informer\")",
      "PrimeFlow",
      "CPA (transcribed as \"CPADR\")",
      "scGeneScope",
      "BERT",
      "Human Cell Atlas",
      "NeurIPS",
      "arXiv",
      "OSK reprogramming medicine"
    ],
    "quotes": [
      {
        "text": "We're taking a snapshots from from a whole movie.",
        "at": "09:29",
        "url": "https://www.youtube.com/watch?v=-561cZmir5Q&t=569s"
      },
      {
        "text": "simple linear models are on par or like sometimes even outperforming this, uh, these, uh, models that, you know, it's like, you know, complex models with a lot of compute is being used to train those.",
        "at": "12:46",
        "url": "https://www.youtube.com/watch?v=-561cZmir5Q&t=766s"
      },
      {
        "text": "the number of drugs that developed each year is kind of declining, which is surprising with all the advances in technology, in AI.",
        "at": "04:36",
        "url": "https://www.youtube.com/watch?v=-561cZmir5Q&t=276s"
      },
      {
        "text": "for now it seems flow matching models they're doing better for single cell data trying to match to the distribution.",
        "at": "16:11",
        "url": "https://www.youtube.com/watch?v=-561cZmir5Q&t=971s"
      }
    ],
    "words": 2924
  },
  "summary_url": "/#-561cZmir5Q",
  "transcript": {
    "html": "/transcripts/-561cZmir5Q.html",
    "txt": "/transcripts/-561cZmir5Q.txt",
    "vtt": "/transcripts/-561cZmir5Q.vtt"
  }
}