{
  "video": {
    "id": "Yphdry8ttAQ",
    "title": "Trading Desks to Clinical Trials: Parallels in Applied Vertical AI — Ayush Bhardwaj, Allos AI",
    "duration": 1202,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer"
  },
  "analysis": {
    "video_id": "Yphdry8ttAQ",
    "title": "Trading Desks to Clinical Trials: Parallels in Applied Vertical AI — Ayush Bhardwaj, Allos AI",
    "one_liner": "An applied-AI engineer who moved from a hedge fund to a pharma-tech startup argues that building vertical AI is the same seven-step recipe in both industries, and that the only real moat is proprietary domain data and a hired domain expert — not the model, infra or ecosystem everyone else can buy.",
    "summary": "Ayush Bhardwaj defines \"applied vertical AI\" as applied AI built for one specific industry to simulate a person's job in it, and reports that moving from a hedge fund to Allos AI (pharma) changed nothing about the core job. He lays out seven steps — formulate a narrow problem, identify proprietary data, write the prompt as a model of the expert's process, add observability, then *don't* iterate yet: hire the user, build a learning loop, ship. The hard part is that engineers can't judge whether a trade thesis or drug candidate output is good, LLM-as-a-judge fails because the model just \"jargons its way out\", and the data that would teach it (trade theses, failed experiments) is deliberately gatekept, so neither OpenAI nor Anthropic has it. His conclusion: models, infra and ecosystem are commodity; domain expertise and non-public data are the moat, and finance and pharma will kill anything that doesn't pay for itself immediately.",
    "key_points": [
      "\"Are people putting agents into production?\" is the wrong question — everyone is; the question is whether they work, make or save money, and justify ROI end-to-end. At both of his employers the agent either saved or made money.",
      "Step one is a narrow task, not a broad one: not \"fetch me top three market opportunities\", but pick US equities → pick IT → rank stocks on capital expenditure or AI investment. \"There was no tax on building more AI agents\", so build n of them instead of one that does everything.",
      "Proprietary data is the differentiator because everyone has news, JP Morgan/Morgan Stanley sell-side reports, arXiv and PubChem. In finance the proprietary asset is the trade thesis (what worked and why); in pharma it's failed-experiment data. Three years of unstructured internal data can be structured by an LLM workflow overnight.",
      "The first four steps (problem, data, prompt, observability) fit on one screen and a 10x engineer does them in minutes — which is exactly why they're not the moat.",
      "Vertical AI projects quietly die at the iteration step: an engineer can instantly tell that one coding model is worse than another because they've been trained for it, but has no mental model to judge a trade thesis or a drug candidate.",
      "LLM-as-a-judge was \"a really, really stupid mistake\" — it predicts the next probable word and jargons its way out; it doesn't understand what alpha means. RL from verifiable rewards works for math and code because there are answer keys; these fields have none, and errors compound.",
      "The data was never there by design: institutional managers holding over $100M in qualifying US equities must file long holdings quarterly, and returns drop once others reverse-engineer them; in pharma, disclosure of every clinical trial pass or fail is legally required but ~30% of firms never do it, and in 2026 the FDA had to publicly remind about 2,000 sponsors. You also can't buy the annotation — a trader won't do it for $100/hour, and there are NDAs.",
      "Ladder of improvement methods: supervised fine-tuning, RLHF (the current golden standard for edge), rubrics-as-reward (\"RL from AI feedback\", risks an echo chamber), and error analysis — reading the observability logs, touching no weights, cheapest and highest ROI. Fine-tuning has recurring cost: fine-tune GLM 5.2 and you'll have to redo it when DeepSeek or Alibaba Cloud ships the next model.",
      "He disputes the Stanford AI Index stat that 89% of enterprise AI agents never reach production: they all reach production, they just fail to work or justify their cost. In finance and pharma, anything that doesn't make money instantly is \"shown the door\".",
      "Not human-in-the-loop but AI-in-the-loop: the expert still does the work and makes the call — AI hands a trader five candidate trade theses, or narrows drug candidates — it just cuts the expert's time massively. Models do correlation, not causation; per LeCun they're text statistics, not real-world models."
    ],
    "takeaways": [
      "Decompose the job into pointed, narrow agent tasks modelled on how a specific expert would work through it, rather than one agent asked to do everything.",
      "Hire the user — the trader, the senior scientist — before you try to iterate. At the pharma startup, a bunch of young engineers hiring an experienced scientist changed the trajectory of the tools and made big pharma buyers respond, because the tools spoke their language instead of jargonish LLM language.",
      "Don't reach for LLM-as-a-judge in a domain where you can't verify the output yourself; put the domain expert in a learning loop where they refine prompts, pick which sources are reliable, decompose the problem, and judge — that loop generates the dataset of what works and what doesn't.",
      "Start improvement with error analysis over your observability traces (no weights touched, highest ROI) and only climb toward SFT/RLHF when that stops paying.",
      "Build the moat on curated proprietary data and domain expertise; treat models, infra and vendor tooling as commodity anyone can buy for a subscription."
    ],
    "topics": [
      "vertical ai",
      "agents",
      "evals",
      "domain expertise",
      "proprietary data",
      "observability",
      "rlhf",
      "fintech"
    ],
    "tools": [
      "Allos AI",
      "Google Translate",
      "ChatGPT",
      "Claude",
      "OpenAI",
      "Anthropic",
      "Sonnet 5",
      "Fable 5",
      "GLM 5.2",
      "DeepSeek",
      "Alibaba Cloud",
      "arXiv",
      "PubChem",
      "JP Morgan",
      "Morgan Stanley",
      "Stanford AI Index",
      "FDA"
    ],
    "quotes": [
      {
        "text": "The question to ask is whether they actually work, whether they actually make or save money, whether they justify their ROI.",
        "at": "03:23",
        "url": "https://www.youtube.com/watch?v=Yphdry8ttAQ&t=203s"
      },
      {
        "text": "I thought I could LLM as a judge my way out of it.",
        "at": "09:09",
        "url": "https://www.youtube.com/watch?v=Yphdry8ttAQ&t=549s"
      },
      {
        "text": "you hire the person who you want to sell it to cuz there is, to be honest, no other way around. I have tried a lot of stuff. You just need to hire the user.",
        "at": "11:55",
        "url": "https://www.youtube.com/watch?v=Yphdry8ttAQ&t=715s"
      },
      {
        "text": "Model infra ecosystem, everyone selling you tons of stuff at this conference is just commodity.",
        "at": "19:09",
        "url": "https://www.youtube.com/watch?v=Yphdry8ttAQ&t=1149s"
      }
    ],
    "words": 4312
  },
  "summary_url": "/#Yphdry8ttAQ",
  "transcript": {
    "html": "/transcripts/Yphdry8ttAQ.html",
    "txt": "/transcripts/Yphdry8ttAQ.txt",
    "vtt": "/transcripts/Yphdry8ttAQ.vtt"
  }
}