{
  "video": {
    "id": "u6jJcIFDLE4",
    "title": "Why We Killed Our Multi-Agent Pipeline — Subbiah Sethuraman and Abhilash Asokan, ZS Associates",
    "duration": 900,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": "In person"
  },
  "analysis": {
    "video_id": "u6jJcIFDLE4",
    "title": "Why We Killed Our Multi-Agent Pipeline — Subbiah Sethuraman and Abhilash Asokan, ZS Associates",
    "one_liner": "ZS Associates rebuilt a five-agent pharma commercial-analytics pipeline into one reasoning agent plus a deterministic signal-detection pipeline and a knowledge graph used as a control plane, after the multi-agent version produced facts that were individually right but mutually incoherent.",
    "summary": "Subbiah Sethuraman and Abhilash Asokan describe mirroring the four steps a pharma commercial analyst performs — signal detection, source localization, driver attribution, synthesis — as one agent each under an orchestrator, and finding the output incoherent: the system correctly identified that a payer had moved a drug to a lower tier, then recommended sending more sales reps and forecast an improved outlook off that wrong action. They diagnose three causes — an LLM doing work that is purely statistical, context lost at every inter-agent handoff, and no shared domain model of pharma KPIs — and fix them by watching Claude Code work in an empty directory with only bash and the database. The rebuild pulls signal detection out into a deterministic pipeline that queues signals for the agent, consolidates judgment into one agent that may spawn sub-agents for scoped investigations, and adds a domain-expert-built knowledge graph the agent traverses hypothesis by hypothesis. The result reproduces in 20–30 minutes, over 50+ turns, an analysis an analyst took three or four weeks to produce.",
    "key_points": [
      "Original architecture mimicked the human analyst workflow one-to-one: a signal detection agent, a source localization agent, a driver attribution agent, a synthesis agent, and an orchestrator agent wiring them together.",
      "The failure case shown: prescriptions down 18% in a territory over 4 weeks, cause correctly identified as a payer moving the drug to a lower tier (making it expensive for patients), but the recommended action was 'send more sales reps to talk to doctors' — ignoring the insurance problem — so the outlook was wrong too. Every agent derived the right fact; none owned the end-to-end picture.",
      "Three diagnosed causes: (1) an LLM was deciding what counts as a signal, which statistical methods can do deterministically; (2) context degraded at every handoff — the synthesis agent never got the weighting of the payer coverage drop; (3) no shared business domain knowledge, so agents didn't understand metrics like TRX or how KPIs relate.",
      "Instead of redesigning topology, skills, tools or handoff schemas, they opened an empty directory, ran Claude Code with only bash and the database, gave it a real signal and watched what it did — every fix came out of that observation.",
      "Signal detection became a pure deterministic pipeline with statistical methods, guardrails, thresholds and prioritization, scanning each KPI for anomalies and trend breaks and pushing signals to a queue; the agent wakes up on a queued signal. Its job is to investigate, not identify.",
      "Judgment was consolidated into a single agent; parallelism was kept but distributed reasoning removed. Copying Claude Code's behaviour, they gave the agent a database query tool (it kept writing a function and querying the DB) and let it spawn sub-agents dynamically for focused investigations — e.g. rep activity in a region — which return results, never reasoning or judgment.",
      "Domain experts who do this commercial analytics work for clients built a knowledge graph of geographies, payers, accounts, brands and primary/secondary/tertiary KPIs, including which KPI drives which — used as a control plane that dictates what the agent may look into, which paths it can take and which hypotheses it can evaluate. Every edge is a hypothesis, and the agent cannot go outside the graph.",
      "The agent loop: start at an entity, read its graph neighborhood, derive hypotheses from the edges, go back to the raw data, check the actual numbers, reason over whether evidence supports or contradicts, and traverse further where supported — until hypotheses run out or the root cause is found.",
      "Reported outcome: 50+ turns and a lot of tokens, producing in 20–30 minutes what an analyst produced in three or four weeks."
    ],
    "takeaways": [
      "Don't encode the human org chart or workflow steps as your agent topology — 'let the architecture be derived' from what the model actually does, not from how analysts divide the work.",
      "Split every complex workflow into its deterministic and agentic parts, and keep agents out of the deterministic part: thresholds, anomaly detection and prioritization belong in code that runs before the agent starts.",
      "Keep reasoning in one agent that owns the problem end to end; it can delegate to sub-agents, tools and skills for investigation, but sub-agents return findings, not judgment.",
      "Treat a knowledge graph as a control plane rather than a lookup layer — let its edges define the hypothesis space the agent is allowed to explore, which bounds the combinatorics of where-and-why analysis and stops the agent inventing relationships not present in the data.",
      "Debug a broken agent system by running a capable coding agent on the bare problem (empty directory, bash, the database, a real input) and copying the operations it converges on into your architecture."
    ],
    "topics": [
      "multi-agent",
      "agent-architecture",
      "knowledge-graphs",
      "root-cause-analysis",
      "pharma-analytics",
      "determinism",
      "subagents",
      "context-engineering"
    ],
    "tools": [
      "Claude Code (transcribed as \"Cloud Code\")",
      "ZS Associates",
      "bash"
    ],
    "quotes": [
      {
        "text": "So, obviously it's not the LLM which failed, right? It's the way how we split the work, right? Because we tried mimicking the analyst behavior, and we did it.",
        "at": "04:44",
        "url": "https://www.youtube.com/watch?v=u6jJcIFDLE4&t=284s"
      },
      {
        "text": "The moment a signal comes to the queue, the agent wakes up. So, the agent's job is to investigate, not to identify.",
        "at": "07:49",
        "url": "https://www.youtube.com/watch?v=u6jJcIFDLE4&t=469s"
      },
      {
        "text": "So, the knowledge graph is not just something the agent looks up for data. It is a control plane for the agent.",
        "at": "10:56",
        "url": "https://www.youtube.com/watch?v=u6jJcIFDLE4&t=656s"
      },
      {
        "text": "So, once you build this, so maybe like after like 50 plus turns, a whole lot of tokens, it's able to produce something an analyst was able to produce maybe in three or four weeks in like maybe 20 30 minutes.",
        "at": "13:28",
        "url": "https://www.youtube.com/watch?v=u6jJcIFDLE4&t=808s"
      }
    ],
    "words": 3159
  },
  "summary_url": "/#u6jJcIFDLE4",
  "transcript": {
    "html": "/transcripts/u6jJcIFDLE4.html",
    "txt": "/transcripts/u6jJcIFDLE4.txt",
    "vtt": "/transcripts/u6jJcIFDLE4.vtt"
  }
}