{
  "video": {
    "id": "DrTdD-ttjCY",
    "title": "Building GTM AI Agents: Lessons from Deploying to 6,000 Users — Sait Izmit, Snowflake",
    "duration": 1239,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer"
  },
  "analysis": {
    "video_id": "DrTdD-ttjCY",
    "title": "Building GTM AI Agents: Lessons from Deploying to 6,000 Users — Sait Izmit, Snowflake",
    "one_liner": "Snowflake's internal go-to-market agent has answered over 1 million questions for 6,000 sellers, and the speaker argues its success came from choosing quality over coverage, investing heavily in change management, and constantly re-architecting rather than waiting for the perfect stack.",
    "summary": "Sait Izmit runs Snowflake's internal AI tools for sales, where a go-to-market assistant launched in September last year has answered 1.2 million questions and now handles ~40,000 a week across 6,000 users. He argues that with non-deterministic systems, user trust is earned in the first five questions and lost overnight, so the team runs 'quality is P-minus-one': answer 50 questions at 95% rather than 100 at 70%, launch in phases (pilot → 10% beta of 600 people → GA), and only expand coverage after trust exists. The rest of the talk covers the failure modes that follow launch — activation and change management, the 'collapsing wow factor' as your innovation becomes the baseline, and the need to keep re-architecting — plus using LLM classification of chat logs as the feedback loop that produces the hockey-stick.",
    "key_points": [
      "The assistant launched September last year, has answered 1.2 million questions total, ~40,000 questions a week, serving 6,000 go-to-market users; it is built on Snowflake Cortex/Co-work (formerly Snowflake Intelligence), Snowflake being customer zero for its own product.",
      "The core failure principle: 'user trust is earned extremely hard and is lost overnight' — if users like the first five answers they return; if not, it's 10x the effort to win them back, if ever.",
      "Before touching the agent, the speaker opened a spreadsheet, walked the sales process and wrote 150 test questions — even though the engineering team objected that the data wasn't connected. First test run: 50% accuracy.",
      "Quality over coverage as doctrine: 'we don't want to try to answer 100 questions and get them 70% right, we want to answer 50 questions but get them 95% right.' 60% of the data was added after launch, over the 6-7 months following.",
      "Today's agent: 15 semantic views, 85 tables, 3,000 columns, five to six MCP connections, close to 20 skills.",
      "Phased launch: pilot with AI-native early adopters to prove accuracy; a 10% beta with 600 people to prove MVP coverage and retention — they exited beta at >70% weekly-active retention; then GA.",
      "Change management is where AI projects actually fail, not technology: two weeks post-launch only 20% of the org had tried it. The speaker spends 60-70% of his time in sales meetings, giving demos, building adoption dashboards by team, shaming managers and getting sales-leader sponsorship — without which they'd be at half of today's usage.",
      "'Collapsing of the wow factor': the maturity ladder runs talk-to-your-data → automate my workflows (agent monitors inbox and Slack, drafts Gmail responses, automates outreach) → team-level skills, dashboards, apps and alerts → hyper-personalization. Stall at stage one and you get disrupted in a month or two because switching is now overnight.",
      "They shipped deliberately unfinished: launch was a nine-page agent instruction doc, a couple of Cortex Analyst tools with semantic views and a Cortex Search service, with instruction versions managed in a Google Doc — to 6,000 people. CI/CD, eval infrastructure (unit tests, routing tests), a skill library, MCPs, progressive disclosure, user memory, task scheduling and a Slack interface all came after.",
      "Only 20% of the original PRD/architecture diagram matches today's architecture; 30-40% of sprint work is constant re-architecting onto new technology, the rest features and quality.",
      "Logs as the feedback loop: LLMs classify 40,000 weekly questions into topic categories and subcategories, surfacing feature gaps in real time — including where users repeat questions or swear at the agent. For sales enablement this replaces interviewing ~100 sellers a week: gaps are visible in a minute or two, and they can pull from Confluence, Jira, Slack and PRDs to generate battle cards and feed them back into the agent. Logs also enable matchmaking between sales teams unknowingly targeting the same accounts."
    ],
    "takeaways": [
      "Write your evaluation set from the business process before you build — the speaker's 150 questions came from the sales process, not from what data was already connected — and treat the first five questions a user asks as the trust budget you're spending.",
      "Deliberately narrow scope to what you can answer at ~95% and add data after launch; going for coverage first will 'shoot yourself in the foot.'",
      "Stage the rollout with an explicit exit criterion per phase: pilot proves accuracy, a ~10% beta proves MVP coverage and retention (they used >70% weekly-active retention), then GA.",
      "Budget engineering-adjacent time for activation: demos, adoption dashboards by team, and executive sponsorship. If people never try the product, low usage is not a product problem you can fix with code.",
      "Ship on today's stack in days and weeks and expect to re-architect continuously (they run 30-40% of sprints on it); don't buy or design a perfect architecture before launching.",
      "Instrument and classify your chat logs with LLMs — the categorized question stream is your real-time feature-gap list, enablement-content backlog, and the source of the compounding growth curve."
    ],
    "topics": [
      "agents",
      "go-to-market",
      "enterprise-ai",
      "evals",
      "change-management",
      "adoption",
      "observability",
      "mcp"
    ],
    "tools": [
      "Snowflake",
      "Snowflake Cortex",
      "Snowflake Co-work (formerly Snowflake Intelligence)",
      "Cortex Analyst",
      "Cortex Search",
      "Cortex Sense",
      "semantic views",
      "MCP",
      "Salesforce",
      "Slack",
      "Gmail",
      "Google Docs",
      "Confluence",
      "Jira"
    ],
    "quotes": [
      {
        "text": "User trust is earned extremely hard and is lost overnight",
        "at": "03:31",
        "url": "https://www.youtube.com/watch?v=DrTdD-ttjCY&t=211s"
      },
      {
        "text": "We don't want to try to answer 100 questions and get them 70% right. We want to answer 50 questions, but get them 95% right.",
        "at": "04:47",
        "url": "https://www.youtube.com/watch?v=DrTdD-ttjCY&t=287s"
      },
      {
        "text": "And we were managing the agent instructions versions out of a Google Doc. That's how we launched it. To 6,000 people.",
        "at": "12:30",
        "url": "https://www.youtube.com/watch?v=DrTdD-ttjCY&t=750s"
      },
      {
        "text": "Every time people are happy, you should be paranoid.",
        "at": "17:23",
        "url": "https://www.youtube.com/watch?v=DrTdD-ttjCY&t=1043s"
      }
    ],
    "words": 4825
  },
  "summary_url": "/#DrTdD-ttjCY",
  "transcript": {
    "html": "/transcripts/DrTdD-ttjCY.html",
    "txt": "/transcripts/DrTdD-ttjCY.txt",
    "vtt": "/transcripts/DrTdD-ttjCY.vtt"
  }
}