{
  "video": {
    "id": "Lrw0jqBNaw0",
    "title": "The Death of Developer Advocates — Stephanie Jarmak, Sourcegraph",
    "duration": 1096,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer"
  },
  "analysis": {
    "video_id": "Lrw0jqBNaw0",
    "title": "The Death of Developer Advocates — Stephanie Jarmak, Sourcegraph",
    "one_liner": "A Sourcegraph research scientist argues developer advocacy isn't dead but has gained a second user — the agent — and shows how to measure it: an SDLC benchmark of her company's MCP tool plus a GEO experiment where their product was recommended 65% of the time to comparison shoppers and 0% of the time to someone describing the actual pain it solves.",
    "summary": "Stephanie Jarmak presents as an \"agent advocate\" (her DevRel manager wrote the title, then went on vacation), tracing the arc from 1980s software evangelism to 2010s developer advocacy to 2026, where developers orchestrate fleets of agents and non-engineers can drive dev tools. She argues the agent is simultaneously a user of your product — reading docs, calling APIs, recovering from errors — and a recommender of it, so DevRel must instrument both sides. She shows two concrete measurement projects: CodeScaleBench, hundreds of SDLC-shaped tasks run with and without Sourcegraph's code-navigation MCP tool, yielding thousands of traces that expose per-turn friction; and a GEO (generative engine optimization) pilot measuring whether chatbots surface the product at a user's moment of need. Her closing frame is the curb cut: build for the agent and the human path gets cleared too.",
    "key_points": [
      "The role arc: 1980s software evangelism (one-way) → 2010s developer advocacy (two-way feedback loop, developers as kingmakers, DX as GTM strategy) → 2026, where \"what it means to be a developer is completely changing,\" so the advocate role must change with it.",
      "The agent is both user and recommender: it reads your docs (differently, as a machine), calls your API, hits and recovers from your errors — and it also installs libraries and embeds frameworks into workflows on the developer's behalf, driving bottom-up adoption that DevRel used to own.",
      "CodeScaleBench: she built hundreds of tasks reflective of the software development life cycle and ran agents with and without Sourcegraph's code navigation MCP tool, producing thousands of traces. One trace showed the model assuming a `read line` parameter from training-data bias instead of `start line`; the error message was good enough that it self-corrected, but it burned an entire turn — a fixable tool-description problem.",
      "Buyers now evaluate tools on agent-facing metrics: not just does it work, but how many tokens the agent burns using it and how fast it is.",
      "GEO pilot: with prompts written as an active comparison shopper for code intelligence tooling, their product was recommended ~65% of the time; with the more typical pain-shaped prompt — \"we keep breaking downstream services when we change shared libraries because we can't see all the consumers\" — zero mentions. The agent suggested the developers make a wiki page instead.",
      "Model staleness compounds: the pilot ran on Claude Sonnet 4, which kept pitching Cody, an older Sourcegraph product. Re-running it that afternoon on 4.6 thinking pitched Cody *more*, because old model output keeps accumulating on the internet — \"you have to figure out how to bury all of that noise with your true signal.\"",
      "Guidance for agent discoverability: llms.txt-style authoritative pages, give the agent something quotable, keep examples current even if the product hasn't changed (freshness feeds relevance algorithms), agents like charts and FAQs, be present in marketplaces and MCP registries, and remove friction — an agent will not recommend a tool that requires three demos and emailing a sales rep.",
      "Three flavors of the role: engineering (MCP interfaces, evals, instrumentation with the eng team), product (own the end-to-end agentic experience and agent-experience rubrics), marketing (pipegen — how agents enter the funnel and bring developers along).",
      "Credibility now splits by audience: don't sling Claude slop at humans (\"tell your AEs to stop that as well\"), but agents show a bias toward their own content, so agent-facing material can have as many em dashes as it wants provided it's structured.",
      "The DevRel core survives with a changed audience: enablement for both agent-orchestrating developers and for agents (machine-readable content, agent-friendly APIs); community with new privacy questions as people bring their Claudes into Discord and record conversations; feedback loops where you can spin up thousands of agents to run experiments developers wouldn't sit for."
    ],
    "takeaways": [
      "Point a coding agent at your docs, read the resulting transcript, and write an agent experience report — she names this as the one thing DevRel can do immediately.",
      "On the GTM side, build GEO experiments: write prompts the way your ICP actually describes their pain (not comparison-shopping prompts), and track mentions versus recommendations separately — the gap between the two is where the messaging work is.",
      "Instrument your MCP server/tool with SDLC-shaped eval tasks run with and without your tooling, then read the traces for wasted turns caused by parameter names or descriptions that fight the model's training-data priors.",
      "Track token cost and latency of your tool as first-class product metrics, because that is how organizations are now evaluating it.",
      "Make your product reachable where agents look — MCP registries and marketplaces — and cut any human-gated step (demo, sales email) between discovery and use, or agents will silently route around you.",
      "Re-run GEO measurements against new model versions rather than assuming they improve: newer models can amplify stale product information rather than correct it."
    ],
    "topics": [
      "devrel",
      "agents",
      "evals",
      "geo",
      "mcp",
      "benchmarks",
      "developer-experience",
      "go-to-market"
    ],
    "tools": [
      "Sourcegraph",
      "Cody",
      "CodeScaleBench",
      "MCP",
      "Claude",
      "Claude Sonnet 4",
      "Claude 4.6 thinking",
      "ChatGPT",
      "GitHub",
      "Discord",
      "llms.txt"
    ],
    "quotes": [
      {
        "text": "I had like zero commits on GitHub last year, and now I have 12,000, and I'm like an open source maintainer for multi-agent orchestration framework.",
        "at": "03:54",
        "url": "https://www.youtube.com/watch?v=Lrw0jqBNaw0&t=234s"
      },
      {
        "text": "In the previous model, it kept pitching Cody, which was like one of our older products. But when I ran it again, it pitched Cody even more, right? Cuz like now you have all of these like old models outputting content that then is like compounding in the internet. So, you have to figure out like how to bury all of that noise with your true signal.",
        "at": "11:04",
        "url": "https://www.youtube.com/watch?v=Lrw0jqBNaw0&t=664s"
      },
      {
        "text": "You also want to make sure your product is where the agents are, right? You're going to market. So, go go to agent market, right?",
        "at": "12:14",
        "url": "https://www.youtube.com/watch?v=Lrw0jqBNaw0&t=734s"
      },
      {
        "text": "Curb cuts were built for wheelchairs, like built for a specific user to use them. But now everybody benefits from that, right? Anybody with wheels, strollers and suitcases and all of those things. So my argument is that by serving the agents, the human path gets cleared, too.",
        "at": "17:02",
        "url": "https://www.youtube.com/watch?v=Lrw0jqBNaw0&t=1022s"
      }
    ],
    "words": 3994
  },
  "summary_url": "/#Lrw0jqBNaw0",
  "transcript": {
    "html": "/transcripts/Lrw0jqBNaw0.html",
    "txt": "/transcripts/Lrw0jqBNaw0.txt",
    "vtt": "/transcripts/Lrw0jqBNaw0.vtt"
  }
}