{
  "video": {
    "id": "tZycGA_Fbfo",
    "title": "Paving the Road to Production: Governed Enterprise AI Agents — Harish Gaggar, Credit Karma",
    "duration": 1767,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": ""
  },
  "analysis": {
    "video_id": "tZycGA_Fbfo",
    "title": "Paving the Road to Production: Governed Enterprise AI Agents — Harish Gaggar, Credit Karma",
    "one_liner": "Credit Karma ships enterprise agents by having a dev portal scaffold every agent repo with standard memory, tracking/eval, PII-blocking and context-optimizer frameworks, and by forcing every tool call through a central authenticating MCP gateway to internally reviewed MCP servers. Their context optimizer cuts token use by about 90% in demos and 70–78% in typical use.",
    "summary": "Harish Gaggar (Intuit Credit Karma) argues that agents fail in production because the foundational controls are missing, not because the agents are weak. His answer is to 'build an enterprise framework, not just an agent'. Registering an agent in a dev portal produces a repo pre-wired with LangGraph plus memory, tracking/eval, PII-blocking and context-optimizer frameworks, and it also provisions workload identity, memory DB, eval tables and secrets. The talk covers the CI/CD path to Kubernetes, a custom MCP gateway with JWT auth and Redis-backed sessions, gateway monitoring on OpenTelemetry/Tempo/Prometheus, and a LangGraph flow with human-in-the-loop approval. It ends with demos of the memory checkpoints, the eval tables and the token savings.",
    "key_points": [
      "Agents usually fail in production for six reasons: no gateway for authentication and routing, MCP servers without the right tools, no tracking or eval, no memory management, no context optimisation and no PII/secret protection.",
      "A dev portal takes the agent ID, use case, framework (LangGraph/LangChain), language (Python/TypeScript), skills and prompts. It generates a repo containing the agent, memory, tracking/eval, context-optimizer and PII frameworks, and it provisions workload identity (K8s-to-GSA binding), memory DB, eval tables and secret management. The reason given is that developers otherwise stick to one or two frameworks, and centralising keeps them consistent and visible across the company.",
      "The path to production: build and test a local Docker container, get two PR approvals, register in a central agent registry, then CI/CD builds and vulnerability-scans the image, pushes it to the registry and deploys it to a Kubernetes namespace through an internal orchestrator.",
      "In air-gapped environments, build your own MCP servers (with the FastMCP wrapper) and MCP gateway. Every tool must pass internal security review before production, and tools enforce filters and limits and block fields that shouldn't reach the LLM.",
      "Gateway flow: the first query sends a one-time auth request. The gateway saves session state in Redis and returns a JWT. The client then calls a namespace endpoint such as /github/mcp or /artifactory/mcp with a bearer JWT. The gateway unwraps it and attaches the user token, and the MCP server exchanges the ID token so responses only cover what that user is authorised to see. No endpoint gets direct access.",
      "Gateway monitoring sends operational logs to ELK, Loki or CloudWatch (e.g. 401s in the last 10 minutes), traces through OpenTelemetry to Tempo (which tool was slow, agent or IDE), metrics to Prometheus (P95 latency per tool, auth-rejection spikes), and visualises it all in Grafana.",
      "The LangGraph flow is interpret → classify (read, modify, delete or sensitive) → plan subgraph (fetch evidence through the gateway and MCP tools) → validate → policy check. Read-only operations execute directly. Write or sensitive operations wait for human approval. Every outcome goes to the audit log.",
      "Memory: each thread gets a run ID per question and a checkpoint per node. It is stored in SQLite locally and Cloud Spanner on GCP in production. Tracking/eval uses tables per agent project (job tracking, eight quality checks per run, eval scores from 0 to 1, quality metrics, tool invocations, agent trajectory) so 100+ agents can be rolled up into executive ROI dashboards. PII blocking redacts anything above about 90% severity before it reaches the LLM, memory DB, eval tables or traces. The context optimizer selects by token limit, re-ranks, dedupes and compresses: the same Artifactory query dropped from 24,000 to 2,400 tokens, and a Slack bot query from 24,000 to about 1,500."
    ],
    "takeaways": [
      "Build the enterprise framework before shipping individual agents. Scaffold every new agent from a template that already includes memory, tracking/eval, PII blocking and context optimisation, and provision its identity, DBs and secrets automatically, so you don't discover the gaps later.",
      "Route every tool call, from local coding assistants and from agents alike, through one central MCP gateway that authenticates first. Give no direct endpoint access, and have security review every MCP tool before it goes to production.",
      "Classify agent operations by risk. Execute read-only actions directly, require human-in-the-loop approval for write, delete or sensitive ones, and audit-log every outcome.",
      "Store per-agent tracking and eval data (queries, model, token cost, quality scores, tool calls, trajectory) from day one, so you can answer ROI questions across all agents months later.",
      "Redact secrets and PII at ingestion, and put a context optimizer (prune, re-rank, dedupe, compress) in front of the LLM to justify agent cost."
    ],
    "topics": [
      "agents",
      "mcp",
      "ai-gateway",
      "governance",
      "observability",
      "evals",
      "agent-memory",
      "token-optimization"
    ],
    "tools": [
      "Intuit",
      "Credit Karma",
      "LangGraph",
      "LangChain",
      "FastMCP",
      "MCP",
      "Redis",
      "JWT",
      "Docker",
      "Kubernetes",
      "SQLite",
      "BigQuery",
      "Cloud Spanner",
      "GCP",
      "Azure",
      "AWS",
      "Langfuse",
      "ELK",
      "Loki",
      "CloudWatch",
      "OpenTelemetry",
      "Tempo",
      "Prometheus",
      "Grafana",
      "JFrog Artifactory",
      "GitHub",
      "Slack",
      "Amazon Kindle"
    ],
    "quotes": [
      {
        "text": "the AI agent cannot be trusted without continuous monitoring, evaluation, and feedback",
        "at": "12:13",
        "url": "https://www.youtube.com/watch?v=tZycGA_Fbfo&t=733s"
      },
      {
        "text": "There should not be direct access provided um on any endpoints. They has to go through the MCP gateway",
        "at": "11:48",
        "url": "https://www.youtube.com/watch?v=tZycGA_Fbfo&t=708s"
      },
      {
        "text": "In most of the cases, we get like 70 to 78% token saves.",
        "at": "27:14",
        "url": "https://www.youtube.com/watch?v=tZycGA_Fbfo&t=1634s"
      },
      {
        "text": "Don't try to be in rush to building some agent quickly and deploy ship it.",
        "at": "27:47",
        "url": "https://www.youtube.com/watch?v=tZycGA_Fbfo&t=1667s"
      }
    ],
    "words": 6004
  },
  "summary_url": "/#tZycGA_Fbfo",
  "transcript": {
    "html": "/transcripts/tZycGA_Fbfo.html",
    "txt": "/transcripts/tZycGA_Fbfo.txt",
    "vtt": "/transcripts/tZycGA_Fbfo.vtt"
  }
}