{
  "video": {
    "id": "RGSFUqzqErE",
    "title": "On AI and Knowledge — Pablo Castro, Distinguished Engineer & CVP for AI Knowledge, Microsoft",
    "duration": 1055,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": "In person"
  },
  "analysis": {
    "video_id": "RGSFUqzqErE",
    "title": "On AI and Knowledge — Pablo Castro, Distinguished Engineer & CVP for AI Knowledge, Microsoft",
    "one_liner": "Microsoft's Pablo Castro splits agent knowledge into intrinsic (in the model), extrinsic (retrieval/grounding via Microsoft IQ and Foundry IQ) and learned (agents tuned from their own traces by Foundry's agent optimizer), and demos the last two live.",
    "summary": "Castro argues the current exponential — IntelliSense in '96, ML-ranked completions 22 years later, GitHub Copilot 3 years after that, then Cursor, Copilot X, Opus 4.5 and Open Claw — was driven by models' intrinsic parametric knowledge plus reasoning, but that intrinsic knowledge alone can't make an agent participate in a company's work. For that he presents Microsoft IQ (Work IQ, Fabric IQ, Foundry IQ, Web IQ) as one entry point into an organization's ambient data, and a layered retrieval stack where you can hand Foundry a pile of PDFs or drop to vector quantization and lexical retrieval in the same system. The third category, learned knowledge, is materialized as Foundry's agent optimizer, which generates an evaluation from traces, hill-climbs candidate agent configurations against it, and swaps in the winner. Both middle and final sections are shown as live demos in Foundry, Azure and VS Code.",
    "key_points": [
      "Knowledge for agents splits three ways: intrinsic (model parametric memory from training), extrinsic (retrieval/grounding), and learned (captured by observing and tuning the work itself).",
      "The coding-assistance exponential, as he dates it: IntelliSense in 1996; 22 years to ML-ranked IntelliSense options; 3 years later GitHub Copilot — which shipped before ChatGPT was announced; then Cursor and GitHub Copilot X; then Opus 4.5 late last year; then Open Claw, 'incredibly successful software' with not a single line of code written by hand.",
      "Microsoft IQ is a set of capabilities, not one feature: Work IQ (SharePoint documents, email, calendar, chats and the connections between people), Fabric IQ (data warehouses, data lakes, Power BI reports), Foundry IQ (push your own data for agent grounding) and Web IQ (public web).",
      "Evaluations from AI Search — the search technology behind Foundry IQ — show individual retrieval methods consistently underperform combined methods, especially on real-world customer scenarios; the 'cosine similarity is all you need' phase was wrong.",
      "Foundry IQ is deliberately layered: hand it PDFs or images and it does chunking, vectorization, relevance/ranking and agentic retrieval for you, or go to the bottom of the same stack to control vector index structure, quantization approach, indexing algorithm and lexical retrieval — and move up and down as needs change.",
      "Agentic retrieval reflects on the data set and decides whether the stated information need is satisfied before returning results; across metrics like evidence recall and answer completeness it beats single-shot retrieval on difficult cases.",
      "Demo: creating a Foundry knowledge base over a movies dataset — choose the model powering the retrieval workflow, set an effort level that trades latency against quality, and ground on three sources at once (unstructured PDFs in blob storage, structured parquet tables of statistics, and the web). Every knowledge base is an MCP server, so an existing harness connects with no glue code.",
      "Agent optimizer demo in VS Code with the Foundry toolkit: 'eval generate' builds a task-adherence evaluation from the agent's traces and instructions; 'optimize' hill-climbs that metric for roughly 45 minutes generating candidate configurations; 'optimize apply' swaps the winning config in. The optimized instructions are not handwritten — they emerge from the loop and from reflecting on real user traces.",
      "Retrieval is evaluated for token efficiency too: the goal is the most information-dense answer in the fewest tokens.",
      "Claude in Microsoft Foundry went generally available the day before the talk, alongside thousands of models in the Foundry catalog."
    ],
    "takeaways": [
      "Stop treating retrieval as a single method — combine lexical, vector and agentic retrieval, because evaluations repeatedly show combined methods beat any individual one on real customer data.",
      "Externalize your agent's configuration (instructions, tool definitions, skills) as a precondition for automated optimization; the optimizer works by swapping configurations, so inlined prompts block the whole loop.",
      "If you have no evaluation set, generate one from existing traces and instructions rather than skipping evaluation — then hill-climb the agent against it and only deploy candidates that beat the baseline.",
      "Ground agents on the organization's ambient data (docs, mail, chat, warehouses, web), not just the dataset you hand-curated for the agent — and expose knowledge bases as MCP servers so any harness can use them.",
      "Budget retrieval output by tokens, not just relevance: judge a retrieval system on information density per token, since those tokens compete with everything else in the agent's context."
    ],
    "topics": [
      "rag",
      "agents",
      "retrieval",
      "evals",
      "context-engineering",
      "knowledge-bases",
      "mcp",
      "agent-optimization"
    ],
    "tools": [
      "Microsoft Foundry",
      "Foundry IQ",
      "Work IQ",
      "Fabric IQ",
      "Web IQ",
      "Microsoft IQ",
      "Azure AI Search",
      "Foundry toolkit for VS Code",
      "VS Code",
      "MCP",
      "GitHub",
      "GitHub Copilot",
      "GitHub Copilot X",
      "IntelliSense",
      "ChatGPT",
      "Cursor",
      "Claude",
      "Opus 4.5",
      "Open Claw",
      "SharePoint",
      "Power BI",
      "Azure Blob Storage",
      "Parquet",
      "ai.azure.com",
      "Microsoft"
    ],
    "quotes": [
      {
        "text": "for a hot second as an industry, we thought that if we could get really, really good at computing cosine similarity between vectors, we were all set for retrieval.",
        "at": "07:27",
        "url": "https://www.youtube.com/watch?v=RGSFUqzqErE&t=447s"
      },
      {
        "text": "incredibly successful software like like Open Claw comes out to existence with not a single line of code written by hand.",
        "at": "03:27",
        "url": "https://www.youtube.com/watch?v=RGSFUqzqErE&t=207s"
      },
      {
        "text": "every knowledge base is an MCP server, so you can just connect to it without having to write any glue code in the middle.",
        "at": "11:26",
        "url": "https://www.youtube.com/watch?v=RGSFUqzqErE&t=686s"
      },
      {
        "text": "So this is a real learning loop materialized in practice.",
        "at": "16:42",
        "url": "https://www.youtube.com/watch?v=RGSFUqzqErE&t=1002s"
      }
    ],
    "words": 3570
  },
  "summary_url": "/#RGSFUqzqErE",
  "transcript": {
    "html": "/transcripts/RGSFUqzqErE.html",
    "txt": "/transcripts/RGSFUqzqErE.txt",
    "vtt": "/transcripts/RGSFUqzqErE.vtt"
  }
}