{
  "video": {
    "id": "kRkcNOsRyYg",
    "title": "AI on Your Lakehouse: Context Comes in Shapes, Not Queries — Zach Blumenfeld, Neo4j",
    "duration": 7150,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": "In person"
  },
  "analysis": {
    "video_id": "kRkcNOsRyYg",
    "title": "AI on Your Lakehouse: Context Comes in Shapes, Not Queries — Zach Blumenfeld, Neo4j",
    "one_liner": "A Neo4j workshop showing three reusable graph \"shapes\" — a metadata semantic layer over BigQuery, a table-of-contents/outline tree over documents, and Leiden-detected themes — that give a Claude Code agent context text-to-SQL and vector search can't provide, especially for estate-level questions like \"what documentation are we missing?\"",
    "summary": "Zach Blumenfeld argues that on a lakehouse you no longer struggle to *access* data — text-to-SQL and vector search work — but agents still get \"confidently wrong\" once you hit hundreds of tables or millions of documents, and they fail entirely at estate-level questions that require proving a negative or seeing global patterns. His answer is to give the agent context in shapes rather than queries: a Neo4j metadata graph (built by the labs project NeoCarta) as a semantic layer over BigQuery so the agent joins correctly without ETL-ing the data; a deterministically loaded containment tree with named links and hierarchical URIs so the agent can *navigate* documents like a table of contents; and Leiden community detection over the documents' links-to graph to surface themes with no LLM labelling at all. The whole workshop is run through Claude Code with the Neo4j CLI plus Cypher and GDS skills, writing Cypher from specs rather than by hand, and ends with live demos on a fictional AutoFix Group auto-repair chain.",
    "key_points": [
      "Three shapes: **connection** (semantic layer over the warehouse), **outline/table of contents** (containment tree + cross-document links over unstructured docs), and **themes** (community detection to surface unknown groupings).",
      "The connection shape uses **NeoCarta**, a Neo4j *labs* project with an MCP server that pulls only BigQuery **metadata** into a graph — 6 tables and 5 reference join paths in the demo, derived from primary/foreign keys in BigQuery's information schema. Explicitly not an ETL: \"we're not using the graph to copy the data over.\"",
      "Reasons given for *not* moving OLTP/warehouse data into the graph: terabytes updated continuously need syncing, custom ETL for extra properties, and security posture — you may physically be able to move sensitive data but not be permitted to. ETL still wins when you need recursive-join performance, graph algorithms, embeddings or clustering.",
      "The document graph is loaded **deterministically** with regex/plain NLP — library → folder → document → nested sections, plus `NEXT_SECTION` ordering and Obsidian-style named `LINKS_TO` edges — rather than LLM entity extraction. Benefits claimed: idempotent, faster, reload-safe. Every node gets a hierarchical URI so subtrees can be fetched or filtered.",
      "Search is a **Lucene full-text index inside Neo4j** on document and section nodes (not folders), with the hierarchical URI used as a `STARTS WITH` **post-filter** to scope to e.g. just the recalls subfolder — no vector search used, which he notes also sidesteps wiring embeddings across six different vendors. He adds \"semantic expansion\": let the model turn \"engine shuddering\" into `misfire OR rough idle`.",
      "Themes use **Leiden** community detection from Neo4j's GDS library over an in-memory projection (sections collapsed to document level), reporting **conductance** as tightly/loosely interlinked. A `gamma` parameter tunes granularity — gamma=1 gave 13 groups, gamma=2 gave 14. Themes are never named by an LLM; labels come straight from document titles and link names, so the same data always yields the same result. Microsoft's GraphRAG uses the same algorithm but adds LLM summaries per community — more money, slower, non-reproducible.",
      "Cypher is written by the agent, not by hand: the **Neo4j CLI** installs skills (Cypher, GDS, agent memory, drivers), and the workshop pattern is to leave a `# WIRE` placeholder in a Python script and point Claude at a spec file in `docs/` describing the desired output shape. Without the skill, models pull \"Stack Overflow questions from like four or five or six years ago\" and emit outdated Cypher.",
      "Live demos on the fictional AutoFix Group: \"which vehicles received part IC 2042\" (connection shape); a technician's VIN + DTC code question that grounded in the tree, found a revised ignition coil, then joined to work-order history — motivated by minimizing the shop's **comeback ratio**; a mismatch/gap question that found two field DTC codes with no code-level documentation; and a coverage question correlating themes to work orders. In the gap demo the agent *skipped* the outline and did an exhaustive full-text crawl instead — slower, but the hierarchical URIs still got it there.",
      "The structured graph and the unstructured graph are **not linked** in this workshop — the agent extracts a code from one side and joins in real time. He says linking them would give more deterministic mapping but was skipped for speed of getting started and model-agnosticism."
    ],
    "takeaways": [
      "Build a metadata/semantic-layer graph over your warehouse instead of ETL-ing the warehouse into a graph — it's how you let an agent find a 3–4 hop join it would otherwise have to understand hundreds of tables to reach, and it dodges the sync and security problems.",
      "If your documents already have inherent structure and interlinking, load them deterministically (containment tree + ordering + named links + hierarchical URIs) before reaching for LLM entity extraction — it's idempotent, faster, and cheap to rebuild from source.",
      "Give the agent *navigation* alongside search, not instead of it: he explicitly says the outline shape is not a replacement for semantic search, and pairs it with Lucene + semantic expansion; but only navigation answers \"what's missing\", because similarity search by definition can only find things that are there.",
      "Use the Neo4j CLI plus the Cypher and GDS skills for any agentic Cypher writing, and drive complex queries from a written spec file with a placeholder in the script rather than prompting ad hoc.",
      "Keep relationship naming coarse (`HAS`, `LINKS_TO`) for multi-level traversal — granular names let the agent be more precise but hundreds of relationship types become hard to fit in a context window and hurt Cypher generation.",
      "Watch document and link naming quality: with metadata-only themes, badly named or drifted/outdated documents make the whole view uninformative — he's considering adding last-updated dates and an authoritative flag."
    ],
    "topics": [
      "graphrag",
      "knowledge-graphs",
      "agents",
      "rag",
      "semantic-layer",
      "lakehouse",
      "text-to-sql",
      "community-detection"
    ],
    "tools": [
      "Neo4j",
      "NeoCarta",
      "Neo4j Graph Data Science (GDS)",
      "Neo4j CLI",
      "Cypher",
      "Neo4j virtual graph (preview)",
      "Claude Code",
      "Claude Opus 4.8",
      "Anthropic API",
      "MCP",
      "Claude Skills",
      "BigQuery",
      "Google Cloud Storage",
      "Databricks",
      "Databricks Genie",
      "Snowflake",
      "GitHub Codespaces",
      "Graph Academy",
      "Lucene",
      "Leiden algorithm",
      "Louvain algorithm",
      "Microsoft GraphRAG",
      "PageIndex",
      "Obsidian",
      "GLiNER",
      "Cursor"
    ],
    "quotes": [
      {
        "text": "So this is sort of like proving a negative which can be very hard with something like semantic search uh which can only match similar things, right? It can't really find a negative example.",
        "at": "05:29",
        "url": "https://www.youtube.com/watch?v=kRkcNOsRyYg&t=329s"
      },
      {
        "text": "the thing that we're doing here really is we're not using the graph to copy the data over. There's not like an ETL into graph. What we're doing is we're using the graph as a semantic layer.",
        "at": "27:04",
        "url": "https://www.youtube.com/watch?v=kRkcNOsRyYg&t=1624s"
      },
      {
        "text": "an ontology helps something interpret and reason about the data. A semantic layer helps something understand what the consistent agreed upon terms are so it can query the data accurately.",
        "at": "23:48",
        "url": "https://www.youtube.com/watch?v=kRkcNOsRyYg&t=1428s"
      },
      {
        "text": "otherwise what what it's going to do is it's going to go on the internet and get the Stack Overflow questions from like four or five or six years ago and it's going to give you outdated bad cipher basically.",
        "at": "41:20",
        "url": "https://www.youtube.com/watch?v=kRkcNOsRyYg&t=2480s"
      }
    ],
    "words": 20731
  },
  "summary_url": "/#kRkcNOsRyYg",
  "transcript": {
    "html": "/transcripts/kRkcNOsRyYg.html",
    "txt": "/transcripts/kRkcNOsRyYg.txt",
    "vtt": "/transcripts/kRkcNOsRyYg.vtt"
  }
}