{
  "video": {
    "id": "H7puB0RwJMM",
    "title": "Citation Needed: Provenance for LLM-Built Knowledge Graphs — Daniel Chalef, Zep AI",
    "duration": 1254,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": "In person"
  },
  "analysis": {
    "video_id": "H7puB0RwJMM",
    "title": "Citation Needed: Provenance for LLM-Built Knowledge Graphs — Daniel Chalef, Zep AI",
    "one_liner": "Zep's Daniel Chalef argues that because LLM-built knowledge graphs synthesize facts that never appear verbatim in any source, provenance has to be engineered into the data structure as graph edges back to verbatim source episodes — not logged afterwards.",
    "summary": "LLMs pull together data from many sources non-deterministically, and the synthesized output — a summary, an extracted fact, a structured record — may not appear verbatim in the inputs, destroying the paper trail. Chalef shows why a simple source ID on a fact breaks down for LLM context pipelines (many facts from many sources, entity merges, facts invalidated by new data) and argues that provenance in a context store is itself a knowledge graph, where tracing a fact to its source is just a graph walk. He walks through how Graphiti models this: source data as episode nodes, entity/edge triples hydrated into facts, metadata tags projected from episodes down to derived facts, and deletion rules driven by remaining parent episodes. He is explicit that the graph exposes which sources carry which tags, but the agent must apply the business rules — the policy is situational, not baked in.",
    "key_points": [
      "The stylized failure mode: an agent returns \"Patient has a penicillin allergy\" synthesized from an EHR record, a PDF lab report, and something the patient typed into an AI intake chat — presented to a doctor without indicating it came from the patient themselves, it can mislead treatment.",
      "Source IDs work in structured data warehouses where a pipeline outputs one value copied or mutated deterministically, but break for LLM context pipelines: many facts each synthesized from one or more sources, \"J. Smith\" and \"John Smith\" merged into one entity, and new data invalidating old facts so the store keeps changing underneath your pointer. An append-only log gets very hard to manage at scale.",
      "In Graphiti, source data becomes episodes (nodes on the graph); extracted entities plus the edge between them form a triple that hydrates into a fact. When two entities merge, the merged entity must keep all source links from both or you silently drop a source; when new data contradicts a fact, an invalid date is added to the mutated edge and the source episodes that caused the mutation are noted against the fact.",
      "Metadata projection: tag episodes at ingestion (e.g. an EHR tag) and all entities and facts derived from them inherit the tag, so an agent filtering for only verified clinical sources is a tag filter during the graph walk — one tagging action at ingestion supports evaluating veracity.",
      "Multi-parent facts need opposite policies with identical shapes: for an allergy flag, any one of three sources carrying it should block the prescription; for \"consent on file for procedure\", every parent episode must be verified. The graph exposes which episodes carry the tag; the agent executes the business rule.",
      "Deletion under retention policies or right-to-be-forgotten: delete the intake chat episode and the allergy fact survives (two remaining parent episodes support it) while the contact preference fact is deleted (derived solely from the deleted source). The rule — a fact is only deleted if no remaining episodes support it — is easy to apply because the link exists.",
      "In Q&A: the add_episode pipeline is structured extraction of entities, relationships and candidate facts as subject-verb-object triples, then a deduplication and deconfliction pass — e.g. \"Daniel loves Adidas shoes\" invalidated when Daniel returns the shoes with a nasty gram. They try hard NOT to use LLMs where traditional IR/NLP works — entropy, SimHash — because it is far cheaper, far faster and far more deterministic. Extraction itself is single-shot, with a reflection step for accuracy and for capturing why something changed, not just that it was related.",
      "Chalef concedes graph construction in Graphiti is really expensive, and that significant effort has gone into reducing cost and latency of generating graph artifacts. Not all provenance lives in the graph — relevancy/weight tracing sits in Zep in a separate data structure, outside the graph."
    ],
    "takeaways": [
      "Stop trying to retrofit lineage onto LLM-derived facts with a source ID column or an append-only log — model source→artifact links as graph relationships so tracing becomes a graph walk that survives entity merges and fact mutation.",
      "Tag episodes at ingestion with the classifications you will later want to filter on (source system, verified/unverified); inheritance down to derived facts means one action at write time gives you veracity filtering at read time.",
      "Keep source data verbatim alongside the derived artifacts, and when entities merge, carry over all source links from both sides — dropping one silently loses lineage.",
      "Write the multi-parent veracity policy in your agent, not the store: decide per fact type whether any tagged source is enough (allergy-style blocking) or every parent must be verified (consent-style), because identically shaped facts take opposite rules.",
      "For deletion and right-to-be-forgotten, apply \"a fact is only deleted if no remaining episodes support it\" rather than cascading from the source — partial source deletion should leave multi-source facts standing.",
      "Reach for traditional IR/NLP (SimHash, entropy) for dedup and resolution before reaching for an LLM — cheaper, faster, and more deterministic."
    ],
    "topics": [
      "provenance",
      "knowledge-graphs",
      "agent-memory",
      "temporal-graphs",
      "compliance",
      "entity-resolution",
      "context-engineering"
    ],
    "tools": [
      "Graphiti",
      "Zep",
      "GitHub",
      "SimHash",
      "EHR / electronic health record systems"
    ],
    "quotes": [
      {
        "text": "Synthesis often destroys the paper trail of how these outputs were originated.",
        "at": "00:36",
        "url": "https://www.youtube.com/watch?v=H7puB0RwJMM&t=36s"
      },
      {
        "text": "So provenence in a context store containing facts is a knowledge graph.",
        "at": "04:20",
        "url": "https://www.youtube.com/watch?v=H7puB0RwJMM&t=260s"
      },
      {
        "text": "Lineage needs to be built in to the data structure, engineered into the data structure, which is a graph, not logged afterwards.",
        "at": "10:26",
        "url": "https://www.youtube.com/watch?v=H7puB0RwJMM&t=626s"
      },
      {
        "text": "A lot of what we do uses LLMs but we try very hard not to use LLMs in this process as well.",
        "at": "17:04",
        "url": "https://www.youtube.com/watch?v=H7puB0RwJMM&t=1024s"
      }
    ],
    "words": 2976
  },
  "summary_url": "/#H7puB0RwJMM",
  "transcript": {
    "html": "/transcripts/H7puB0RwJMM.html",
    "txt": "/transcripts/H7puB0RwJMM.txt",
    "vtt": "/transcripts/H7puB0RwJMM.vtt"
  }
}