{ "video": { "id": "-x5GEVnkuRw", "title": "Structuring the Unstructured - Cedric Clyburn, Red Hat", "duration": 1241, "upload_date": null, "channel": "AI Engineer", "source": "AI Engineer", "event_format": "Online" }, "analysis": { "video_id": "-x5GEVnkuRw", "title": "Structuring the Unstructured - Cedric Clyburn, Red Hat", "one_liner": "Cedric Clyburn (Red Hat) shows how Docling — an open-source, Linux Foundation, local-first CLI and library — turns PDFs, images, tables and invoices into markdown/JSON/Pydantic documents that RAG and agents can actually use, without sending private data to a vendor or needing a GPU.", "summary": "The talk argues that unstructured data is the new context layer for AI, but naive PDF parsers truncate and merge text while frontier models are expensive and non-deterministic at scale — so Docling is the middle ground: fast, cheap, local, open source. Clyburn demos converting Docling's own research paper to markdown and Pydantic, extracting eight tables to data frames, pulling and annotating images with a local Granite model via Ollama, and visualising layout bounding boxes. He then shows 'chunkless'/agentic RAG where the markdown outline of the document *is* the retrieval index — no chunker, embedding model or vector database — tested against the IBM 2025 annual report at 418 sections. Finally he scales it out with `docling-serve` as a REST microservice and the Docling MCP server driven from Claude Code, Cursor or Codex.", "key_points": [ "A viral tweet showed 20 scientific papers now feature a nonsensical term invented when AI misread an old scanned PDF and merged two words from two different columns — and those papers are now being cited; in Docling the two words are visibly far apart and should never have been combined.", "Three-way comparison of approaches: a simple PDF parser is fast and CPU-cheap but truncates and merges text, spits tables out linearly and drops image content entirely; frontier models give decent quality but at e.g. $30 per million output tokens across thousands of PDFs, plus hallucination risk and structured output that breaks between a deprecated 5.1 and a 5.2 model version.", "Docling is a local CLI and library (`pip install docling`) that converts PDFs, websites and other formats to markdown, JSON, HTML, dict and a Pydantic data type, preserving page layout via a combination of OCR, layout analysis and vision models.", "Leandro at Hugging Face used Docling with OCR to pre-clean common crawl PDFs for the FinePDFs export and achieved 50x cost savings versus naive VLM/OCR — running on CPU, no GPU needed.", "Live demo on Docling's own eight-page research paper: markdown export, exploring pages/tables in the Pydantic object, extracting 8 tables to data frames in Jupyter, a PDF pipeline that scales up images to get picture + caption + embedded text elements, and a layout visualiser drawing bounding boxes — also usable to strip PII before extraction.", "Image enrichment: a PDF pipeline calls a locally running Granite model through Ollama's OpenAI-compatible completions endpoint to generate a detailed description of a diagram, so context survives even when the SME who drew it has left the organisation.", "Chunkless / agentic RAG: the markdown outline with per-section summaries *is* the entire retrieval index. A RAG agent iterating up to ~5 times answered 'What are the main AI models used in Docling?' in one iteration from 20 sections, and the same pattern scales to the IBM 2025 annual report's 418 sections — no chunker, embedding model or vector DB.", "For hundreds of thousands of documents, `pip install docling-serve` runs Docling as a REST API microservice (container or Kubernetes) with per-request options for OCR, backend and image annotation; the Docling MCP server (run via uvx) exposes conversion, generation and manipulation tools to agents like Claude Code, Cursor, Codex or Continue.", "Structured output: from an invoice you can request just the bill number, total invoice price and sender name as Pydantic, instead of dragging in headings and titles." ], "takeaways": [ "Stop sending raw PDFs to a frontier model for parsing at scale — run Docling locally on CPU first and keep the token spend for reasoning, not extraction.", "Treat document processing accuracy as the determining factor in answer correctness; a merged-column artefact from a bad parse propagates into papers, citations and customer answers.", "Try chunkless RAG before standing up a vector database: index the Docling markdown outline and let the agent pick sections in a loop.", "Use the PDF pipeline with a local VLM (e.g. Granite via Ollama) to annotate images and diagrams so their content enters retrieval instead of being dropped.", "Scale with `docling-serve` as a REST microservice for bulk ingestion, and wire the Docling MCP server into your coding agent so it can parse documents without you memorising the arguments." ], "topics": [ "document-processing", "rag", "pdf-extraction", "open-source", "agents", "mcp", "vision-models", "local-inference" ], "tools": [ "Docling", "docling-serve", "Docling MCP server", "Red Hat", "Linux Foundation", "Pydantic", "Ollama", "Granite", "Hugging Face", "FinePDFs", "Common Crawl", "NVIDIA", "Jupyter", "uvx", "VS Code", "Claude Code", "Cursor", "Codex", "Continue", "MLX", "Qwen", "IBM", "Hybrid Chunker", "OpenAI completions endpoint" ], "quotes": [ { "text": "20 scientific papers now feature a new nonsensical term that doesn't exist because AI misinterpreted a very old article that was scanned and taken to a PDF, merging two different words from two different columns in this PDF.", "at": "03:03", "url": "https://www.youtube.com/watch?v=-x5GEVnkuRw&t=183s" }, { "text": "that data and the way you process it is the key determining factor in whether your answer is going to be correct or incorrect for the user or customer at the end of the day", "at": "02:43", "url": "https://www.youtube.com/watch?v=-x5GEVnkuRw&t=163s" }, { "text": "we're doing RAG but without having to use a chunker or embedding model or vector database, etc., etc. So the index ends up being the markdown outline of the document.", "at": "14:36", "url": "https://www.youtube.com/watch?v=-x5GEVnkuRw&t=876s" }, { "text": "the two comparisons he did using GPU and using CPU for DocLing allowed him to do this at 50 times of a cost savings compared to VLMs and OCR naively", "at": "08:04", "url": "https://www.youtube.com/watch?v=-x5GEVnkuRw&t=484s" } ], "words": 4499 }, "summary_url": "/#-x5GEVnkuRw", "transcript": { "html": "/transcripts/-x5GEVnkuRw.html", "txt": "/transcripts/-x5GEVnkuRw.txt", "vtt": "/transcripts/-x5GEVnkuRw.vtt" } }