ToolFlowKit
Research / practical guide

Long Document Chunking for AI: Keep Conditions and Sources

Split a long document around meaning and keep the references needed to join its conditions back together. A price-table chunk may need a later billing clause before you can answer a pricing question. The two linked chunks below show the finished evidence record.

Define the question, preserve document structure and source IDs, extract relevant evidence from manageable sections, and reconcile cross-section conditions before drafting the combined answer. Keep the originals available for verification.

In this guide
  1. What does a source-preserving chunk contain?
  2. Split by meaning and retain the route back
  3. Inspect what reaches the model, not just the converted file
  4. Decide what the document processing must answer
  5. Keep reconciliation as a named step

What does a source-preserving chunk contain?

The document and amounts are authored training material. Save the heading path and page reference with each chunk, even when the model accepts the whole PDF. Do not split a table away from its headers or discard a see-section reference as boilerplate. The combined answer should be checked against both original locations before use.

What does a source-preserving chunk contain (practice)
ChunkRetained source textDependency / answer
D1-C01 / Price table / p2Plan amount: USD 20 per month; see billing terms section 7.Must read D1-C02 before stating charge period
D1-C02 / Billing / p8Listed monthly equivalents require annual billing.D1-C01 is an annual-plan monthly equivalent
Combined answerUSD 20 monthly equivalent, billed annually under section 7.Retain p2 and p8 references together

Split by meaning and retain the route back

  1. Give the document a stable ID/version and preserve its source pages or section positions.
  2. Split around useful headings and tables where possible, attaching the document title, section path, and relevant definitions.
  3. Keep table headers and any conditions needed to interpret extracted rows with the evidence.
  4. Ask each local pass for question-relevant facts, source positions, dependencies, and unresolved references.
  5. Combine the findings and inspect duplicated claims, contradictions, missing cross-references, and changed scope.
  6. Draft the answer after checking the decisive evidence against the original document.

Inspect what reaches the model, not just the converted file

Docling documents structure-aware chunking and approaches that preserve contextual information such as headings or table headers. Use the options that match your installed version and inspect the actual chunks produced for representative documents.

Text present in a converted document is not proof that every later retrieval or chunking step included it. When a final answer misses a passage, check the source, extraction output, chunks, and selected context in sequence. That diagnosis is more useful than repeatedly asking for a more thorough answer.

Decide what the document processing must answer

A procurement question about payment terms and an editorial question about the report's argument need different evidence. State the reader question first, then identify definitions, tables, footnotes, appendices, and sections that could alter the answer.

Check the input and output limits of the actual model and service you use. A file that uploads successfully may still need a deliberate processing plan. Keep enough context for interpretation rather than assuming all material receives equal attention in a single large request.

      Reusable structure
      QUESTION: [bounded task]
      DOCUMENT ID/VERSION: [state]
      SECTION PATH AND SOURCE POSITIONS: [state]
      RELEVANT DEFINITIONS: [supplied context]
      CURRENT SECTION: [text or table]
      
      Extract evidence relevant to the question. Attach source positions, preserve conditions and table headers, and list references to other sections that must be checked. Mark uncertainty and missing context. Return facts and dependencies for a later reconciliation pass; do not assume this section represents the entire document.

        Keep reconciliation as a named step

        Save the local evidence records and the combined review decisions. A final answer should expose unresolved cross-references and conflicting passages rather than smoothing them into a single confident narrative. Use the source positions to check the claims that matter to the reader.

        References