How do I split a PDF into chunks for AI search, and what do I get?

PDF for AI gives you a chunk file with one JSON line per chunk, each with its text, page numbers and section path. On 2026-10-19 we ran a real 8-page paper through it and got 25 chunks of 58 to 1,989 characters. The table we checked and the formulas stayed intact. Five chunks were short, one sentence was cut in two by a figure, and one section path was wrong.

What did we test?

In our test we used an 8-page excerpt of a published machine-learning paper from arXiv. It has headings numbered down to the third level, three tables, six formulas and three images. We uploaded it to PDF for AI on osmanthdocs.com. The result was labelled "Review recommended" because two formulas were flagged. We then read the chunk file line by line and measured it.

What we measured	Result
Number of chunks	25
Fields on each line	id, source, section, pages, text
Chunk length	58 to 1,989 characters, median 1,138
Chunks under 300 characters	5 (c1, c9, c10, c19, c25)
Overlap between neighbouring chunks	None found
Chunks that end at the end of a sentence or table	24 of 25
A table with its caption	Kept in one chunk, with the caption above it
Formulas	Kept inside the chunk text between $$ signs

The section field holds the heading path, such as 3 Model Architecture > 3.1 Encoder and Decoder Stacks, so a chunk says where it came from. A page number list is on every line, and a chunk that crosses a page break lists both pages.

What went wrong?

• A figure cut a sentence in two. In the paper, a figure sits in the middle of a paragraph. Chunk c9 ends with "The output is computed as a weighted sum" and chunk c10 starts with the figure caption and then continues "of the values, where the weight...". A search that finds only one of the two chunks gets half a sentence.
• A figure label became a section. The paper has a heading-like label, "Scaled Dot-Product Attention", that appears to be text inside a figure. The tool treated it as a heading. As a result the seven chunks c10 to c16 carry a section path that begins with that label, and they lose the parent "3 Model Architecture". We did not compare it with the page image.
• Some chunks are very short. Five of the 25 chunks are under 300 characters, and one is 58. A tiny chunk gives a search little to match on.
• Nothing overlaps. If an answer straddles two chunks, neither contains all of it.

How do I prepare the chunks for use?

Pinecone's guide says that chunk size is a trade-off: small chunks capture finer meaning, large chunks keep more context, and structure-aware splitting uses headings and similar features to make chunks more coherent. Anthropic's article says that a chunk can lose its context, for example a sentence about a company's revenue that never names the company, and that chunk size, boundaries and overlap can all change retrieval, so you should experiment for your own case. Neither page gives one right size. This short Python script, which we ran on the chunk file from the test, adds the source and section path in front of each chunk and lists the short ones:

import json

with open("chunks.jsonl", encoding="utf-8") as f:
    chunks = [json.loads(line) for line in f if line.strip()]

texts = []
for c in chunks:
    prefix = f"{c['source']} | {c['section']}\n" if c["section"] else f"{c['source']}\n"
    texts.append(prefix + c["text"])

short = [c["id"] for c in chunks if len(c["text"]) < 300]
print(len(texts), "texts ready to embed; chunks under 300 characters:", short)

For our file it printed 25 texts and the five short chunk ids. What you do with the short ones is your choice. You could join each to its neighbour, or leave them. We did not test which gives better answers.

What should I check before using the chunks?

1. Read the label and notes for the PDF first.
2. Look at the section paths. A path that starts with a figure label, a running header or an author name is a sign of a wrong heading.
3. Find the chunks that end without a full stop and read the start of the next one.
4. Check chunk lengths against the limit of your embedding model.
5. Compare any table or formula that matters with the PDF.

What are the limits?

This is one 8-page paper, so it shows what the tool did here and not how often it will do the same on your file. We did not test whether the chunks give better answers in a real search system. Chunk sizes of 1,000 to 2,000 characters may be too large or too small for your setup. For giving a whole PDF to an assistant rather than a search index, see how to give a PDF to ChatGPT or Claude. For how the other JSON output is laid out, see how to turn a PDF into JSON for a script.

Your PDF is uploaded to our servers, processed with open-source software and deleted when processing finishes. The results stay for 60 minutes.