Chunk Visualizer

Retrieval quality dies at chunk boundaries. Run seven real chunking strategies over your own document and see the cuts painted in place, with size distribution, the overlap you actually got, and every boundary that lands mid-sentence.

Runs entirely in this tab. No network requests, no logging, nothing leaves the page.

Document & In-Place Boundary Map

Samples:

Legend:■ Chunk■ Overlap⚠️ Cut

4 Chunks:

# Retrieval-augmented generation Retrieval quality is decided long before the model sees a prompt. Chunking strategy, embedding choice, and the reranker do most of the work; the generation step mostly reveals whether the earlier decisions were sound. ## Chunking A fixed-size splitter is the baseline. It cuts every N characters, which means it cuts sentences in half, and a chunk that begins mid-clause carries a fragment the embedding cannot represent well. The recursive character splitter descends a ladder of separators: paragraphs, then lines, then sentences, then words. It takes the largest unit that fits inside the budget. Overlap exists to survive the boundary case: a fact stated across two sentences lands wholly inside at least one chunk. It is not free. Overlap of 20% inflates the index by roughly a fifth, and every duplicated token is paid for twice, once at embedding time and again on every retrieval that returns both neighbours. ## Evaluation Measure recall at k before anything else. A pipeline that never retrieves the right chunk cannot be rescued by a better model, and a pipeline that retrieves it at rank 40 cannot be rescued by a bigger context window either. Then measure faithfulness: whether the answer is actually supported by what was retrieved. Numbers beat intuition here. Chunk size is the single most consequential knob in most RAG systems and the one most often set by copying a tutorial.

# Retrieval-augmented generation

Retrieval quality is decided long before the model sees a prompt. Chunking
strategy, embedding choice, and the reranker do most of the work; the generation
step mostly reveals whether the earlier decisions were sound.

## Chunking

A fixed-size splitter is the baseline. It cuts every N characters, which means it
cuts sentences in half, and a chunk that begins mid-clause carries a fragment the
embedding cannot represent well. The recursive character splitter descends a
ladder of separators: paragraphs, then lines, then sentences, then words. It
takes the largest unit that fits inside the budget.

Overlap exists to survive the boundary case: a fact stated across two sentences
lands wholly inside at least one chunk. It is not free. Overlap of 20% inflates
the index by roughly a fifth, and every duplicated token is paid for twice, once
at embedding time and again on every retrieval that returns both neighbours.

## Evaluation

Measure recall at k before anything else. A pipeline that never retrieves the
right chunk cannot be rescued by a better model, and a pipeline that retrieves it
at rank 40 cannot be rescued by a bigger context window either. Then measure
faithfulness: whether the answer is actually supported by what was retrieved.

Numbers beat intuition here. Chunk size is the single most consequential knob in
most RAG systems and the one most often set by copying a tutorial.

Splitter Strategy & Tuning

Budget Unit:

The default in most RAG stacks, and the right first choice. It tries to split on paragraphs, then lines, then sentences, then words, taking the largest unit that fits, so boundaries land at natural breaks unless the text leaves no option. Merging the pieces back up to the size budget is what keeps it from emitting a pile of one-line chunks.

Chunk Size (chars)500

Overlap (chars)80

Window (Sentences either side)1

Separator ladder (preferred first, comma separated)

Diagnostic Metrics & Sizing Curve

chunks

mean size

366c

median 359c · 88 tok

range

266–478c

index expansion

1.02×

chars stored ÷ source

overlap achieved

asked 80c

sentence cuts

0 runts · 0 oversize

Chunk size distribution

0 charsbudget 500500 chars

About this tool

01 · In-Place Boundary Map

Tracks exact source offsets to paint cuts directly over document text, distinguishing overlapping regions from single-coverage zones.

Visual: Highlights mid-sentence breaks and fragment tails.

02 · Strategy Support

Seven standard splitter implementations: LangChain recursive-character, character, Markdown header, semantic topic boundary, code AST, sliding window, and sentences.

Real behavior: Descends separator tiers and merges up to target budget.

03 · Diagnostic Metrics

Measures achieved vs requested overlap, mid-sentence severed boundaries, index expansion factor, and size distribution clustering.

Sizing: Dynamic token-to-character ratio calibration.

Operational Limits & Assumptions

· Lexical Semantic Split: The semantic strategy uses term-vector shift client-side without remote model calls; treats synonym rephrasing as topic movement.
· Sentence Heuristics: Rule-based sentence segmentation may miss boundaries in OCR dumps, unpunctuated voice transcripts, or rare abbreviations.
· Index Overhead: Higher overlap increases total vector store storage and retrieval re-ranking costs.
· Eval Boundary: Visual inspection verifies splitter mechanics; end-to-end recall requires validation against your evaluation query set.

Found a case it gets wrong? Tell me about it .