TokenPad

10 tools

RAG and Embedding Tools

Chunk, retrieve and pack context without paying for what you discard.

Most retrieval systems that answer badly do not have a model problem. They have a chunking problem: the passage that would have answered the question was split across two chunks, or buried inside one large enough to dilute it.

Retrieved context also dominates the prompt in most RAG systems — eight chunks of three hundred tokens is 2,400 tokens on every request, billed every time. Retrieving fewer, better chunks improves the answer and the bill together, which is an unusually clean trade.

Which one you need

Chunking is where retrieval quality is decided, so start there. The chunker shows you what your documents actually break into, which is usually less tidy than the mental model — long unbroken sections, tables that lose meaning when split, headings that carry context the chunk then loses.

The overlap tools answer the question that follows: how much duplication to accept in exchange for not cutting a fact in half. Ten to fifteen percent is a common landing point, and the cost of getting it wrong is visible at scale rather than in testing.

The embedding and storage calculators are for the decision nobody revisits later — dimension count. It multiplies storage and query latency across the whole corpus, and at tens of millions of chunks it is an infrastructure line item rather than a detail.

If answers are disappointing, resist tuning the prompt first. Measure retrieval separately on questions with known correct sources; in most underperforming systems the right chunk was never in the context at all.

Questions

What chunk size should I use?

The one your documents support, found by testing retrieval on real questions. Dense reference material wants small chunks and narrative wants larger ones — the default in whichever tutorial you read first was chosen for neither.

Do I need a vector database?

Below roughly a hundred thousand chunks, usually not. A brute-force similarity scan over an array is fast enough and vastly simpler to operate. The database earns its complexity at scale, and adopting it early mostly buys operational overhead.

Other categories