Why even bother chunking? Well, long documents might contain the...
Matt Pocock@mattpocockuk
42 views
Nov 01, 2025
~1 min read
1
Why even bother chunking?
Well, long documents might contain the info you need - but they probably also contain a lot of cruft.
Chunking lets you make individual parts of documents searchable. Pretty important for any RAG app.
Well, long documents might contain the info you need - but they probably also contain a lot of cruft.
Chunking lets you make individual parts of documents searchable. Pretty important for any RAG app.
2
@SrinivasSi78619 Let me ask some dumb questions:
Are you processing the embeddings in parallel or sequentially?
Can you work with a subset of the data in dev?
Are you processing the embeddings in parallel or sequentially?
Can you work with a subset of the data in dev?