Building even a simple 𝗽𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻 𝗴𝗿𝗮𝗱𝗲 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝗔𝘂𝗴𝗺𝗲𝗻𝘁𝗲𝗱...

Building even a simple 𝗽𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻 𝗴𝗿𝗮𝗱𝗲 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝗔𝘂𝗴𝗺𝗲𝗻𝘁𝗲𝗱 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻 (𝗥𝗔𝗚) 𝗯𝗮𝘀𝗲𝗱 𝗔𝗜 𝘀𝘆𝘀𝘁𝗲𝗺 is a challenging task. Read until the end to understand why 👇

Here are some of the moving parts in the RAG based systems that you will need to take care of and continuously tune in order to achieve desired results:

𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹:

𝘍 ) Chunking - how do you chunk the data that you will use for external context.

- Small, Large chunks.
- Sliding or tumbling window for chunking.
- Retrieve parent or linked chunks when searching or just use originally retrieved data.

𝘊 ) Choosing the embedding model to embed and query and external context to/from the latent space. Considering Contextual embeddings.

𝘋 ) Vector Database.

- Which Database to choose.
- Where to host.
- What metadata to store together with embeddings.
- Indexing strategy.

𝘌 ) Vector Search

- Choice of similarity measure.
- Choosing the query path - metadata first vs. ANN first.
- Hybrid search.

𝘎 ) Heuristics - business rules applied to your retrieval procedure.

- Time importance.
- Reranking.
- Duplicate context (diversity ranking).
- Source retrieval.
- Conditional document preprocessing.

𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻:

𝘈 ) LLM - Choosing the right Large Language Model to power your application.

✅ It is becoming less of a headache the further we are into the LLM craze. The performance of available LLMs are converging, both open source and proprietary. The main choice nowadays is around using a proprietary model or self-hosting.

𝘉 ) Prompt Engineering - having context available for usage in your prompts does not free you from the hard work of engineering the prompts. You will still need to align the system to produce outputs that you desire and prevent jailbreak scenarios.

And let’s not forget the less popular part:

𝘏) Observing, Evaluating, Monitoring and Securing your application in production!

What other pieces of the system am I missing? Let me know in the comments 👇

#LLM #AI #MachineLearning