@akshay_pachaar: Let's compare DeepSeek-R1 and ...
@akshay_pachaar
29 views
Jan 27, 2025
1
Let's compare DeepSeek-R1 and OpenAI-o1 using RAG:
2
DeepSeek-R1 delivers OpenAI-o1 level intelligence at 90% less cost.
Today, we build a Streamlit app to compare and evaluate them using RAG.
Tech stack:
- @Llama_Index for orchestration
- @Cometml Opik for evaluation
- @Streamlit for the UI
Let's go! 🚀
Today, we build a Streamlit app to compare and evaluate them using RAG.
Tech stack:
- @Llama_Index for orchestration
- @Cometml Opik for evaluation
- @Streamlit for the UI
Let's go! 🚀
3
The architecture presented below illustrates some of the key components & how they interact with each other!
For those who are new, I've provided detailed descriptions & code for each component.
For those who are new, I've provided detailed descriptions & code for each component.
4
1️⃣ & 2️⃣ : Loading the knowledge base
A knowledge base is a collection of relevant and up-to-date information that serves as a foundation for RAG. In our case it's the docs stored in a directory.
Here's how you can load it as document objects in LlamaIndex:
A knowledge base is a collection of relevant and up-to-date information that serves as a foundation for RAG. In our case it's the docs stored in a directory.
Here's how you can load it as document objects in LlamaIndex:
5
3️⃣ The embedding model
The embedding model Embedding is a meaningful representation of text in form of numbers.
The embedding model is responsible for creating embeddings for the document chunks & user queries.
The embedding model Embedding is a meaningful representation of text in form of numbers.
The embedding model is responsible for creating embeddings for the document chunks & user queries.
6
4️⃣ Indexing & storing
Embeddings created by embedding model are stored in a vector store that offers fast retrieval and similarity search by creating an index over our data.
By default, LlamaIndex provides a in-memory vector store that’s great for quick experimentation.
Embeddings created by embedding model are stored in a vector store that offers fast retrieval and similarity search by creating an index over our data.
By default, LlamaIndex provides a in-memory vector store that’s great for quick experimentation.
7
5️⃣ Creating a prompt template
Creating a prompt template A custom prompt template is use to refine the response from LLM & include the context as well:
Creating a prompt template A custom prompt template is use to refine the response from LLM & include the context as well:
8
6️⃣ Setting up a query engine
The query engine takes a query string & use it to fetch relevant context and then sends them both as a prompt to the LLM to generate a final natural language response.
Here's how you set it up:
The query engine takes a query string & use it to fetch relevant context and then sends them both as a prompt to the LLM to generate a final natural language response.
Here's how you set it up:
9
8️⃣ The Chat interface
We create a UI using Streamlit to provide a chat interface for our RAG application.
The code for this & all we discussed so far is shared in the next tweet!
Check this out👇
We create a UI using Streamlit to provide a chat interface for our RAG application.
The code for this & all we discussed so far is shared in the next tweet!
Check this out👇
10
Finally, we will conduct a proper evaluation.
For this, we'll use @Cometml's Opik, a 100% open-source platform for evaluation and observability.
I have shared a notebook where you'll find all the code for this evaluation.
For this, we'll use @Cometml's Opik, a 100% open-source platform for evaluation and observability.
I have shared a notebook where you'll find all the code for this evaluation.
11
I used @LightningAI ⚡️ Studio for developing this application!
The studio reads like a blog, encapsulating all my code & environment to run it!
Clone a FREE studio now & take it for a spin...👇
lightning.ai/akshay-ddods/s…
The studio reads like a blog, encapsulating all my code & environment to run it!
Clone a FREE studio now & take it for a spin...👇
lightning.ai/akshay-ddods/s…
12
If you're interested in:
- Python 🐍
- Machine Learning 🤖
- AI Engineering ⚙️
Find me → @akshay_pachaar ✔️
Cheers! 🥂
- Python 🐍
- Machine Learning 🤖
- AI Engineering ⚙️
Find me → @akshay_pachaar ✔️
Cheers! 🥂