Avi Chawla (@_avichawla)

8 techniques to make an LLM reason better at inference time, covered with tradeoffs and practical notes. Every one of them is running in production at a frontier lab today, and the research behind them comes from Google, OpenAI, and Anthropic....

The technique behind vLLM's 23x throughput jump and the default scheduler in every serving engine you use. Covered with internals, the token budget, KV allocation, and the preemption that makes you pay for the same prefill twice....

Every generate() call to an LLM runs two distinct computational phases on the same GPU:...

- Google Maps uses graph ML to predict ETA - Netflix uses graph ML in recommendation - Spotify uses graph ML in recommendation - Pinterest uses graph ML in recommendation Here are 6 must-know ways for graph feature engineering (with code):...

Fine-tuning LLM Agents without Fine-tuning LLMs! Imagine improving your AI agent's performance from experience without ever touching the model weights. It's just like how humans remember past episodes and learn from them. That's precisely what Memento does. The core concept: Instead of updating...

KV caching in LLMs, clearly explained (with visuals):...

The growth of LLM context length with time: - GPT-3.5-turbo → 4k tokens - OpenAI GPT4 → 8k tokens - Claude 2 → 100k tokens - Llama 3 → 128k tokens - Gemini → 1M tokens Let's understand how they extend the context length of LLMs:...

A new embedding model cuts vector DB costs by ~200x. It also outperforms OpenAI and Cohere models. Here's a complete breakdown (with visuals):...

An MCP server that makes anyone a database engineer (open-source)! @MongoDB just released an MCP Server that lets AI tools like Claude, Cursor, and GitHub Copilot talk directly to a MongoDB deployment. That means anyone (technical or non-technical) can now say: - “Show me the most active users” -...