Mastering Retrieval for LLMs - BM25, Fine-tuned Embeddings, and Re-Rankers

Video Description

➡️ Get Life-time Access to the Complete Scripts (and future improvements): https://Trelis.com/ADVANCED-fine-tuning ➡️ Runpod Affiliate Link: https://runpod.io?ref=jmfkcdio ➡️ One-click fine-tuning and API templates: https://github.com/TrelisResearch/one-click-llms ➡️ Newsletter: https://blog.Trelis.com ➡️ Resources/Support/Discord: https://Trelis.com/About VIDEO RESOURCES: - Slides: https://docs.google.com/presentation/d/1-hw1e1JgYkAxc-DTxBDmF1XXXTvWc4NtWnJ8-_yHsbQ/edit?usp=sharing - Sentence Bert Models: https://sbert.net/index.html - ALL-SORT Video (LLM retrieval): https://www.youtube.com/watch?v=biJmRQF8bmY TIMESTAMPS: 0:00 Mastering Retrieval (RAG) for LLMs 0:44 Video Overview 13:19 Baseline Performance with No Retrieval 17:29 Document Chunking - Naive vs Sentence based 24:34 BM25 33:20 Semantic / Vector / Embeddings Search 39:59 Cosine vs Dot Product Similarity 43:21 Generating Chunks and Embeddings 50:50 Running BM25 and Similarity Retrieval 55:22 Performance with BM25 vs Similarity 58:36 Fine-tuning embeddings / encoders 1:04:00 Preparing fine-tuning datasets 1:14:54 Embeddings Training Continued 1:22:00 Performance after Fine-tuning 1:25:58 Re-rankers 1:27:10: Cross-encoders 1:30:47 LLM re-rankers 1:36:11 Re-ranking performance 1:48:50 Final Tips