Self-Discover in DSPy with Chris Dossman - Weaviate Podcast #90!

Video Description

Hey everyone! Thank you so much for watching the 90th Weaviate Podcast with Chris Dossman! Self-Discover is a fascinating new algorithm from researchers at Google DeepMind and USC that searches for "atomic reasoning modules". One of the quickest ways to improve your LLM programs is to add Chain-of-Thought, "Let's think step by step ...", another primitive like this could be "Let's think out of the box" or "Think symbolically to calculate an answer...". Self-Discover searches for these thinking primitives that work best for YOUR LLM programs!! Learn more about Dicer.ai: https://www.dicer.ai/ Chris’ implementation of Self-Discover in DSPy! https://colab.research.google.com/drive/1GkAQKmw1XQgg5UNzzy8OncRe79V6pADB?usp=sharing Self-Discover: https://arxiv.org/pdf/2402.03620.pdf Example of combining You Web Search and Weaviate with DSPy: https://github.com/weaviate/recipes/blob/main/integrations/you/Web-Research-Agent-with-Weaviate-and-You.ipynb Chapters 0:00 Welcome Chris! 1:35 Chris’ Entrepreneurial Journey, FAQx and Dicer AI 8:42 Existential Risk of DSPy 12:42 LangChain vs. DSPy 16:14 Q for Sam Altman 19:10 Lessons from Jason Liu 21:30 DSPy Compilers 25:30 DSPy Discord 26:05 Atomic Reasoning Modules 29:45 Compiling Reasoning for a Particular Task 35:48 Prior Knowledge in AI 42:25 In-Context Vectors 50:24 Dicer AI 57:58 People Driven Development