Building the next generation of AI models: Rohan Taori | Researcher at Anthropic & Alpaca co-creator

Video Description

Welcome to AI in the Real World! In this episode, Foundation Capital Partner Jaya Gupta sits down with Rohan Taori, a researcher on Anthropic's multimodal pre-training team. Within the AI community, Rohan is best known for co-creating Alpaca, a project that demonstrated how fine-tuning Meta’s LLaMA model could achieve ChatGPT-level performance for under $600. Rohan shares his journey from early work in computer vision at UC Berkeley to his Ph.D. at Stanford, where he explored methods for making AI more accessible. He explains the technical breakthroughs behind Alpaca, including self-instruct, a method that uses a stronger language model (OpenAI’s text-davinci-003) to generate synthetic data that is then used to fine-tune a weaker model (Llama). This approach, which underpins Alpaca and its follow-up projects AlpacaFarm and AlpacaEval, illustrates how small-scale post-training can significantly enhance model performance. The conversation also covers: - The promise and challenges of synthetic data for training AI models - What it will take to build foundation models that are 100x better - The future of multimodal AI and why it matters - Why better evals are critical to the next wave of AI advances Chapters 00:00 Cold open 1:40 Rohan’s journey into AI research 04:50 Transitioning from vision research to LLMs 06:18 The story behind Alpaca 08:55 How Alpaca works 10:45 The AI community’s reception of Alpaca 12:26 The evolution of Alpaca related projects 14:22 The role of synthetic data 19:38 Challenges in multimodal AI 24:31 Future of foundation models 30:00 Importance of data in AI 34:48 Staying up to date with the latest AI research 36:12 Advice for founders