What You MUST Know About AI Engineering | Chip Huyen, Author of “AI Engineering”

Video Description

In this episode, we dive deep into the world of AI engineering with Chip Huyen, author of the excellent, newly released book "AI Engineering: Building Applications with Foundation Models". We explore the nuances of AI engineering, distinguishing it from traditional machine learning, discuss how foundational models make it possible for anyone to build AI applications, and cover many other topics including the challenges of AI evaluation, the intricacies of the generative AI stack, why prompt engineering is underrated, why the rumors of the death of RAG are greatly exaggerated, and the latest progress in AI agents. Book: https://www.oreilly.com/library/view/ai-engineering/9781098166298/ Chip Huyen Website - https://huyenchip.com LinkedIn - https://www.linkedin.com/in/chiphuyen Twitter/X - https://x.com/chipro FIRSTMARK Website - https://firstmark.com Twitter - https://twitter.com/FirstMarkCap Matt Turck (Managing Director) LinkedIn - https://www.linkedin.com/in/turck/ Twitter - https://twitter.com/mattturck LISTEN ON: Spotify - https://open.spotify.com/show/7yLATDSaFvgJG80ACcRJtq Apple - https://podcasts.apple.com/us/podcast/the-mad-podcast-with-matt-turck/id1686238724 00:00 - Intro 02:45 - What is new about AI engineering? 06:11 - The product-first approach to building AI applications 07:38 - Are AI engineering and ML engineering two separate professions? 11:00 - The Generative AI stack 13:00 - Why are language models able to scale? 14:45 - Auto-regressive vs. masked models 16:46 - Supervised vs. unsupervised vs. self-supervised 18:56 - Why does model scale matter? 20:40 - Mixture of Experts 24:20 - Pre-training vs. post-training 28:43 - Sampling 32:14 - Evaluation as a key to AI adoption 36:03 - Entropy 40:05 - Evaluating AI systems 43:21 - AI as a judge 46:49 - Why prompt engineering is underrated 49:38 - In-context learning 51:46 - Few-shot learning and zero-shot learning 52:57 - Defensive prompt engineering 55:29 - User prompt vs. system prompt 57:07 - Why RAG is here to stay 01:00:31 - Defining AI agents 01:04:04 - AI agent planning 01:08:32 - Training data as a bottleneck to agent planning