The math behind Attention: Keys, Queries, and Values matrices
Serrano.Academy
@serranoacademyAbout
Welcome to Serrano.Academy! I'm Luis Serrano and I love demystifying concepts, capturing their essence, and sharing these videos with you. I prefer illustrations, analogies, and cartoons, rather than formulas (although we don't shy away from the math when needed). The topics I have are machine learning, mathematics (probability and statistics), but I'm open to many others. If you have any topics you'd like to suggest, feel free to add them in the comments or drop me a line! For more information, check out http://serrano.academy. And also check out my book! Grokking Machine Learning http://manning.com/books/grokking-machine-learning (40% discount code: serranoyt)
Latest Posts
Video Description
Check out the latest (and most visual) video on this topic! The Celestial Mechanics of Attention Mechanisms: https://www.youtube.com/watch?v=RFdb2rKAqFw This is the second of a series of 3 videos where we demystify Transformer models and explain them with visuals and friendly examples. Video 1: The attention mechanism in high level https://youtu.be/OxCpWwDCDFQ Video 2: The attention mechanism with math (this one) Video 3: Transformer models https://www.youtube.com/watch?v=qaWMOYf4ri8 If you like this material, check out LLM University from Cohere! https://llm.university 00:00 Introduction 01:18 Recap: Embeddings and Context 04:46 Similarity 11:09 Attention 20:46 The Keys and Queries Matrices 25:02 The Values Matrix 28:41 Self and Multi-head attention 33:54: Conclusion
Upgrade Your Learning: Essential Tools
AI-recommended products based on this video
Google Pixel Buds Pro 2 - Noise Canceling Earbuds - Up to 31 Hour Battery Life with Charging Case - Bluetooth Headphones - Compatible with Android - Hazel
Deeyaple USB C to Aux, 4FT/1.2M, Type C to 3.5mm Audio Cable Headphone Jack Cable for Car Mobile Phone, iPhone 16 15, iPad Pro, Samsung Galaxy S24 S23 S2010, Google Pixel,Oneplus Grey (1)
Car Carplay Woven Cable for iPhone 16 15 3.3FT USB A to USB C 3.2 Gen 2 Carplay Adapter Wire for iPhone 16 15 Pro Max, iPad Pro/Air, Samsung Galaxy S25/S24/S23/S22/S21 Google Pixel, Car Charger Cable
Seasonic Focus V4 GX-1000 (ATX3) - 1000W - 80+ Gold - ATX 3.0 & PCIe 5.1 Ready -Full-Modular -ATX Form Factor -Premium Japanese Capacitor -10 Year Warranty -Nvidia RTX 30/40 Super & AMD GPU Compatible
PNY NVIDIA Quadro RTX 4000 - The World’S First Ray Tracing GPU
UCANUUP 130W Tip 4.5mm Compatible with Dell XPS 15 7590 9530 9550 9560 9570 9575 Inspiron 7348 Precision M3800 5510 5520 5530 06TTY6 AC Adapter Power Replacement Supply Cord
Laptop Sleeve Case 11–14 Inch MacBook Air MacBook Pro Surface Laptop Dell XPS 13 HP Envy Pavilion Lenovo Yoga Slim IdeaPad Acer Swift Chromebook Samsung Galaxy Book Shockproof Leather Bag Colour Blue
tomtoc 360° Protective Laptop Sleeve for 15-inch MacBook Air M4/A3241 2025, M3/A3114 2024, M2/A2941 2023, 15-inch MacBook Pro A1990 A1707, Dell XPS 15 Plus Laptop, Water-Resistant Computer Case Bag Global Recycled Standard
MOSISO 360 Protective Laptop Bag 15 inch, 15 inch Computer Shoulder Bag Compatible with MacBook Air 15 M4 M3 M2 2025-2023, Dell XPS 15, Side Open Messenger Case &4 Zipper Pockets&Handle, Black Global Recycled Standard
Logitech G203 Wired Gaming Mouse, 8,000 DPI, Rainbow Optical Effect LIGHTSYNC RGB, 6 Programmable Buttons, On-Board Memory, Screen Mapping, PC/Mac Computer and Laptop Compatible - Black
Logitech G305 Lightspeed Wireless Gaming Mouse, Hero 12K Sensor, 12,000 DPI, Lightweight, 6 Programmable Buttons, 250h Battery Life, On-Board Memory, PC/Mac - Black
Monster N-Lite 211 Wireless Earbuds， Bluetooth 6.0 Stereo Headphones with Noise-Cancelling Mic and LED Display, Built-in Mic for Clear Calls, Waterproof Design for Sports
