The moment we stopped understanding AI [AlexNet]
Welch Labs
@welchlabsAbout
New Book! The Welch Labs Illustrated Guide to AI is now available for pre-order: https://www.welchlabs.com/resources/ai-book
Latest Posts
Video Description
Thanks to KiwiCo for sponsoring today's video! Go to https://www.kiwico.com/welchlabs and use code WELCHLABS for 50% off your first month of monthly lines and/or for 20% off your first Panda Crate. Activation Atlas Posters! https://www.welchlabs.com/resources/5gtnaauv6nb9lrhoz9cp604padxp5o https://www.welchlabs.com/resources/activation-atlas-poster-mixed5b-13x19 https://www.welchlabs.com/resources/large-activation-atlas-poster-mixed4c-24x36 https://www.welchlabs.com/resources/activation-atlas-poster-mixed4c-13x19 Special thanks to the Patrons: Juan Benet, Ross Hanson, Yan Babitski, AJ Englehardt, Alvin Khaled, Eduardo Barraza, Hitoshi Yamauchi, Jaewon Jung, Mrgoodlight, Shinichi Hayashi, Sid Sarasvati, Dominic Beaumont, Shannon Prater, Ubiquity Ventures, Matias Forti Welch Labs Ad free videos and exclusive perks: https://www.patreon.com/welchlabs Watch on TikTok: https://www.tiktok.com/@welchlabs Learn More or Contact: https://www.welchlabs.com/ Instagram: https://www.instagram.com/welchlabs X: https://twitter.com/welchlabs References AlexNet Paper https://proceedings.neurips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf Original Activation Atlas Article- explore here - Great interactive Atlas! https://distill.pub/2019/activation-atlas/ Carter, et al., "Activation Atlas", Distill, 2019. Feature Visualization Article: https://distill.pub/2017/feature-visualization/ `Olah, et al., "Feature Visualization", Distill, 2017.` Great LLM Explainability work: https://transformer-circuits.pub/2024/scaling-monosemanticity/index.html Templeton, et al., "Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet", Transformer Circuits Thread, 2024. “Deep Visualization Toolbox" by Jason Yosinski video inspired many visuals: https://www.youtube.com/watch?v=AgkfIQ4IGaM Great LLM/GPT Intro paper https://arxiv.org/pdf/2304.10557 3B1Bs GPT Videos are excellent, as always: https://www.youtube.com/watch?v=eMlx5fFNoYc https://www.youtube.com/watch?v=wjZofJX0v4M Andrej Kerpathy's walkthrough is amazing: https://www.youtube.com/watch?v=kCc8FmEb1nY Goodfellow’s Deep Learning Book https://www.deeplearningbook.org/ OpenAI’s 10,000 V100 GPU cluster (1+ exaflop) https://news.microsoft.com/source/features/innovation/openai-azure-supercomputer/ GPT-3 size, etc: Language Models are Few-Shot Learners, Brown et al, 2020. Unique token count for ChatGPT: https://cookbook.openai.com/examples/how_to_count_tokens_with_tiktoken GPT-4 training size etc, speculative: https://patmcguinness.substack.com/p/gpt-4-details-revealed https://www.semianalysis.com/p/gpt-4-architecture-infrastructure Historical Neural Network Videos https://www.youtube.com/watch?v=FwFduRA_L6Q https://www.youtube.com/watch?v=cNxadbrN_aI Errata 1:40 should be: "word fragment is appended to the end of the original input". Thanks for Chris A for finding this one. CFAQJOTYQHT7JYIT
You May Also Like
Upgrade Your AI Learning
AI-recommended products based on this video

Skytech Archangel Gaming PC Desktop – AMD Ryzen 5 3600 3.6 GHz, NVIDIA RTX 3060, 1TB NVME SSD, 16GB DDR4 RAM 3200, 600W Gold PSU, 11AC Wi-Fi, Windows 11 Home 64-bit

Skytech Blaze 3.0 Gaming PC Desktop – Intel Core i5 12400F 2.5 GHz, NVIDIA RTX 3060, 500GB NVME SSD, 16GB DDR4 RAM 3200, 600W Gold PSU, 11AC Wi-Fi, Windows 11 Home 64-bit

NVIDIA RTX A2000 - Graphics Card - RTX A2000-6 GB GDDR6 - PCIe 4.0 x16-4 x Mini DisplayPort

NVIDIA RTX A4500

Freenove Ultimate Starter Kit for BBC micro bit (V2 Included), 316-Page Detailed Tutorial, 225 Items, 44 Projects, Blocks and Python Code

Magnetic Nasal Strips Starter Kit: Comfortable Nasal Breathing Support for Sleep, Helps Reduce Snoring Noise, Includes 60 Tabs (30 Uses) with 4 Sizes

Environet Hydroponic Growing Kit, Self-Watering Mason Jar Herb Garden Starter Kit Indoor, Windowsill Herb Garden, Grow Your Own Herbs from Organic Seeds (Basil)

Herb Garden Planter Indoor Kit 21Pcs Kitchen Herb Garden Starter Kit Growing Kit Including Wooden Box Burlap Pots Soil Discs Gardening Tools Unique Easter Birthday Christmas Gift Ideas for Women Mom

Bonsai Starter Kit – 1x Bonsai Tree | Complete Indoor Starter Kit for Growing Plants with Bonsai Seeds, Tools & Planters – Gardening Gifts for Women & Men

Asus Zenbook Duo (2025), AI, Dual 14” WUXGA OLED Touch Laptop, Intel Evo, Intel Core Ultra 9 285H CPU, Intel Arc, 16GB RAM, 1TB SSD, Wi-Fi 7, Windows 11 Home, Inkwell Gray, UX8406CA-DS91T-CA

Asus Zenbook S 14, Next-Level AI PC, 14” Ultrathin Laptop, 3K 120Hz OLED Touchscreen, Intel Core™ Ultra 7 (Series 2) Processor, 16GB Memory, 1TB SSD, Wi-Fi 7, Zumaia Gray, UX5406SA-DH71T-CB

Yiupea Yiupea Baby Weaning Bib With Long Sleeves - Machine Washable & Waterproof Mess Proof Bibs 6-24 Months for Led Weaning, Coverall Feeding Bibs - Light Yellow



![Why Deep Learning Works Unreasonably Well [How Models Learn Part 3]](https://imgz.pc97.com/?width=500&fit=cover&image=https://i.ytimg.com/vi/qx7hirqgfuU/hqdefault.jpg)
![The Misconception that Almost Stopped AI [How Models Learn Part 1]](https://imgz.pc97.com/?width=500&fit=cover&image=https://i.ytimg.com/vi/NrO20Jb-hy0/hqdefault.jpg)
![How DeepSeek Rewrote the Transformer [MLA]](https://imgz.pc97.com/?width=500&fit=cover&image=https://i.ytimg.com/vi/0VLAoVGf_74/hqdefault.jpg)
![ChatGPT is made from 100 million of these [The Perceptron]](https://imgz.pc97.com/?width=500&fit=cover&image=https://i.ytimg.com/vi/l-9ALe3U-Fg/hqdefault.jpg)
![The Dark Matter of AI [Mechanistic Interpretability]](https://imgz.pc97.com/?width=500&fit=cover&image=https://i.ytimg.com/vi/UGO_Ehywuxc/hqdefault.jpg)










![The most beautiful equation in math, explained visually [Euler’s Formula]](https://imgz.pc97.com/?width=500&fit=cover&image=https://i.ytimg.com/vi/f8CXG7dS-D0/hqdefault.jpg)
