LLMs Don’t Calculate - They Just Remember Everything

Video Description

I spent a week inside GPT-OSS-20B extracting how it does arithmetic. What I found: it doesn't compute. It remembers. Everything. Every operation is a lookup table. Change one character in your input — different answer. Train on 2-9, test on 11 — catastrophic failure. Single-digit and two-digit multiplication use completely different neurons (89° apart). Addition and subtraction aren't even opposites. This isn't a calculator. It's a filing cabinet. I provide the first causal proof: steering a single neuron changes whether the model looks up an answer or decomposes the problem. You can watch the intermediate steps appear in the layers. All findings are reproducible. Commands in pinned comment. — TIMESTAMPS 0:00 - Cold Open: Did it compute that, or remember it? 00:48 - Discovery: Watching math happen in the layers 07:40 - The layers of gpt-oss 20b 09:50 - Six Strategies: All variations of lookup 17:50 - The Lookup Table: Works → Fails → The Format Twist 22:43 - Four Operations: All separate, all orthogonal 24:00 - Single vs Two-Digit: No composition 27:54 - Neurons: The "Is This Memorized?" Detector 31:00 - Steering: Causal proof 34:00 - Conclusion — KEY FINDINGS - Every arithmetic operation is a separate lookup table (~80° apart) - Zero neuron overlap between operations - Addition and subtraction aren't opposites (80°, not 180°) - Single-digit doesn't help two-digit (89° apart, different neurons) - Exact string matching: "7*8=" works, "7*8" fails - OOD failure: Train on 2-9 (MAE 0.00), Test on 1/10/11 (MAE 28) - Causal steering: Push one neuron, change the strategy — TOOLS & RESOURCES - Lazarus (circuit extraction): https://github.com/chrishayuk/chuk-lazurus - GPT-OSS-20B — RELATED WORK This extends prior mechanistic interpretability research such as logit lens from Neel Nanda — #MachineLearning #AI #MechanisticInterpretability #LLM #Mathematics