Large Language Models in Five Formulas

Video Description

Tutorial on building intuition about LLMs. Slides: https://link.excalidraw.com/p/readonly/aBWlNjEckdUlrszwwo6V or https://github.com/srush/LLM-Talk/blob/main/Tutorial.pdf 00:00 - Intro 02:15 - 1: Generation (Perplexity) 15:40 - 2: Memory (Attention) 28:00 - 3: Efficiency (GEMM) 38:40 - 4: Scaling (Chinchilla) 46:37 - 5: Reasoning (RASP) 55:33 - Conclusion Developed for an invited tutorial at the Harvard Data Science Initiative. Note: This tutorial is rather high-level and leaves out much of the scientific and citation history. There are other great guides that provide this in detail. My goal here was chalk-board level intuition.