The Bias Variance Trade-Off

Video Description

The machine learning consultancy: https://truetheta.io Join my email list to get educational and useful articles (and nothing else!): https://mailchi.mp/truetheta/true-theta-email-list Want to work together? See here: https://truetheta.io/about/#want-to-work-together Article on the topic: https://truetheta.io/concepts/machine-learning-and-other-topics/bias-variance-trade-off/ The Bias Variance Trade-Off is an essential perspective for developing models that will perform well out-of-sample. In fact, it's so important for modeling that most hyperparameters are designed to move you between the high bias-low variance and low bias-high variance ends of the spectrum. In this video, I explain what it says exactly, how it works intuitively, and how it's used typically. SOCIAL MEDIA LinkedIn : https://www.linkedin.com/in/dj-rich-90b91753/ Twitter : https://twitter.com/DuaneJRich Enjoy learning this way? Want me to make more videos? Consider supporting me on Patreon: https://www.patreon.com/MutualInformation TIMESTAMPS 0:00 The Importance and my Approach 0:46 The Bias Variance Trade off at a High Level 2:06 A Supervised Learning Regression Task and Our Goal 3:41 Evaluating a Learning Algorithm 5:39 The Bias Variance Decomposition 7:19 An Example True Function 8:07 An Example Learning Algorithm 9:41 Seeing the Bias Variance Trade Off 12:59 Final Comments SOURCES The explanation I've reviewed the most is in section 2.9 of [1]. Also, I found Kilian Weinberger's excellent lecture [2] useful. If you'd like to learn how this concept generalizes beyond a regression model's square error, see [3] [1] Hastie, T., Tibshirani, R., & Friedman, J. H. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. 2nd ed. New York: Springer. [2] Weinberger, K. (2018). Machine Learning Lecture 19 Bias Variance Decomposition -Cornell CS4780 SP17, YouTube, https://www.youtube.com/watch?v=zUJbRO0Wavo&t=357s [3] Tibshirani, R (1996), Bias, variance and prediction error for classification rules. Department of Preventive Medicine and Biostatistics and Department of Statistics, University of Toronto, Toronto, Canada