Shannon Entropy and Information Gain
Serrano.Academy
@serranoacademyAbout
Welcome to Serrano.Academy! I'm Luis Serrano and I love demystifying concepts, capturing their essence, and sharing these videos with you. I prefer illustrations, analogies, and cartoons, rather than formulas (although we don't shy away from the math when needed). The topics I have are machine learning, mathematics (probability and statistics), but I'm open to many others. If you have any topics you'd like to suggest, feel free to add them in the comments or drop me a line! For more information, check out http://serrano.academy. And also check out my book! Grokking Machine Learning http://manning.com/books/grokking-machine-learning (40% discount code: serranoyt)
Latest Posts
Video Description
CORRECTION: at 13:41, the probability is 6.1e-5 and not 4.8e-4 (however, the entropy is 1.75, which is correct). Thank you @dlyChimi! Learn Shannon entropy and information gain by playing a game consisting in picking colored balls from buckets. Announcement: New Book by Luis Serrano! Grokking Machine Learning. bit.ly/grokkingML 40% discount code: serranoyt Accompanying blog post: https://medium.com/p/5810d35d54b4/ 0:00 Shannon Entropy and Information Gain 2:22 What ball will we pick? 4:33 Quiz 5:06 Question 5:14 Game 7:17 Probability of Winning 7:45 Products 11:00 What if there are more classes? 12:34 Sequence 2 13:44 Sequence 3 14:57 Naive Approach 15:34 Sequence 1 19:44 General Formula
Data Science Toolkit Essentials
AI-recommended products based on this video
NEEWER Advanced 18 inch LED Ring Light for Phone, LCD Touch Screen, 2.4G Remote Lights Control, 3200-5600K, Tripod Light for iPhone Action Camera, for Studio Makeup TikTok YouTube Video Salon (Black)
NEXPOW Car Jump Starter,Car Battery Jump Starter Pack 1500A Peak Q10S for Up to 7.0L Gas and 5.5L Diesel Engine12V Auto Battery Booster,Jumper Cables,Portable Lithium Jump Box with LED Light/USB QC3.0
Firefly Variety 8 Pack - Fire Starter Accessory for Swiss Army Victorinox Knives (Neon Green-Yellow Glow)
9-in-1 5000A 150PSI Car Battery Booster Jump Starter with Air Compressor (All Gas/9L Diesel), Portable Car Battery Booster Pack, Safe Durable Car Jump Starter with Extended Jumper Cables, Glove, Light
Laptop Sleeve Case 11–14 Inch MacBook Air MacBook Pro Surface Laptop Dell XPS 13 HP Envy Pavilion Lenovo Yoga Slim IdeaPad Acer Swift Chromebook Samsung Galaxy Book Shockproof Leather Bag Colour Blue
Charger for Dell Computer Inspiron XPS Laptop 65W 45W Power Supply AC Adapter for Dell-Inspiron 15-3000 15-5000 15-7000 11-3000 13-5000 13-7000 17-5000 XPS 13 Series 5559 5558 5755 5758 Laptop Charger
tomtoc 360° Protective Laptop Sleeve for 15-inch MacBook Air M4/A3241 2025, M3/A3114 2024, M2/A2941 2023, 15-inch MacBook Pro A1990 A1707, Dell XPS 15 Plus Laptop, Water-Resistant Computer Case Bag Global Recycled Standard
Replacement for Dell 130W Laptop Charger USB C - XPS 17 15 7590 9700 9500 9510 Precision 5560 3560 5540 5570 5550 3561 3550 5510 5520 Latitude 7410 7310 7210 Type C Computer AC Adapter Power Cord
Laptop Sleeve Case 11–14 Inch MacBook Air MacBook Pro Surface Laptop Dell XPS 13 HP Envy Pavilion Lenovo Yoga Slim IdeaPad Acer Swift Chromebook Samsung Galaxy Book Shockproof Leather Bag Colour Blue
Original HP 63 Black/Tri-color Ink (2-pack) | Works with HP DeskJet 1112, 2130, 3630 Series; HP ENVY 4510, 4520 Series; HP OfficeJet 3830, 4650, 5200 Series | Eligible for Instant Ink | L0R46AN
Original HP 63 Black Ink Cartridge | Works with HP DeskJet 1112, 2130, 3630 Series; HP ENVY 4510, 4520 Series; HP OfficeJet 3830, 4650, 5200 Series | Eligible for Instant Ink | F6U62AN ECOLOGO
