Best Practices Working with Billion-row Tables in Databases
Hussein Nasser
@hnasrAbout
Join me as we discuss various software engineering topics with examples and add fun elements to them. We always try to learn by example here in this educational Youtube channel which we believe is the right way to learn. I love Software engineering (especially the backend) and I strive to be a better software engineer every day. Join me on this journey and let us learn together. All opinions, suggestions, and tips I provide in all of this channel's content are my own. Information provided is for educational purposes only. Consider becoming a member to support the channel https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join Check out my courses https://courses.husseinnasser.com Stay awesome! Hussein Nasser, http://www.husseinnasser.com Feel free to shoot me an email [email protected] twitter: @hnasr
Latest Posts
Video Description
In this video, I discuss 3 methods to work with tables of billion rows. This discussion is inspired by a comment on my YouTube video on Twitter System Design right here https://www.youtube.com/watch?v=gfq-LG9ZJQA&lc=UgyYbm5889dW0XtKhsV4AaABAg Chapters Intro 0:00 1. Brute Force Distributed Processing 2:30 2. Working with a Subset of table 3:35 2.1 Indexing 3:55 2.2 Partitioning 5:30 2.3 Sharding 7:30 3. Avoid it all together (reshuffle the whole design) 9:10 Summary 11:30 ๐๏ธListen to the Backend Engineering Podcast https://husseinnasser.com/podcast ๐ญ Backend Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQUNnO4p00ua_C5mKTfldiYT ๐พ Database Engineering Videos https://www.youtube.com/playlist?list=PLQnljOFTspQXjD0HOzN7P2tgzu7scWpl2 ๐ฐ Load Balancing and Proxies Videos https://www.youtube.com/playlist?list=PLQnljOFTspQVMeBmWI2AhxULWEeo7AaMC ๐๏ธ Software Archtiecture Videos https://www.youtube.com/playlist?list=PLQnljOFTspQXNP6mQchJVP3S-3oKGEuw9 ๐ฉ Messaging Systems https://www.youtube.com/playlist?list=PLQnljOFTspQVcumYRWE2w9kVxxIXy_AMo Become a Member https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join Support me on PayPal https://bit.ly/33ENps4 Stay Awesome, Hussein
You May Also Like
iPhone 14 Upgrade Essentials
AI-recommended products based on this video

BASESAILOR iPhone 16 16e Car Charger Adapter 3Pack,USB to USBC Connector Type C Female to A Male Cable Converter for iPhone 15 14 Pro Max Plus,Apple Watch 10,Airpods 4,iPad mini 7,Samsung Galaxy S25

URVNS 240W USB C Car Charger, Super Fast Charging 4-Ports PD3.1 140W/PD3.0 100W PPS 45W QC65W Type-C Car Adapter for MacBook Pro Laptop iPhone 15 14 Max iPad Samsung Galaxy S24/23/22 Pixel etc

UGREEN 130W Car Charger USB C Fast Charger PD3.0 QC4.0 PPS Fast Charging Car Adapter with 100W USB C Cable Compatible with Dell XPS, MacBook, iPhone 16 15 14 Pro Max, Galaxy S25 S24 Ultra, iPad Pro
![VANMASS [85+LBS Strongest Suction & Clip] 2025 Upgrade Car Phone Holder [Patent & Safety Certs] Cell Phone Mount Truck Stand for Dashboard Windshield Vent for iPhone 16 Pro Max 15 14 13 12 Samsung](https://m.media-amazon.com/images/I/71bHSP1cRkL._AC_UL960_FMwebp_QL65_.jpg)
VANMASS [85+LBS Strongest Suction & Clip] 2025 Upgrade Car Phone Holder [Patent & Safety Certs] Cell Phone Mount Truck Stand for Dashboard Windshield Vent for iPhone 16 Pro Max 15 14 13 12 Samsung

2Pack 10ft iPhone Charger 90 Degree Lightning Cable Right Angle Long USB Charging Cord Compatible with Apple iPhone 14 13 12 11 Pro Max XS XR X 8 7 6 5 Plus iPad iPod

Magnetic Wireless Charger for iPhone: Fodable 3 in 1 Charging Station for Multiple Apple Devices - Travel Charging Pad Dock for Apple Watch iPhone 16 15 14 13 12 Pro Max Plus & Airpod Global Recycled Standard

Magnetic Wireless Charger for iPhone - 3 in 1 Mag-Safe Charging Station for Multiple Devices Apple - Charger Stand Dock for Apple Watch iPhone 16 15 14 13 12 Pro Max Plus Airpods
![iPhone Charger [Apple MFi Certified] 3FT, 3Pack USB A to Lightning Cable Braided Fast Charging Apple Cord Compatible with iPhone 14 13 12 11 Pro Max XS XR X 8 7 6 5 Plus iPad/iPod/AirPods](https://m.media-amazon.com/images/I/71xZOlUELbL._AC_UL960_FMwebp_QL65_.jpg)
iPhone Charger [Apple MFi Certified] 3FT, 3Pack USB A to Lightning Cable Braided Fast Charging Apple Cord Compatible with iPhone 14 13 12 11 Pro Max XS XR X 8 7 6 5 Plus iPad/iPod/AirPods

for Magsafe Car Mountใ20 Strong Magnets ใMagnetic Phone Holder for Car Phone Holder Mount Dashใ360ยฐใCell Phone Mount for Car Accessories,Car Mount for iPhone 16 Pro Max 15 14 13 12 Plus MagSafe Cases

soundcore by Anker P20i True Wireless Earbuds, 10mm Drivers with Big Bass, Bluetooth 5.3, 30H Long Playtime, IPX5 Water-Resistant, 2 Mics for AI Clear Calls, 22 Preset EQs, Customization via App

Anker USB C Charger Cable (6ft 100W, 2Pack), USB 2.0 Type C Fast Charging Cable for iPhone 15 / 15Pro / 15Plus / 15ProMax MacBook Pro 2020, iPad Pro 2020, iPad Air 4, Samsung Galaxy S23+/S23 Ultra ClimatePartner certified

soundcore by Anker Q20i Hybrid Active Noise Cancelling Headphones, Wireless Over-Ear Bluetooth, 40H Long ANC Playtime, Hi-Res Audio, Big Bass, Customize via an App, Transparency Mode, Ideal for Travel

soundcore by Anker Q20i Hybrid Active Noise Cancelling Headphones, Wireless Over-Ear Bluetooth, 40H Long ANC Playtime, Hi-Res Audio, Big Bass, Customize via an App, Transparency Mode, Ideal for Travel

OHLPRO Tablet Holder for Car, iPad Holder for Car Air Vent Mount, Universal Dashboard Windshield 2-in-1 Cradle TPU Suction Sticky Gel for iPad/iPad mini Samsung Galaxy Size 6"- 10.5" iPad Car Holder

Magnetic Car Mount Air Vent Cell Phone Holder, Universal Vehicle Dash Dashboard Ac Airvent Cellphone Magic Stand with Clip Hook, for iPhone 16 15 Samsung Lenovo Google etc

AINOPE Phone Holder for Car Vent 2025 Gravity Air Vent Car Phone Holder Mount for iPhone 17 Pro Max Vent Clip Hand Free Cell Phone Car Mount for iPhone Air 17 16 Pro Max Samsung S25 Car Accesorries
![Miracase Car Phone Mount, [2025 Upgraded Strong Military Grade Suction Cup] Long Arm Phone Holder for Car Dashboard Windshield Vent, Hands Free Cell Phone Holder Compatible with All Mobile Phones Global Recycled Standard](https://m.media-amazon.com/images/I/81oehGW5OLL._AC_UL960_FMwebp_QL65_.jpg)
Miracase Car Phone Mount, [2025 Upgraded Strong Military Grade Suction Cup] Long Arm Phone Holder for Car Dashboard Windshield Vent, Hands Free Cell Phone Holder Compatible with All Mobile Phones Global Recycled Standard
![Spigen Case for Galaxy S25 Case, Rugged Armor MagFit [Fast Wireless Charging] Compatible with MagSafe - Black](https://m.media-amazon.com/images/I/81osE87mFrL._AC_UL960_FMwebp_QL65_.jpg)
Spigen Case for Galaxy S25 Case, Rugged Armor MagFit [Fast Wireless Charging] Compatible with MagSafe - Black
![Vihibii for Samsung Galaxy S25+ Plus Wallet Case with Card Holder (4 Cards) [RFID Blocking] & Slide Camera Cover & Raised Edges Protect Screen, Shockproof Rugged Case for S25+ Plus 6.7" 2025, Black](https://m.media-amazon.com/images/I/71YSDNV+PHL._AC_UL960_FMwebp_QL65_.jpg)
Vihibii for Samsung Galaxy S25+ Plus Wallet Case with Card Holder (4 Cards) [RFID Blocking] & Slide Camera Cover & Raised Edges Protect Screen, Shockproof Rugged Case for S25+ Plus 6.7" 2025, Black
![Vihibii for Samsung Galaxy S25+ Plus Wallet Case with Card Holder (4 Cards) [RFID Blocking] & Slide Camera Cover & Raised Edges Protection, Shockproof Rugged Case for S25+ Plus 6.7" 2025, Hot Pink](https://m.media-amazon.com/images/I/71K-GtDw48L._AC_UL960_FMwebp_QL65_.jpg)
Vihibii for Samsung Galaxy S25+ Plus Wallet Case with Card Holder (4 Cards) [RFID Blocking] & Slide Camera Cover & Raised Edges Protection, Shockproof Rugged Case for S25+ Plus 6.7" 2025, Hot Pink
![Spigen Case for iPhone 16 Pro Max Case, Rugged Armor [MagFit] Compatible with MagSafe - Matte Black](https://m.media-amazon.com/images/I/71Gh0VOQtlL._AC_UL960_FMwebp_QL65_.jpg)
Spigen Case for iPhone 16 Pro Max Case, Rugged Armor [MagFit] Compatible with MagSafe - Matte Black

Spigen Case for iPhone 17 Case, Liquid Air Designed for Apple iPhone 17 Case (2025) - Matte Black ClimatePartner certified

Spigen Case for iPhone 16 Case, Liquid Air Armor Designed for Apple iPhone 16 - Matte Black

Anker USB C Charger Cable (6ft 100W, 2Pack), USB 2.0 Type C Fast Charging Cable for iPhone 15 / 15Pro / 15Plus / 15ProMax MacBook Pro 2020, iPad Pro 2020, iPad Air 4, Samsung Galaxy S23+/S23 Ultra ClimatePartner certified
![Anker USB C Cable, [2-Pack, 6 ft] Type C Charger Premium Nylon USB Cable, USB A to Type C Charging Cable Fast Charge for Samsung Galaxy S10 S10+ / Note 8, LG V20 and Other USB C Charger (Black)](https://m.media-amazon.com/images/I/71N7xhCJnoL._AC_UL960_FMwebp_QL65_.jpg)
Anker USB C Cable, [2-Pack, 6 ft] Type C Charger Premium Nylon USB Cable, USB A to Type C Charging Cable Fast Charge for Samsung Galaxy S10 S10+ / Note 8, LG V20 and Other USB C Charger (Black)

Anker Nano USB C Wall Charger,45W Fast Charging Smart Display Charger,with 180ยฐFoldable Plug,Smart Recognition,Built-in Care Mode,for iPhone17/16/15 (Non-Battery,One USB-C Port,No Cable Included) ClimatePartner certified

BASESAILOR iPhone 16 16e Car Charger Adapter 3Pack,USB to USBC Connector Type C Female to A Male Cable Converter for iPhone 15 14 Pro Max Plus,Apple Watch 10,Airpods 4,iPad mini 7,Samsung Galaxy S25

URVNS 240W USB C Car Charger, Super Fast Charging 4-Ports PD3.1 140W/PD3.0 100W PPS 45W QC65W Type-C Car Adapter for MacBook Pro Laptop iPhone 15 14 Max iPad Samsung Galaxy S24/23/22 Pixel etc

UGREEN 130W Car Charger USB C Fast Charger PD3.0 QC4.0 PPS Fast Charging Car Adapter with 100W USB C Cable Compatible with Dell XPS, MacBook, iPhone 16 15 14 Pro Max, Galaxy S25 S24 Ultra, iPad Pro
![VANMASS [85+LBS Strongest Suction & Clip] 2025 Upgrade Car Phone Holder [Patent & Safety Certs] Cell Phone Mount Truck Stand for Dashboard Windshield Vent for iPhone 16 Pro Max 15 14 13 12 Samsung](https://m.media-amazon.com/images/I/71bHSP1cRkL._AC_UL960_FMwebp_QL65_.jpg)
VANMASS [85+LBS Strongest Suction & Clip] 2025 Upgrade Car Phone Holder [Patent & Safety Certs] Cell Phone Mount Truck Stand for Dashboard Windshield Vent for iPhone 16 Pro Max 15 14 13 12 Samsung

soundcore by Anker P20i True Wireless Earbuds, 10mm Drivers with Big Bass, Bluetooth 5.3, 30H Long Playtime, IPX5 Water-Resistant, 2 Mics for AI Clear Calls, 22 Preset EQs, Customization via App

soundcore P31i by Anker, Real-Time Adaptive Noise Cancelling, Hi-Res Sound, Translation Earbuds, 50H Playtime, Wireless Earbuds, Bluetooth Earphones, Spatial Audio, Fast Charging, IP55 ClimatePartner certified

soundcore by Anker Q20i Hybrid Active Noise Cancelling Headphones, Wireless Over-Ear Bluetooth, 40H Long ANC Playtime, Hi-Res Audio, Big Bass, Customize via an App, Transparency Mode, Ideal for Travel

soundcore by Anker Q20i Hybrid Active Noise Cancelling Headphones, Wireless Over-Ear Bluetooth, 40H Long ANC Playtime, Hi-Res Audio, Big Bass, Customize via an App, Transparency Mode, Ideal for Travel

Apple AirPods Pro 2 Wireless Earbuds, Bluetooth Headphones, Active Noise Cancellation, Transparency, Personalized Spatial Audio, High-Fidelity Sound, H2 Chip, USB-C Charging

3 in 1 Charging Station for iPhone, Wireless Charger for iPhone 16 15 14 13 12 11 X Pro Max & Apple Watch - Charging Stand Dock for AirPods Global Recycled Standard

3 in 1 Charging Station for iPhone, Wireless Charger for iPhone 16 15 14 13 12 11 X Pro Max & Apple Watch - Charging Stand Dock for AirPods (Blue) Global Recycled Standard
![iPhone 16 15 Charger [Apple MFi Certified] 2 Pack USB C Wall Charger Block 6FT USB C to Type C Cable Compatible with iPhone 16 15 Pro Max,16e, AirPods Pr, iPad Pro12.9/11, Samsung/Android Phones](https://m.media-amazon.com/images/I/51GjkZn3ACL._AC_UL960_FMwebp_QL65_.jpg)
iPhone 16 15 Charger [Apple MFi Certified] 2 Pack USB C Wall Charger Block 6FT USB C to Type C Cable Compatible with iPhone 16 15 Pro Max,16e, AirPods Pr, iPad Pro12.9/11, Samsung/Android Phones

BASESAILOR iPhone 16 16e Car Charger Adapter 3Pack,USB to USBC Connector Type C Female to A Male Cable Converter for iPhone 15 14 Pro Max Plus,Apple Watch 10,Airpods 4,iPad mini 7,Samsung Galaxy S25

URVNS 240W USB C Car Charger, Super Fast Charging 4-Ports PD3.1 140W/PD3.0 100W PPS 45W QC65W Type-C Car Adapter for MacBook Pro Laptop iPhone 15 14 Max iPad Samsung Galaxy S24/23/22 Pixel etc

UGREEN 130W Car Charger USB C Fast Charger PD3.0 QC4.0 PPS Fast Charging Car Adapter with 100W USB C Cable Compatible with Dell XPS, MacBook, iPhone 16 15 14 Pro Max, Galaxy S25 S24 Ultra, iPad Pro
![VANMASS [85+LBS Strongest Suction & Clip] 2025 Upgrade Car Phone Holder [Patent & Safety Certs] Cell Phone Mount Truck Stand for Dashboard Windshield Vent for iPhone 16 Pro Max 15 14 13 12 Samsung](https://m.media-amazon.com/images/I/71bHSP1cRkL._AC_UL960_FMwebp_QL65_.jpg)



















