End To End Data Engineering Project Using Snowflake | Real Cricket Analytics Use Case

Video Description

🏏 Dive into the world of one day international (ODI) cricket data with my latest data engineering project, where we've built a comprehensive end to end data engineering solution using Snowflake. This video covers everything from design considerations to the intricacies of data flow and database layer/table architecture. 🔍 Design Considerations: Learn about the key decisions made in designing the data modelling and the overall project architecture. 🌐 Data Flow: From data loading and curation to transformation into Snowflake, understand the seamless flow of information. We'll break down the steps involved in ensuring data quality, integrity, and how Snowflake enhances the efficiency of this process. Once you complete this end to end real time snowflake based data engineering project (ETL/ELT), you would be able answer of following questions 1. How to load data from location machine to Snowflake stage location.(Small JSON files as well as many JSON files at on go). 2. How to load delta data set from local machine to internal stages? 3. How to use $ notation to query stage file? 4. How to run copy command to load JSON files into tables? 5. How to design different layers & fact/dimension tables? 6. How to automate data flow using task and task tree? 7. How to build quick dashboard using Snowsight? 🚀🚀 How To Connect ✏ Instagram: https://www.instagram.com/learn_dataengineering/ ✏ Medium Blog: https://medium.com/@data-engineering-simplified ✏ Exclusive Snowflake Facebook Group: https://www.facebook.com/groups/627874916138090/ 🚀🚀 Sample JSON Files & SQL Scripts ✏ https://medium.com/@data-engineering-simplified/8f8e4f0fd1d0 🚀🚀 Chapters ➥ 00:00:00 Introduction ➥ 00:01:56 Welcome Note ➥ 00:03:07 Udemy Course Offers ➥ 00:03:33 Review Match Level JSON File ➥ 00:06:19 Review E2E Data Flow Diagram ➥ 00:11:07 Part-01 - Pre Requisite ➥ 00:11:58 Part-02 - Database/Schema Creation ➥ 00:15:13 Part-02.1 - Load JSON Files Using SnowSight ➥ 00:17:35 Part-02.2 - Bulk Load JSON Files Using SnowSQL CLI ➥ 00:19:01 Part-03 - Stage to Raw Data Loading ➥ 00:28:46 Part-04 - Cleansing + Curation Introduction ➥ 00:36:09 Part-04.1 - Extract Clean Match Data ➥ 00:48:28 Part-04.2 - Extract Clean Player Data ➥ 00:54:01 Part-04.3 - Extract Clean Bowling Data ➥ 01:12:57 Step-05 - Quick Data Validation ➥ 01:15:22 Step-06 - Populate Fact & Dimension ➥ 01:41:36 Step-07 - Quick Visualisation ➥ 01:43:49 Step-08 - Automate Continuous Data Flow ➥ 01:50:12 Step-09 - Summary ➥ 01:51:00 Thank you note 🚀🚀 Other Popular Playlist ✏ Complete Snowflake Master Class - https://www.youtube.com/playlist?list=PLba2xJ7yxHB7SWc4Sm-Sp3uGN74ulI4pS ✏ Snowflake Productivity Hacks https://www.youtube.com/playlist?list=PLba2xJ7yxHB5W-n7DJyyLfUufoVEkNKq- ✏ Snowflake Cost Optimisation & Cost Management Techniques | Tips To Reduce Snowflake Cost https://www.youtube.com/playlist?list=PLba2xJ7yxHB4KI_FYWVxg4YFVxaKiynrZ ✏ Snowflake Table - Complete Guide With Example https://www.youtube.com/playlist?list=PLba2xJ7yxHB7nzbAHtfhU4pyLRdvDjSBr ✏ Snowflake SnowPro Core Certification - Complete Guide https://www.youtube.com/playlist?list=PLba2xJ7yxHB6-w7PHPi-wvj1SdbrIjxk2 ✏ SnowPro Core Exam Mock Test - https://www.youtube.com/playlist?list=PLba2xJ7yxHB5X2CMe7qZZu-V4LxNE1HbF ✏ Learn Snowpark Using Python API Examples - https://www.youtube.com/playlist?list=PLba2xJ7yxHB7RNDaCSrw3wqJwaxKgj1O5 ✏ Everything About Snowpark - https://www.youtube.com/playlist?list=PLba2xJ7yxHB4yPg3pUrobdzeMxk4mP24S ✏ Working With JSON Data In Snowflake - https://www.youtube.com/playlist?list=PLba2xJ7yxHB6ybgtaIsTKmmF2Nl2wAe2S ✏ How To Load Data Into Snowflake - Masterclass - https://www.youtube.com/playlist?list=PLba2xJ7yxHB6NPEv8pp7j3zWibLZzwjvO #dataengineering #snowflake #snowflaketutorial #snowflakedatawrehouse #snowflakecomputing #clouddatawarehouse #snowparktutorial Disclaimer: All snowflake-related learning materials and tutorial videos published in this channel are the personal opinions of the data engineering simplified team and they're neither authorised by nor associated with Snowflake, Inc.