Extract and Load from External API to Lakehouse using Data Pipelines (Microsoft Fabric)

Video Description

10+ hours of FREE Fabric Training: https://www.skool.com/microsoft-fabric/classroom/d154aad4?md=3b108b0e216c46c88d891407ccd8647b Looking for Fabric consultancy? Fill in this form: https://forms.office.com/e/h5FaVBwQ7s #datapipeline #microsoftfabric #lakehouse In this video, I walk through how to query an external API using a Data Pipeline, extracting some JSON data and loading it into a Lakehouse in Microsoft Fabric. This video is the first in an End-To-End Microsoft Fabric Project, you can follow the series via this Playlist: https://www.youtube.com/playlist?list=PLug2zSFKZmV1BHk129X_2xi53ZRwQaTkb In my experience, this is one of THE most important and impactful things you can learn as a data engineer. Next parts of the series include: - Transforming data in the Lakehouse using Data Engineering Notebooks. - Visualising the data in Power BI. --LINK TO OPENWEATHERMAP-- Here's the API used in this tutorial: https://openweathermap.org/api --OTHER VIDEOS YOU MIGHT LIKE-- LAKEHOUSE - https://youtu.be/wb4HzFoqsU8 Data warehouse - https://youtu.be/UFF_QpHJJNQ Data pipelines - https://youtu.be/suL76brljZI End-to-end data flows project - https://youtu.be/ZmgZPhl2LRU OneLake - https://youtu.be/hQs_JxCmL0U --TIMELINE-- 0:00 Intro 1:27 Exploring the API 4:10 Looking at the Lakehouse Destination 4:56 Creating the data pipeline 6:43 Source configuration 12:56 Destination configuration 15:28 Running the pipeline 16:00 Summary 6:14 Creating tables 8:09 Saving & sharing queries 10:03 Insert data 10:23 Selecting (viewing) data 10:58 Caution 11:36 Joining data 12:36 Creating views 14:42 Cross data warehouse querying 17:09 Summary --LINKEDIN-- Not following the LinkedIn page yet? Here's the link: https://www.linkedin.com/company/learnmicrosoftfabric/ --ABOUT WILL-- Hi, I'm Will! I'm hugely passionate about data and using it to create a better world. I currently work as a Consultant, focusing on Data Strategy, Data Engineering and Business Intelligence (within the Microsoft/Azure/Fabric environment). I have previously worked as a Data Scientist. I started Learn Microsoft Fabric to share my learnings on how Microsoft Fabric works and help you build your career and build meaningful things in Fabric. --SUBSCRIBE-- Not subscribed yet? You should! There are lots of new videos in the pipeline covering all aspects of Microsoft Fabric.