How to build an ETL pipeline with Python | Data pipeline | Export from SQL Server to PostgreSQL
BI Insights Inc
View ChannelAbout
Hi, I am Haq, a Data Analyst and Engineer. On this channel I talk/share tips and tricks on Business Intelligence tools, Data, Data Warehouse and Source Control. I have been involved in many Data Analytics and Process Automation projects. I am passionate about finding new ways and techniques to enhance the Data Analysis process. On this channel I create tutorials for people who: • are looking for a way to learn and use the power of Python in their day-to-day work. • are looking to automate Data Analysis processes using PowerBI, Cognos and Python • want to learn Data Analysis & Data Science to perform impactful analysis. • are working with Data and want to find new ways to of doing things • are looking to use Source Control to house their code If this sounds exciting to you, consider subscribing and turning on the notifications, so you don’t miss any content. Happy Coding everyone. May the power of the code be with you!
Latest Posts
No results found. Try different keywords.
Video Description
In this video, we will discuss what ETL is. ETL stands for Extract, Transform, Load. ETL is a set of processes that extracts data from one or more sources (API, a database or a file), transforms it to match the destination system’s requirements and loads it into the destination system. In this tutorial we will see how to design ETL Pipeline with Python. We will use SQL Server’s AdventureWorks database as a source and load data in PostgreSQL with Python. We often have requirements for data analytics and/or reporting projects that requires optimized data for querying. ETL moves and amalgamate the data from various sources and stores in the destination where it is available for data analytics, reporting. Furthermore, data scientist and data analysts can use this optimized data come up with new findings. Link to medium article on this topic: https://medium.com/@hnawaz100/how-to-build-an-etl-pipeline-with-python-1b78407c3875 Link to Microsoft ODBC driver: https://www.microsoft.com/en-us/download/details.aspx?id=36434 Link to ETL Script: https://github.com/hnawaz007/pythondataanalysis/blob/main/ETL%20Pipeline/build_etl_pipeline_python.py Link to GitHub repo: https://github.com/hnawaz007/pythondataanalysis/tree/main/ETL%20Pipeline Link to the Python ETL series: https://hnawaz007.github.io/etl.html How to connect to SQL Server via Python: https://www.youtube.com/watch?v=zdezE6TWSQQ&t SQL Server install video: https://www.youtube.com/watch?v=e5mvoKuV3xs&t PostgreSQL Install video: https://www.youtube.com/watch?v=fjYiWXHI7Mo&t Subscribe to our channel: https://www.youtube.com/c/HaqNawaz --------------------------------------------- Follow me on social media! Github: https://github.com/hnawaz007 Instagram: https://www.instagram.com/bi_insights_inc LinkedIn: https://www.linkedin.com/in/haq-nawaz/ --------------------------------------------- #ETL #Python #SQL Topics covered in this video: 0:00 - Introduction to ETL 1:55 - PostgreSQL Setup 2:49 - Sqlserver Setup 3:32 - Code ETL Pipeline 5:10 - Data Extract from SQL Server 7:30 - Load data to PostgreSQL 9:22 - Test ETL Pipeline
Upgrade Your Camera Gear Today
AI-recommended products based on this video
