Snowflake cloud database with ELT(Airflow+Python+Talend)

Snowflake cloud database with ELT(Airflow+Python+Talend), available at $79.99, has an average rating of 4.2, with 144 lectures, based on 98 reviews, and has 1321 subscribers.

You will learn about Leveraging snowflake cloud data warehouse using Talend. Talend basics. Airflow basics. Building airflow Dags. Connect snowflake with python. Auditing snowflake commands. Capturing cost and performance metrics. Virtual machine with preconfigured airflow. End to End project to load Nyc traffic data(250 + GB ). This course is ideal for individuals who are Developers interested to know how to build workflows for snowflake. It is particularly useful for Developers interested to know how to build workflows for snowflake.

Enroll now: Snowflake cloud database with ELT(Airflow+Python+Talend)

Summary

Title: Snowflake cloud database with ELT(Airflow+Python+Talend)

Price: $79.99

Average Rating: 4.2

Number of Lectures: 144

Number of Published Lectures: 103

Number of Curriculum Items: 167

Number of Published Curriculum Objects: 103

Original Price: $19.99

Quality Status: approved

Status: Live

What You Will Learn

Leveraging snowflake cloud data warehouse using Talend.

Talend basics.

Airflow basics.

Building airflow Dags.

Connect snowflake with python.

Auditing snowflake commands.

Capturing cost and performance metrics.

Virtual machine with preconfigured airflow.

End to End project to load Nyc traffic data(250 + GB ).

Who Should Attend

Developers interested to know how to build workflows for snowflake.

Target Audiences

Developers interested to know how to build workflows for snowflake.

In our previous course Snowflake Masterclass [Real time demos+Best practices+Labs] we deep-dived and understood the fundamentals of snowflake, solved lot of assignments, and understood best practices to load data and unload data.

Also, we closely evaluated most of the snowflake features understanding how they work under the hood. Through these discussions, you realized how to use Snowflake efficiently.

There was one missing piece, How to build and orchestrate ETL workflows on Snowflake. This course is just about that.

In this course, we are going to learn,

1. Build workflows in Airflow.

2. We will leverage Talend capabilities to build generic code to ingest data and process data in snowflake.       

3. We will build audit tables and record every command we fire on the snowflake. We will record the time consumed for each task and capture snowflake credits.

4. Once we build the framework we will build a workflow to process and transform 250 + GB volume of NYC traffic data.

5. At last, we will connect the Snowflake with python and write code to capture stats of data we loaded to the snowflake.

6. you will also get access to preconfigured Jupyter notebook to run your python code on the Snowflake.

If you have previously not worked with Talend, Airflow and Python don’t worry they are very simple tools I will provide the necessary introduction.

I am sure you will learn a lot from this journey. See you in the course!!

Course Curriculum

Chapter 1: /***** Module 1 *****/

Lecture 1: Module 1 Intro

Chapter 2: prerequisites

Lecture 1: Setup snowflake environment

Lecture 2: Virtual machine download

Lecture 3: Mount local folder to vm and start vm

Lecture 4: Setup Vs code for VM

Lecture 5: Configure connection details

Lecture 6: Github link

Chapter 3: Airflow intro

Lecture 1: Airflow introduction

Lecture 2: Airflow architecture

Lecture 3: First Dag

Lecture 4: Introduction to metadata DB

Lecture 5: Airflow scheduler

Lecture 6: Scheduling in airflow

Lecture 7: Executors in airflow

Lecture 8: Airflow webserver

Lecture 9: Handle failures

Chapter 4: Snowflake airflow ELT

Lecture 1: Introduction

Lecture 2: Load to snowflake example 1

Lecture 3: [Demo] Load to snowflake example 1

Lecture 4: Load to snowflake example 2

Lecture 5: Load to snowflake example 3

Chapter 5: Snowflake python api intro Basics

Lecture 1: Introduction

Lecture 2: Connection test 1

Lecture 3: Connection test 2

Lecture 4: execute method

Lecture 5: cursor attributes

Lecture 6: execute many

Lecture 7: execute async part 1

Lecture 8: execute async part 2

Lecture 9: execute async part 3

Lecture 10: Formatting: pyformat

Lecture 11: Formatting: qmark

Lecture 12: Result metadata

Lecture 13: Type code

Chapter 6: Snowflake python api intro Advanced

Lecture 1: Fetch pandas arrow

Lecture 2: Context manager

Lecture 3: Result batch

Lecture 4: Get result batch

Lecture 5: Fetch arrow batches

Lecture 6: Fetch pandas batches

Lecture 7: To pandas arrow

Lecture 8: Execute stream

Lecture 9: Execute string

Lecture 10: Cursor dict

Chapter 7: Snowflake python ELT

Lecture 1: Copy data : s3 to snowflake

Lecture 2: Copy data : s3 to snowflake ( parallel )

Lecture 3: Execute ELT query : Python way

Lecture 4: Schedule airflow job : For python code

Chapter 8: Talend introduction.

Lecture 1: Talend as open source ETL tool.

Lecture 2: Download and install talend.

Lecture 3: Talend workspace introduction.

Lecture 4: Build first job and configure talend

Lecture 5: Build sample job : Level 1

Lecture 6: Build sample job : Level 2

Lecture 7: Build sample job : Level 3

Lecture 8: Building sample job : Level4

Lecture 9: Build sample job : Level5

Lecture 10: Build sample job : Schedule using airflow

Lecture 11: Summary

Chapter 9: / **** Module 2 **** /

Lecture 1: Module 2 intro

Chapter 10: Create generic components.

Lecture 1: Analyst requirement.

Lecture 2: S3 to snowflake copy app demo

Lecture 3: Requirement to process data.

Lecture 4: Elt app demo

Lecture 5: Scheduling job.

Lecture 6: Writing generic dag code part1

Lecture 7: Writing generic dag code part2

Lecture 8: Writing generic dag code part3

Chapter 11: Building workflow

Lecture 1: Introduction

Lecture 2: High level design

Lecture 3: Copy data from s3

Lecture 4: Parsing data

Lecture 5: Audit src records

Lecture 6: Archive and create view part 1

Lecture 7: Archive and create view part 2

Lecture 8: Score records part 1

Lecture 9: Score records part 2

Lecture 10: prepare dag code

Lecture 11: Execute airflow job

Lecture 12: Improve the approach

Lecture 13: Parameter registry

Lecture 14: Summary

Chapter 12: Nyc traffic data demo

Lecture 1: Introduction.

Lecture 2: Nyc traffic data overview

Lecture 3: Load sample data

Lecture 4: Nyc full data load

Lecture 5: Snow sight Dashboard

Chapter 13: /****Bonus sections starts****/

Instructors

Pradeep H C
Data engineer

Rating Distribution

1 stars: 4 votes

2 stars: 1 votes

3 stars: 10 votes

4 stars: 27 votes

5 stars: 56 votes

Frequently Asked Questions

How long do I have access to the course materials?

You can view and review the lecture materials indefinitely, like an on-demand channel.

Can I take my courses with me wherever I go?

Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don’t have an internet connection, some instructors also let their students download course lectures. That’s up to the instructor though, so make sure you get on their good side!