Databricks Master Azure Databricks for Data Engineers

Databricks – Master Azure Databricks for Data Engineers, available at $54.99, has an average rating of 4.68, with 91 lectures, based on 990 reviews, and has 8407 subscribers.

You will learn about Databricks in Azure Cloud Working with DBFS and Mounting Storage Unity Catalog – Configuring and Working Unity Catalog User Provisioning and Security Working with Delta Lake and Delta Tables Manual and Automatic Schema Evolution Incremental Ingestion into Lakehouse Databricks Autoloader Delta Live Tables and DLT Pipelines Databricks Repos and Databricks Workflow Databricks Rest API and CLI Capstone Project This course is ideal for individuals who are Data Engineers or Data Engineering Solution Architects It is particularly useful for Data Engineers or Data Engineering Solution Architects.

Enroll now: Databricks – Master Azure Databricks for Data Engineers

Summary

Title: Databricks – Master Azure Databricks for Data Engineers

Price: $54.99

Average Rating: 4.68

Number of Lectures: 91

Number of Published Lectures: 91

Number of Curriculum Items: 91

Number of Published Curriculum Objects: 91

Original Price: $99.99

Quality Status: approved

Status: Live

What You Will Learn

Databricks in Azure Cloud

Working with DBFS and Mounting Storage

Unity Catalog – Configuring and Working

Unity Catalog User Provisioning and Security

Working with Delta Lake and Delta Tables

Manual and Automatic Schema Evolution

Incremental Ingestion into Lakehouse

Databricks Autoloader

Delta Live Tables and DLT Pipelines

Databricks Repos and Databricks Workflow

Databricks Rest API and CLI

Capstone Project

Who Should Attend

Data Engineers

Data Engineering Solution Architects

Target Audiences

Data Engineers

Data Engineering Solution Architects

About the Course

I am creating Databricks – Master Azure Databricks for Data Engineersusing the Azure cloud platform. This coursewill help you learn the following things.

1. Databricks in Azure Cloud

2. Working with DBFS and Mounting Storage

3. Unity Catalog – Configuring and Working

4. Unity Catalog User Provisioning and Security

5. Working with Delta Lake and Delta Tables

6. Manual and Automatic Schema Evolution

7. Incremental Ingestion into Lakehouse

8. Databricks Autoloader

9. Delta Live Tables and DLT Pipelines

10. Databricks Repos and Databricks Workflow

11. Databricks Rest API and CLI

Capstone Project

This course also includes an End-To-End Capstone project. The project will help you understand the real-life project design, coding, implementation, testing, and CI/CD approach.

Who should take this Course?

I designed this course for data engineers who are willing to develop Lakehouse projects following the Medallion architecture approach using the Databrick cloud platform. I am also creating this course for data and solution architects responsible for designing and building the organization’s Lakehouse platform infrastructure. Another group of people is the managers and architects who do not directly work with Lakehouse implementation. Still, they work with those implementing Lakehouse at the ground level.

Spark Version used in the Course.

This course uses Databricks in Azure Cloud and Apache Spark 3.5. I have tested all the source codes and examples used in this course on Azure Databricks Cloud using Databricks Runtime 13.3.

Course Curriculum

Chapter 1: Before you start

Lecture 1: Course Prerequisites

Lecture 2: About the Course

Lecture 3: How to access Course Material and Resources

Lecture 4: Note for Students – Before Start

Chapter 2: Introduction

Lecture 1: Introduction to Data Engineering

Lecture 2: Apache Spark to Data Engineering Platform

Lecture 3: Introduction to Databricks Platform

Chapter 3: Getting Started

Lecture 1: What will you learn in this section

Lecture 2: Creating Azure Cloud Account

Lecture 3: Azure Portal Overview

Lecture 4: Creating Databricks Workspace Service

Lecture 5: Introduction to Databricks Workspace

Lecture 6: Azure Databricks Platform Architecture

Chapter 4: Working in Databricks Workspace

Lecture 1: What will you learn in this section

Lecture 2: How to create Spark Cluster

Lecture 3: Working with Databricks Notebook

Lecture 4: Notebook Magic Commands

Lecture 5: Databricks Utilities Package

Chapter 5: Working with Databricks File System – DBFS

Lecture 1: What will you learn in this section

Lecture 2: Introduction to DBFS

Lecture 3: Working with DBFS Root

Lecture 4: Mounting ADLS to DBFS

Chapter 6: Working with Unity Catalog

Lecture 1: What will you learn in this section

Lecture 2: Introduction to Unity Catalog

Lecture 3: Setup Unity Catalog

Lecture 4: Unity Catalog User Provisioning

Lecture 5: Working with Securable Objects

Chapter 7: Working with Delta Lake and Delta Tables

Lecture 1: What will you learn in this section

Lecture 2: Introduction to Delta Lake

Lecture 3: Creating Delta Table

Lecture 4: Sharing data for External Delta Table

Lecture 5: Reading Delta Table

Lecture 6: Delta Table Operations

Lecture 7: Delta Table Time Travel

Lecture 8: Convert Parquet to Delta

Lecture 9: Delta Table Schema Validation

Lecture 10: Delta Table Schema Evolution

Lecture 11: Look Inside Delta Table

Lecture 12: Delta Table Utilities and Optimization

Chapter 8: Working with Databricks Incremental Ingestion Tools

Lecture 1: What will you learn in this section

Lecture 2: Architecture and Need for Incremental Ingestion

Lecture 3: Using Copy Into with Manual Schema Evolution

Lecture 4: Using Copy Into with Automatic Schema Evolution

Lecture 5: Streaming Ingestion with Manual Schema Evolution

Lecture 6: Streaming Ingestion with Automatic Schema Evolution

Lecture 7: Introduction to Databricks Autoloader

Lecture 8: Autoloader with Automatic Schema Evolution

Chapter 9: Working with Databricks Delta Live Tables (DLT)

Lecture 1: What will you learn in this section

Lecture 2: Introduction to Databricks DLT

Lecture 3: Understand DLT Use Case Scenario

Lecture 4: Setup DLT Scenario Dataset

Lecture 5: Creating DLT Workload in SQL

Lecture 6: Creating DLT Pipeline for your Workload

Lecture 7: Creating DLT Workload in Python

Chapter 10: Databricks Project and Automation Features

Lecture 1: What will you learn in this section

Lecture 2: Working with Databricks Repos

Lecture 3: Working with Databricks Workflows

Lecture 4: Working with Databricks Rest API

Lecture 5: Working with Databricks CLI

Chapter 11: Capstone Project

Lecture 1: Project Scope and Background

Lecture 2: Taking out the operational requirement

Lecture 3: Storage Design

Lecture 4: Implement Data Security

Lecture 5: Implement Resource Policies

Lecture 6: Decouple Data Ingestion

Lecture 7: Design Bronze Layer

Lecture 8: Design Silver and Gold Layer

Lecture 9: Setup your environment

Lecture 10: Create a workspace

Lecture 11: Create and Storage Layer

Lecture 12: Setup Unity Catalog

Lecture 13: Create Metadata Catalog and External Locations

Lecture 14: Setup your source control

Lecture 15: Start Coding

Lecture 16: Test your code

Lecture 17: Load historical data

Lecture 18: Ingest into bronze layer

Lecture 19: Process the silver layer

Lecture 20: Handling multiple updates

Lecture 21: Implementing Gold Layer

Lecture 22: Creating a run script

Lecture 23: Preparing for Integration testing

Lecture 24: Creating Test Data Producer

Lecture 25: Creating Integration Test for Batch mode

Lecture 26: Creating Integration Test for Stream mode

Lecture 27: Implementing CI CD Pipeline

Lecture 28: Develop Build Pipeline

Lecture 29: Develop Release Pipeline

Lecture 30: Creating Databricks CLI Script

Instructors

Learning Journal
Online Training Company

Prashant Kumar Pandey
Architect, Author, Consultant, Trainer @ Learning Journal

Rating Distribution

1 stars: 7 votes

2 stars: 5 votes

3 stars: 33 votes

4 stars: 267 votes

5 stars: 678 votes

Frequently Asked Questions

How long do I have access to the course materials?

You can view and review the lecture materials indefinitely, like an on-demand channel.

Can I take my courses with me wherever I go?

Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don’t have an internet connection, some instructors also let their students download course lectures. That’s up to the instructor though, so make sure you get on their good side!