Apache Spark - Master Big Data with PySpark and DataBricks

Apache Spark : Master Big Data with PySpark and DataBricks, available at $44.99, has an average rating of 3.05, with 44 lectures, based on 14 reviews, and has 106 subscribers.

You will learn about Learn the Spark Architecture What is distributed computing Learn Spark Transformations and Actions using the Structured API Learn Spark on Databricks Spark optimization techniques Data Lake House architecture Spark structured streaming using Kafka Information retriever system using word2vec Sentiment analysis using pyspark Training hundreds of time series forecasting models in parallel with Prophet and Spark This course is ideal for individuals who are Data Engineers, Data Architect, ETL developer, Data Scientist, Big Data Developer It is particularly useful for Data Engineers, Data Architect, ETL developer, Data Scientist, Big Data Developer.

Enroll now: Apache Spark : Master Big Data with PySpark and DataBricks

Summary

Title: Apache Spark : Master Big Data with PySpark and DataBricks

Price: $44.99

Average Rating: 3.05

Number of Lectures: 44

Number of Published Lectures: 44

Number of Curriculum Items: 44

Number of Published Curriculum Objects: 44

Original Price: ?1,199

Quality Status: approved

Status: Live

What You Will Learn

Learn the Spark Architecture

What is distributed computing

Learn Spark Transformations and Actions using the Structured API

Learn Spark on Databricks

Spark optimization techniques

Data Lake House architecture

Spark structured streaming using Kafka

Information retriever system using word2vec

Sentiment analysis using pyspark

Training hundreds of time series forecasting models in parallel with Prophet and Spark

Who Should Attend

Data Engineers, Data Architect, ETL developer, Data Scientist, Big Data Developer

Target Audiences

Data Engineers, Data Architect, ETL developer, Data Scientist, Big Data Developer

This course is designed to help you develop the skill necessary to perform ETL operations in Databricks using pyspark, build production ready ML models, learn spark optimization techniques and master distributed computing.

Big Data engineering:

Big data engineers interact with massive data processing systems and databases in large-scale computing environments. Big data engineers provide organizations with analyses that help them assess their performance, identify market demographics, and predict upcoming changes and market trends.

Azure Databricks:

Azure Databricks is a data analytics platform optimized for the Microsoft Azure cloud services platform. Azure Databricks offers three environments for developing data intensive applications: Databricks SQL, Databricks Data Science & Engineering, and Databricks Machine Learning.

Data Lake House:

A data lakehouse is a data solution concept that combines elements of the data warehouse with those of the data lake. Data lakehouses implement data warehouses’ data structures and management features for data lakes, which are typically more cost-effective for data storage .

Spark structured streaming:

Structured Streaming is a scalable and fault-tolerant stream processing engine built on the Spark SQL engine. .In short, Structured Streaming provides fast, scalable, fault-tolerant, end-to-end exactly-once stream processing without the user having to reason about streaming.

Natural language processing:

Natural Language Processing, or NLP for short, is broadly defined as the automatic manipulation of natural language, like speech and text, by software.

The study of natural language processing has been around for more than 50 years and grew out of the field of linguistics with the rise of computers.

Course Curriculum

Chapter 1: Introduction

Lecture 1: Introduction

Lecture 2: Databricks setup

Lecture 3: Upload files to DBFS

Lecture 4: Importing Notebooks into Databricks workspace

Chapter 2: Spark architecture

Lecture 1: Introduction to Apache Spark

Lecture 2: How Filtering works in Apache spark

Lecture 3: How Counting operation works in Apache spark

Lecture 4: How shuffle works in Apache spark

Chapter 3: Spark Transformations – Demo

Lecture 1: Spark Transformations 1 – Hands-on

Lecture 2: Spark Transformations 2 – Hands-on

Lecture 3: Spark Transformations 3- Hands-on

Lecture 4: Aggregations

Lecture 5: Regular expressions

Lecture 6: Window transformations

Chapter 4: Spark Actions – Demo

Lecture 1: Spark actions – Hnads-on

Chapter 5: Spark User Defined Functions

Lecture 1: Pandas overview

Lecture 2: udfs

Chapter 6: Building Blocks of Apache Spark

Lecture 1: Skew

Lecture 2: Spill

Lecture 3: Shuffle

Chapter 7: Spark Optimizations techniques

Lecture 1: Spark ingestion

Lecture 2: Disk partitioning

Lecture 3: Storage

Lecture 4: Predicate Pushdown

Lecture 5: Serialization

Lecture 6: Bucketing

Lecture 7: Zordering

Chapter 8: Adaptive query execution

Lecture 1: AQE1

Lecture 2: AQE2

Chapter 9: Data Lake house Architecture

Lecture 1: What is data lake

Lecture 2: What is Delta Lake

Lecture 3: Elements of Delta Lake

Lecture 4: Delta Lake Demo

Chapter 10: Spark Structured Streaming

Lecture 1: Streaming concepts – Hands-on

Chapter 11: USE CASE : Spark Structured Streaming with Kafka

Lecture 1: Structured streaming with Kafka – Concepts

Lecture 2: Demo – Anonymous wikipedia edits

Chapter 12: USE CASE : Natural Language Processing

Lecture 1: Overview

Lecture 2: Pre-processing

Lecture 3: User Defined functions

Lecture 4: Rule Based Sentiment Analysis

Lecture 5: Information Retravel system using WORD2VEC

Lecture 6: Sentiment Analysis on IMDB dataet

Chapter 13: Training hundreds of time series forecasting models in parallel with spark

Lecture 1: Time series modelling using Facebook Prophet

Lecture 2: Parallelly train the prophet model using spark

Instructors

Data chef
Lead Data Scientist

Rating Distribution

1 stars: 2 votes

2 stars: 1 votes

3 stars: 5 votes

4 stars: 2 votes

5 stars: 4 votes

Frequently Asked Questions

How long do I have access to the course materials?

You can view and review the lecture materials indefinitely, like an on-demand channel.

Can I take my courses with me wherever I go?

Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don’t have an internet connection, some instructors also let their students download course lectures. That’s up to the instructor though, so make sure you get on their good side!