Databricks and PySpark for Big Data- From Zero to Expert

Databricks and PySpark for Big Data: From Zero to Expert, available at $74.99, has an average rating of 3.8, with 101 lectures, based on 141 reviews, and has 690 subscribers.

You will learn about Processing Big Data with PySpark in Databricks Databricks environment and Platform ETL, Dataframes and data visualization in Databricks PySpark in Databricks with RDDs, Spark Dataframes API or Spark SQL Spark Column Expresions and Dataframe Agregations Spark Data Sources and Format types Spark Architecture Concepts and Query Optimization Advanced analytics and data visualization with Databricks Machine Learning with Spark at Databricks Spark Streaming at Databricks This course is ideal for individuals who are Anyone who wants to learn Databricks or Anyone who wants to learn advanced big data skills or Anyone wants to make a career as a data engineer, data analyst or data scientist or Anyone interested in learning Apache Spark and PySpark for Big Data analytics or Anyone wants to learn cutting-edge technology in data processing It is particularly useful for Anyone who wants to learn Databricks or Anyone who wants to learn advanced big data skills or Anyone wants to make a career as a data engineer, data analyst or data scientist or Anyone interested in learning Apache Spark and PySpark for Big Data analytics or Anyone wants to learn cutting-edge technology in data processing.

Enroll now: Databricks and PySpark for Big Data: From Zero to Expert

Summary

Title: Databricks and PySpark for Big Data: From Zero to Expert

Price: $74.99

Average Rating: 3.8

Number of Lectures: 101

Number of Published Lectures: 94

Number of Curriculum Items: 101

Number of Published Curriculum Objects: 94

Original Price: $19.99

Quality Status: approved

Status: Live

What You Will Learn

Processing Big Data with PySpark in Databricks

Databricks environment and Platform

ETL, Dataframes and data visualization in Databricks

PySpark in Databricks with RDDs, Spark Dataframes API or Spark SQL

Spark Column Expresions and Dataframe Agregations

Spark Data Sources and Format types

Spark Architecture Concepts and Query Optimization

Advanced analytics and data visualization with Databricks

Machine Learning with Spark at Databricks

Spark Streaming at Databricks

Who Should Attend

Anyone who wants to learn Databricks

Anyone who wants to learn advanced big data skills

Anyone wants to make a career as a data engineer, data analyst or data scientist

Anyone interested in learning Apache Spark and PySpark for Big Data analytics

Anyone wants to learn cutting-edge technology in data processing

Target Audiences

Anyone who wants to learn Databricks

Anyone who wants to learn advanced big data skills

Anyone wants to make a career as a data engineer, data analyst or data scientist

Anyone interested in learning Apache Spark and PySpark for Big Data analytics

Anyone wants to learn cutting-edge technology in data processing

If you are looking for a hands-on, complete and advanced course to learn Databricks and PySpark, you have come to the right place.

Databricks is a data analytics platform powered by Apache Spark for data engineering, data science, and machine learning. Databricks has become one of the most important platforms to work with Spark, compatible with Azure, AWS and Google Cloud. This makes Databricks and Apache Spark some of the most in-demand skills for data engineers and data scientists, and some of the most valuable skills today. This course will teach you everything you need to know to position yourself in the Big Data job market.

This course is designed to prepare you to learn everything related to Databricks and Apache Spark, from the Databricksenvironment, platform and functionalities, to Spark SQL API, Spark Dataframes, Spark Streaming, Machine Learning, advanced analytics and data visualization in Databricks.

With a complete training, downloadable study guides, hands-on exercises, and real-world use cases, this is the only course you’ll ever need to learn Databricks and Apache Spark. You will learn Databricks, starting from the basics to the most advanced functionalities. To do so, we will use visual  presentations, sharing clear explanationsand useful professional advice.

This course covers the following sections:

Introduction to Big Data and Apache Spark

Spark Fundamentals with Spark RDDs, Dataframes

Databricks environment

Advanced analytics and data visualization with Databricks

Machine Learning with Spark at Databricks

Spark Streaming at Databricks

If you’re ready to improve your skills, increase your career opportunities, and become a Big Data expert, join today and get immediate and lifetime access to:

? Complete Guide to Databricks with Apache Spark (PDF e-book)

? Downloadable project files

? Practical exercises and questionnaires

? Databricks resources such as: Cheatsheets and summaries

? 1 to 1 expert support

? Forum of questions and answers of the course

See you there!

Course Curriculum

Chapter 1: Introduction to this course

Lecture 1: Course Material

Lecture 2: How to get the most out of the course

Chapter 2: Introduction to Apache Spark and Big Data

Lecture 1: Spark Fundamentals

Lecture 2: How Apache Spark works

Lecture 3: Apache Spark ecosystem and official documentation

Lecture 4: PySpark: cluster management and architecture

Chapter 3: Installation of Spark on premises (Addiotional)

Lecture 1: Spark installation: downloading tools

Lecture 2: Installing Spark: setting environment variables

Lecture 3: Running Spark at the prompt and jupyter notebook

Chapter 4: Spark DataFrames and Apache Spark SQL

Lecture 1: Fundamentals and advantages of DataFrames

Lecture 2: Characteristics of DataFrames and data sources

Lecture 3: Creating DataFrames in PySpark

Lecture 4: Operations with PySpark DataFrames

Lecture 5: Different types of joins in DataFrames

Lecture 6: Consultas SQL en PySpark

Lecture 7: Funciones avanzadas para cargar y exportar datos en PySpark

Chapter 5: Spark Advanced Features

Lecture 1: Advanced Features and Performance Optimization

Lecture 2: BroadCast Join and caching

Lecture 3: User Defined Functions (UDF) and advanced SQL functions

Lecture 4: Handling and imputation of missing values

Chapter 6: Databricks Fundamentals

Lecture 1: Introduction to Databricks

Lecture 2: Databricks Terminology and Databricks Community

Lecture 3: Crear una cuenta gratuita de Databricks

Chapter 7: Databricks Platform

Lecture 1: Introduction to the Databricks environment

Lecture 2: First steps with Databricks

Chapter 8: Databricks Utilities

Lecture 1: Databricks Utilities

Lecture 2: Databricks Utils for managing File System and libraries

Lecture 3: Databricks Utils for notebooks, secrets and Widgets

Chapter 9: ETL, Dataframes and data visualization in Databricks

Lecture 1: Creating and saving DataFrames in Databricks

Lecture 2: Transformation and visualization of data in Databricks

Chapter 10: Machine learning with Databricks and Apache Spark

Lecture 1: Fundamentals of Machine Learning with Spark

Lecture 2: Spark Machine Learning components

Lecture 3: Stages in the development of a Machine Learning model

Lecture 4: Machine Learning Model Definition and Pipeline Development

Lecture 5: Model evaluation with PySpark and Databricks

Lecture 6: Hyperparameter setting and logging in MLFlow

Lecture 7: Predictions with new data and visualization of results

Chapter 11: Databricks Koalas: The Pandas API for Apache Spark

Lecture 1: Spark Koalas Fundamentals

Lecture 2: Feature Engineering with Koalas

Lecture 3: Creating DataFrames with Koalas

Lecture 4: Data Manipulation and DataFrames with Koalas

Lecture 5: Working with missing data in Koalas

Lecture 6: Data visualization and graph generation with Koalas

Lecture 7: Import and export data with Koalas

Chapter 12: Spark Streaming at Databricks

Lecture 1: Spark Streaming Fundamentals

Lecture 2: Example of Streaming word count with Spark Streaming

Lecture 3: Spark Streaming Configurations: Output Modes and Operation Types

Lecture 4: Spark Streaming Capabilities

Lecture 5: Hands-on Lab part I: Spark Streaming in Databricks

Lecture 6: Hands-on Lab part II: Spark Streaming in Databricks

Chapter 13: Real-time forecasting with Databricks, Spark ML and Spark Streaming

Lecture 1: Case Study: Preprocessing Pipeline and ML Model Development

Chapter 14: Delta Lake

Lecture 1: Delta Lake Fundamentals

Lecture 2: Delta Lake features and benefits

Lecture 3: Architecture of a Delta Lake in Azure

Lecture 4: Generate a Delta Lake and query the data

Lecture 5: Unifying Batch and Streamning processes with Delta Lake and ACID transactions

Lecture 6: Preserving data integrity with Schema Enforcement and Evolution in Delta Lake

Lecture 7: Delta Lake version recovery

Lecture 8: DML Consultations at Delta Lake

Lecture 9: Delta Lake performance optimization

Chapter 15: Spark Architecture Concepts

Lecture 1: Spark Optimization Techniques

Lecture 2: Lazy Evaluation

Lecture 3: Wide and Narrow Transformations

Lecture 4: Parquet file in Spark

Lecture 5: Parallelism and Partitions

Lecture 6: Shuffling

Lecture 7: Caching and Storage Levels

Chapter 16: Machine Learning with Databricks and Apache Spark

Lecture 1: Import and exploratory analysis of data

Lecture 2: Variable preprocessing with PySpark and Databricks

Lecture 3: Definition of the Machine Learning model and development of the Pipeline

Lecture 4: Model evaluation with PySpark and Databricks

Lecture 5: Hyperparameter tuning and registration in MLFlow

Lecture 6: Predictions with new data and visualization of the results

Chapter 17: Spark DataFrame API

Lecture 1: Spark SQL and SQL Dataframe API

Lecture 2: Temporary Views vs Global Temporary Views

Lecture 3: Spark Dataframes

Lecture 4: Spark SQL and SQL Dataframe API Lab

Chapter 18: Spark Column Expresions

Lecture 1: Introduction to Spark Column Expresions

Lecture 2: Column Expressions, operators and methods

Lecture 3: DataFrame Transformation Methods

Lecture 4: Subset Rows in Dataframe

Chapter 19: Dataframe Agregations

Instructors

Data Bootcamp
data scientist

Rating Distribution

1 stars: 6 votes

2 stars: 8 votes

3 stars: 18 votes

4 stars: 47 votes

5 stars: 62 votes

Frequently Asked Questions

How long do I have access to the course materials?

You can view and review the lecture materials indefinitely, like an on-demand channel.

Can I take my courses with me wherever I go?

Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don’t have an internet connection, some instructors also let their students download course lectures. That’s up to the instructor though, so make sure you get on their good side!