Troubleshooting Apache Spark

Troubleshooting Apache Spark, available at $19.99, has an average rating of 3.35, with 21 lectures, based on 10 reviews, and has 94 subscribers.

You will learn about Solve long-running computation problems by leveraging lazy evaluation in Spark Avoid memory leaks by understanding the internal memory management of Apache Spark Rework problems due to not-scaling out pipelines by using partitions Debug and create user-defined functions that enrich the Spark API Choose a proper join strategy depending on the characteristics of your input data Troubleshoot APIs for joins – DataFrames or DataSets Write code that minimizes object creation using the proper API Troubleshoot real-time pipelines written in Spark Streaming This course is ideal for individuals who are If you are an Apache Spark developer at the beginning of your journey and experience a lot of hard problems when using it, this course is for you. You will learn how to solve the most common problems of Apache Spark users It is particularly useful for If you are an Apache Spark developer at the beginning of your journey and experience a lot of hard problems when using it, this course is for you. You will learn how to solve the most common problems of Apache Spark users.

Enroll now: Troubleshooting Apache Spark

Summary

Title: Troubleshooting Apache Spark

Price: $19.99

Average Rating: 3.35

Number of Lectures: 21

Number of Published Lectures: 21

Number of Curriculum Items: 21

Number of Published Curriculum Objects: 21

Original Price: $109.99

Quality Status: approved

Status: Live

What You Will Learn

Solve long-running computation problems by leveraging lazy evaluation in Spark

Avoid memory leaks by understanding the internal memory management of Apache Spark

Rework problems due to not-scaling out pipelines by using partitions

Debug and create user-defined functions that enrich the Spark API

Choose a proper join strategy depending on the characteristics of your input data

Troubleshoot APIs for joins – DataFrames or DataSets

Write code that minimizes object creation using the proper API

Troubleshoot real-time pipelines written in Spark Streaming

Who Should Attend

If you are an Apache Spark developer at the beginning of your journey and experience a lot of hard problems when using it, this course is for you. You will learn how to solve the most common problems of Apache Spark users

Target Audiences

If you are an Apache Spark developer at the beginning of your journey and experience a lot of hard problems when using it, this course is for you. You will learn how to solve the most common problems of Apache Spark users

Apache Spark has been around quite some time, but do you really know how to solve the development issues and problems you face with it? This course will give you new possibilities and you’ll cover many aspects of Apache Spark; some you may know and some you probably never knew existed. If you take a lot of time learning and performing tasks on Spark, you are unable to leverage Apache Spark’s full capabilities and features, and face a roadblock in your development journey. You’ll face issues and will be unable to optimize your development process due to common problems and bugs; you’ll be looking for techniques which can save you from falling into any pitfalls and common errors during development. With this course you’ll learn to implement some practical and proven techniques to improve particular aspects of Apache Spark with proper research

You need to understand the common problems and issues Spark developers face, collate them, and build simple solutions for these problems. One way to understand common issues is to look out for Stack Overflow queries. This course is a high-quality troubleshooting course, highlighting issues faced by developers in different stages of their application development and providing them with simple and practical solutions to these issues. It supplies solutions to some problems and challenges faced by developers; however, this course also focuses on discovering new possibilities with Apache Spark. By the end of this course, you will have solved your Spark problems without any hassle.

About the Author

Tomasz Lelek is a Software Engineer, programming mostly in Java and Scala. He is a fan of microservice architectures and functional programming. He dedicates considerable time and effort to getting better every day. He is passionate about nearly everything associated with software development, and believes that we should always try to consider different solutions and approaches before solving a problem. Recently he was a speaker at conferences in Poland -, Confitura and JDD (Java Developers Day), and also at Krakow Scala User Group. He has also conducted a live coding session at Geecon Conference.

Course Curriculum

Chapter 1: Common Problems and Troubleshooting the Spark Distributed Engine

Lecture 1: The Course Overview

Lecture 2: Eager Computations: Lazy Evaluation

Lecture 3: Caching Values: In-Memory Persistence

Lecture 4: Unexpected API Behavior: Picking the Proper RDD API

Lecture 5: Wide Dependencies: Using Narrow Dependencies

Chapter 2: Distributed DataFrames Optimization Pitfalls

Lecture 1: Making Computations Parallel: Using Partitions

Lecture 2: Defining Robust Custom Functions: Understanding User-Defined Functions

Lecture 3: Logical Plans Hiding the Truth: Examining the Physical Plans

Lecture 4: Slow Interpreted Lambdas: Code Generation Spark Optimization

Chapter 3: Distributed Joins in Cluster

Lecture 1: Avoid Wrong Join Strategies: Using a Join Type Based on Data Volume

Lecture 2: Slow Joins: Choosing an Execution Plan for Join

Lecture 3: Distributed Joins Problem: DataFrame API

Lecture 4: TypeSafe Joins Problem: The Newest DataSet API

Chapter 4: Solving Problems with Non-Efficient Transformations

Lecture 1: Minimizing Object Creation: Reusing Existing Objects

Lecture 2: Iterating Transformations – The mapPartitions() Method

Lecture 3: Slow Spark Application Start: Reducing Setup Overhead

Lecture 4: Performing Unnecessary Recomputation: Reusing RDDs

Chapter 5: Troubleshooting Real-Time Processing Jobs in Spark Streaming

Lecture 1: Repeating the Same Code in Stream Pipeline: Using Sources and Sinks

Lecture 2: Long Latency of Jobs: Understanding Batch Internals

Lecture 3: Fault Tolerance: Using Data Checkpointing

Lecture 4: Maintaining Batch and Streaming: Using Structured Streaming Pros

Instructors

Packt Publishing
Tech Knowledge in Motion

Rating Distribution

1 stars: 1 votes

2 stars: 2 votes

3 stars: 3 votes

4 stars: 1 votes

5 stars: 3 votes

Frequently Asked Questions

How long do I have access to the course materials?

You can view and review the lecture materials indefinitely, like an on-demand channel.

Can I take my courses with me wherever I go?

Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don’t have an internet connection, some instructors also let their students download course lectures. That’s up to the instructor though, so make sure you get on their good side!