Web Scraping 101 with Python3 using REQUESTS, LXML SPLASH

Web Scraping 101 with Python3 using REQUESTS, LXML & SPLASH, available at $69.99, has an average rating of 4.65, with 65 lectures, 1 quizzes, based on 392 reviews, and has 3354 subscribers.

You will learn about LXML core fundamentals XPath & CSS selectors How send HTTP requests with Python Scraping HTML web pages Scraping multiple pages using recursion Scraping APIs Splash HTTP API Scraping JavaScript websites using Splash Authentication and Login to websites using Requests Web scraping best practices Building datasets This course is ideal for individuals who are Anyone wants to learn Web scraping using Python, Requests and LXML or Anyone wants to learn how to use Splash to scrape JavaScript websites or Complete beginners with no background on web scraping or Those who already have basic familiarity with web scraping and want to fill the gaps It is particularly useful for Anyone wants to learn Web scraping using Python, Requests and LXML or Anyone wants to learn how to use Splash to scrape JavaScript websites or Complete beginners with no background on web scraping or Those who already have basic familiarity with web scraping and want to fill the gaps.

Enroll now: Web Scraping 101 with Python3 using REQUESTS, LXML & SPLASH

Summary

Title: Web Scraping 101 with Python3 using REQUESTS, LXML & SPLASH

Price: $69.99

Average Rating: 4.65

Number of Lectures: 65

Number of Quizzes: 1

Number of Published Lectures: 65

Number of Published Quizzes: 1

Number of Curriculum Items: 70

Number of Published Curriculum Objects: 70

Original Price: $19.99

Quality Status: approved

Status: Live

What You Will Learn

LXML core fundamentals

XPath & CSS selectors

How send HTTP requests with Python

Scraping HTML web pages

Scraping multiple pages using recursion

Scraping APIs

Splash HTTP API

Scraping JavaScript websites using Splash

Authentication and Login to websites using Requests

Web scraping best practices

Building datasets

Who Should Attend

Anyone wants to learn Web scraping using Python, Requests and LXML

Anyone wants to learn how to use Splash to scrape JavaScript websites

Complete beginners with no background on web scraping

Those who already have basic familiarity with web scraping and want to fill the gaps

Target Audiences

Anyone wants to learn Web scraping using Python, Requests and LXML

Anyone wants to learn how to use Splash to scrape JavaScript websites

Complete beginners with no background on web scraping

Those who already have basic familiarity with web scraping and want to fill the gaps

What is web scraping ?

Let’s say your boss at work gave you a task where he wants you to extract about 1000 product from a website, structure the data and save it to a database, would you copy paste manually all the product details from the product name, url and price ? I can imagine you would work days and nights and you wont finish the task, so this is where web scraping shines. So web scraping, or web harvesting or web data extraction is like writing a script that will automate data extraction from websites in a matter of minutes !.

Why learn Web Scraping ?

Whether you’re a data analyst, a web developer or even someone who wants to work as a freelancer you should learn web scraping. 

For a data analyst building a datasetis extremely important, so without web scraping you simply can’t generate it in addition to that adding web scraping in your resume is a plus for you.

Web scraping can be used in a variety of fields, so let me give you some examples on what you can do with it:

1. Generate leads,

2. Drop shipping where basically you gonna constantly scrape products from different online stores and show case them on your website to make money,

3. Monitor products prices to get the best deals,

4. automation,

5. Machine learning,

6. Web scraping freelancer

Of course there are tons and tons and variety of fields where web scraping can be extremely beneficial.

Is this course the right one for you ?

I’ve carefully planned and designed this course to be beginner friendly, from my experience I know those who do web scraping are mostly data analyst with no background knowledge on how the web works, how requests are made, how to locate and parse the data from the web and much much more, in addition to that this the most updated course regarding the material included and the tools used, so in this course

1. I’ll introduce to you the most used web scraping tools/frameworks

2. We will setup the development environment from scratch

3. You will learn and understand LXML core fundamentals

4. How to use XPath & CSS selectors to select the data from a web page

5. How the web works (Request/Response)

6. How to scrape simple HTML web pages

7. How to scrape multiple web pages

8. Extract data from APIs

9. You will learn Splash(crash course) so you can use it to scrape JavaScript websites

10. Authentication/Login

11. Store the extracted data whether to JSON/CSV files or MongoDb/SQLite3

12. Exclusive tips and tricks regarding web scraping

Finally this course is project based, each section starting from the 2nd one we will experiment with a different website, each project has a certain degree of difficulty  and each one is completely independent from other projects. 

Is there is any assignments/exercises included in this course ? 

Yes, each section has an assignment included to it, this will help to get your hands dirty and by the end of each section after doing the assignment included you will feel more confident and comfortable with web scraping.

Why LXML and not BeautifulSoup ?

LXML is a lightweight HTML parser even the most popular web scraping framework (Scrapy) is built on the top of LXML, BeautifulSoup is a little bit overloaded with the number of functions exposed to us, it has more functions to use, yes that’s right ! however in Web Scraping most of the time we use XPath and CSS Selectors to navigate and select what to scrape from the HTML web page (tree) so there is no need to learn about new functions and wasting all that time to familiarize yourself with the BeautifulSoup API and the internal architecture, in addition to all of that LXML in terms of performance is way better than BeautifulSoup.

Who is your instructor ? 

Hi! I’m Ahmed nice to meet you, my students prefer to call me web scraping Ninja and currently I have taught more than 2000 students around the world how to do web scraping. I personally do web scraping on daily basis whether for fun, for personal projects or as a freelancer and guess what ? I even have a master degree in computer science.

Should I enroll to this course ?

Honestly, by enrolling to this course you have nothing to lose, because if this course didn’t meet your requirements, you can always ask for a refund in less than 30 days from the day you enrolled to the course guaranteed by Udemy with NO QUESTION TO ASK!

SO IF YOU DON’T KNOW ANYTHING ABOUT WEB SCRAPING & YOU DON’T KNOW WHERE TO START  ENROLL NOW ! 馃檪

Course Curriculum

Chapter 1: Getting Started

Lecture 1: Course Introduction

Lecture 2: Web Scraping tools

Lecture 3: Setting up the development environnement

Lecture 4: Udemy 101 (OPTIONAL)

Lecture 5: How to Ask questions (Please dont skip)

Chapter 2: LXML core fundamentals

Lecture 1: Section Info

Lecture 2: ElementTree object

Lecture 3: Element object

Lecture 4: Introduction to LXML with XPath

Lecture 5: Introduction to LXML with CSS Selectors

Lecture 6: Code source lecture by lecture

Chapter 3: XPath & CSS Selectors

Lecture 1: Section Info

Lecture 2: What is XPath & CSS

Lecture 3: CSS Selectors fundamentals

Lecture 4: CSS selectors in theory

Lecture 5: XPath fundamentals

Lecture 6: Navigating using XPath(Going UP)

Lecture 7: Navigating using XPath(Going DOWN)

Lecture 8: XPath in theory

Chapter 4: HTTP Requests with Python

Lecture 1: Section Info

Lecture 2: How the web works

Lecture 3: Python Requests

Lecture 4: Request/Response headers

Chapter 5: Project 1: Simple & Clean

Lecture 1: Section Info

Lecture 2: Locating the data

Lecture 3: Building the Scraper

Lecture 4: Cleaning the data

Lecture 5: Writing data to JSON/CSV files

Lecture 6: Turning it into a command line app

Lecture 7: Project 1 source code

Chapter 6: Project 2: Recursion

Lecture 1: Section Info

Lecture 2: Getting rid of unnecessary JavaScript

Lecture 3: Scraping Data

Lecture 4: Scraping multiple pages (Recursion)

Lecture 5: Storing the data in MongoDb cloud

Lecture 6: 6-5 Prevent storing same records and updating records

Lecture 7: CoinMarketCap update

Lecture 8: Project 2 source code

Chapter 7: Project 3: APIs

Lecture 1: Section Info

Lecture 2: API/HTML Whats the difference ?

Lecture 3: Generating code using Postman

Lecture 4: Parsing APIs

Lecture 5: Recursion challenge

Lecture 6: Challenge solution (Scraping APIs recursively)

Lecture 7: Inserting data into SQLite3 database

Lecture 8: Project 3 source code

Chapter 8: Splash crash course

Lecture 1: Section Info

Lecture 2: What is Splash ?

Lecture 3: Setting up Splash

Lecture 4: Intro to Splash

Lecture 5: Selecting Elements, filling Inputs and clicking on Buttons

Lecture 6: Splash Request & Response headers

Lecture 7: Very useful resource (SPLASH FAQ)

Chapter 9: Project 4: Scraping JavaScript websites using Splash, Requests and LXML

Lecture 1: Section Info

Lecture 2: Splash private mode and cookies

Lecture 3: Quick note (Splash private mode)

Lecture 4: Using Splash with Requests

Lecture 5: Parsing the Response

Lecture 6: Project 4 source code

Chapter 10: Project 5: Authentication/Login

Lecture 1: Section Info

Lecture 2: Browser authentication

Lecture 3: Requests authentication

Lecture 4: Parse and clean HTML

Lecture 5: Project 5 source code

Chapter 11: BONUS

Lecture 1: Bonus lecture

Instructors

Ahmed Rafik
Developer and Online Instructor

Rating Distribution

1 stars: 3 votes

2 stars: 5 votes

3 stars: 24 votes

4 stars: 112 votes

5 stars: 248 votes

Frequently Asked Questions

How long do I have access to the course materials?

You can view and review the lecture materials indefinitely, like an on-demand channel.

Can I take my courses with me wherever I go?

Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don’t have an internet connection, some instructors also let their students download course lectures. That’s up to the instructor though, so make sure you get on their good side!