Web Scraping in Python Selenium, Scrapy + ChatGPT Prize 2024

Web Scraping in Python Selenium, Scrapy + ChatGPT Prize 2024, available at $99.99, has an average rating of 4.42, with 95 lectures, based on 1666 reviews, and has 12426 subscribers.

You will learn about Learn to scrape website from scratch with Beautiful Soup Learn the essential stuff you need before learning web scraping (Python, HTML, XPath) How to use ChatGPT for web scraping 4 web scraping projects (from beginner to advanced projects) Handle pagination with any scraping tool Learn to scrape JavaScript website with Selenium and Scrapy-Splash Store the extracted Data in MongoDb & SQLite3 Scrape infinite scroll websites Login to websites using Selenium & Scrapy This course is ideal for individuals who are Anyone who wants to scrape data from any website with Beautiful Soup, Selenium, and Scrapy or Data Scientists who want to take their skills to the next level or Web developers looking to obtain new information or automate tasks or Anyone who wants to stop spending hours manually copying data off a website, and wants to automate this task It is particularly useful for Anyone who wants to scrape data from any website with Beautiful Soup, Selenium, and Scrapy or Data Scientists who want to take their skills to the next level or Web developers looking to obtain new information or automate tasks or Anyone who wants to stop spending hours manually copying data off a website, and wants to automate this task.

Enroll now: Web Scraping in Python Selenium, Scrapy + ChatGPT Prize 2024

Summary

Title: Web Scraping in Python Selenium, Scrapy + ChatGPT Prize 2024

Price: $99.99

Average Rating: 4.42

Number of Lectures: 95

Number of Published Lectures: 95

Number of Curriculum Items: 95

Number of Published Curriculum Objects: 95

Original Price: $27.99

Quality Status: approved

Status: Live

What You Will Learn

Learn to scrape website from scratch with Beautiful Soup

Learn the essential stuff you need before learning web scraping (Python, HTML, XPath)

How to use ChatGPT for web scraping

4 web scraping projects (from beginner to advanced projects)

Handle pagination with any scraping tool

Learn to scrape JavaScript website with Selenium and Scrapy-Splash

Store the extracted Data in MongoDb & SQLite3

Scrape infinite scroll websites

Login to websites using Selenium & Scrapy

Who Should Attend

Anyone who wants to scrape data from any website with Beautiful Soup, Selenium, and Scrapy

Data Scientists who want to take their skills to the next level

Web developers looking to obtain new information or automate tasks

Anyone who wants to stop spending hours manually copying data off a website, and wants to automate this task

Target Audiences

Anyone who wants to scrape data from any website with Beautiful Soup, Selenium, and Scrapy

Data Scientists who want to take their skills to the next level

Web developers looking to obtain new information or automate tasks

Anyone who wants to stop spending hours manually copying data off a website, and wants to automate this task

Have you ever thought about how important is data in our everyday life?

It doesn’t matter if you’re a data scientist, data analyst, or programmer, getting access to the data you need is important! This is why some of the top Fortune 500 companies such as Walmart, Target, and CNN use web scraping to stay ahead with data. We can say this is one of their best secret weapons.

Welcome to Web Scraping in Python with BeautiuflSoup, Selenium, and Scrapy!

The most complete and project-oriented course. In this course, we’re going to learn web scraping solving cool projects that will help us scrape data off some of the most well-known websites. To name a few:

Amazon Audible

Twitter

Google

Football Stats

Worldometers

Movie pages

This is the most complete web scraping course in Python and is split into 3 parts: BeautiuflSoup, Selenium, and Scrapy. My goal is that you become an expert in web scraping. This is why only 10% of the content of this course is focused on Beautiful Soup, while 40% is dedicated to Selenium and 50% to Scrapy.

Why I built the course this way?

Beautiful Soup has a lot of limitations, but it’s still useful for a gentle introduction to web scraping. In contrast, if you learn Selenium you will be able to scrape any website. and if you add Scrapy to your toolkit you will be able to scrape any website faster than with any other web scraping library. Throughout this course, you will see the power of web scraping with each Python library.

What makes this course different from the others, and why you should enroll?

This is the most updated and complete web scraping course in Python. You will be using Python 3.7, Scrapy 2.4, Selenium 3.141.0 and BS4

This is the most project-based course you will find. We will scrape many of the most well-known websites

You will have an in-depth step by step guide on how to become a professional web scraper.

You will learn how to use Selenium, Scrapy, and Splash to scrape JavaScript-driven websites. You won’t find any tutorials out there that teach how to use these web scraping tools the way I do in this course.

You will learn how to use ChatGPT for web scraping

30 days money back guarantee by Udemy

At the end of the course, you’ll know how to build your own web scraper to obtain new data, learn the best techniques for web scraping and more.

Join me now and let’s start scraping the web together!

Course Curriculum

Chapter 1: Introduction

Lecture 1: Update 2024: Read this before you start

Lecture 2: What is the best web scraping library? BS4 vs Selenium vs Scrapy

Lecture 3: Installing Python and PyCharm

Lecture 4: Python Basics for Web Scraping

Lecture 5: Web Scraping Cheat Sheet for The Course (Dont skip it)

Lecture 6: HTML for Web Scraping: Tags and Elements

Lecture 7: HTML for Web Scraping: Tree Structure

Chapter 2: Introduction to Beautiful Soup

Lecture 1: Important: Versions of libraries that work for this section

Lecture 2: Installing Beautiful Soup

Lecture 3: Beautiful Soup Basics

Lecture 4: How To Get The HTML from a Website

Lecture 5: How to Scrape a Single Page

Lecture 6: Exporting Data to a TXT File

Chapter 3: Project #1: Scraping Multiple Pages with Beautiful Soup

Lecture 1: Update Scripts 2024

Lecture 2: Scraping Multiple Links within The Same Page

Lecture 3: Pagination

Chapter 4: XPath

Lecture 1: Important Note (Dont skip it)

Lecture 2: XPath Syntax, Functions and Operators

Lecture 3: Test Your XPath

Lecture 4: Special Characters and Syntax

Chapter 5: Introduction to Selenium

Lecture 1: Important: Versions of libraries that work for this section

Lecture 2: How to Identify Java Script Driven Websites

Lecture 3: Installing Selenium and ChromeDriver

Lecture 4: Importing Libraries and Creating The Driver

Lecture 5: How to Find Elements with Selenium (Theory)

Lecture 6: Clicking on a Button

Lecture 7: Extracting Data from a Table

Lecture 8: Exporting Data to a CSV File with Pandas

Lecture 9: Selecting Elements within a Dropdown

Chapter 6: Project #1: Dealing with Multiple Pages using Selenium (Amazon Audible Bot)

Lecture 1: Update Scripts 2024

Lecture 2: Building the bot

Lecture 3: Headless Mode

Lecture 4: Handling Pagination

Lecture 5: Implicit vs Explicit Waits

Chapter 7: Introduction to Scrapy

Lecture 1: Important: Versions of libraries that work for this section

Lecture 2: Scrapy Installation

Lecture 3: Setting up Scrapy with Pycharm

Lecture 4: Note: How to Show Hidden Folders on Mac

Lecture 5: Scrapys Commands

Lecture 6: Creating our First Project and Spider

Lecture 7: Scrapy Templates & How to Find Elements with Scrapy

Lecture 8: Scrapys Shell Command

Lecture 9: Building a Spider

Lecture 10: Getting links listed in a website

Lecture 11: Relative vs Absolute Links

Lecture 12: Scraping Data from Multiple Links

Chapter 8: Dealing with Multiple Pages using Scrapy

Lecture 1: Update Scripts 2024

Lecture 2: Building The Spider

Lecture 3: Pagination

Lecture 4: Changing User-Agent

Chapter 9: Project #3: Building Crawlers with Scrapy

Lecture 1: Update Scripts 2024

Lecture 2: Crawler – Rules

Lecture 3: Crawler – Pagination

Lecture 4: Crawler – Changing User-Agent

Chapter 10: Exporting Data to a Database: MongoDB and SQLite

Lecture 1: Update Scripts 2024 (and MongoDB)

Lecture 2: Pipelines

Lecture 3: MongoDB

Lecture 4: SQLite3

Chapter 11: Scraping APIs & Login to website with Scrapy

Lecture 1: Scraping APIs – Building the Spider

Lecture 2: Scraping APIs – Dealing with Pagination

Lecture 3: Login to website – Building the Spider

Chapter 12: Splash Basics

Lecture 1: Section Overview

Lecture 2: Setting up Splash with Docker

Lecture 3: Introduction to Splash

Lecture 4: How to Find Elements with Splash

Chapter 13: Project #4: Scraping JavaScript websites using Scrapy/Splash

Lecture 1: Building the Script in Splash

Lecture 2: Using Splash with Scrapy

Lecture 3: Parsing

Lecture 4: Changing User-Agent

Chapter 14: How to Make Money Using Web Scraping

Lecture 1: Section Overview

Lecture 2: How I Monetized My Web Scraping Skills Over The Past 2 Years

Lecture 3: Getting Started – Part 1

Lecture 4: Getting Started – Part 2

Chapter 15: Prize: ChatGPT for Web Scraping

Lecture 1: Section Overview

Lecture 2: Web Scraping with ChatGPT – Part 1

Lecture 3: Web Scraping with ChatGPT – Part 2

Lecture 4: Web Scraping with ChatGPT Plugin

Lecture 5: Web Scraping with ChatGPT Code Interpreter

Lecture 6: How to create a GPT that scrapes websites

Lecture 7: Congratulations!

Chapter 16: Appendix: Infinite Scrolling & Login (Twitter Bot)

Lecture 1: Important Note

Lecture 2: How to Login to Websites

Lecture 3: Building a Function

Lecture 4: Infinite Scrolling

Instructors

Frank Andrade
Data Scientist

Rating Distribution

1 stars: 25 votes

2 stars: 27 votes

3 stars: 122 votes

4 stars: 517 votes

5 stars: 975 votes

Frequently Asked Questions

How long do I have access to the course materials?

You can view and review the lecture materials indefinitely, like an on-demand channel.

Can I take my courses with me wherever I go?

Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don’t have an internet connection, some instructors also let their students download course lectures. That’s up to the instructor though, so make sure you get on their good side!