Skip to content
#

Web Scraping

Star

Web scraping is the process of programmatically retrieving web pages and extracting structured data from them for analysis, storage, or reuse. It powers price monitoring, search indexing, market research, and training-data collection.

Scraping ranges from plain HTTP requests and HTML parsing to full browser automation for JavaScript-rendered and anti-bot-protected pages. The ecosystem includes request libraries, HTML parsers, headless browsers, and cloud extraction platforms.

Here are 1,715 public repositories matching this topic...

Language: JavaScript
Filter by language
All 21,467 Python 10,930 Jupyter Notebook 2,052 JavaScript 1,715 TypeScript 1,342 HTML 770 Go 383 Rust 209 Java 171 R 150 PHP 125
Sort: Most stars
Sort options
Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

cporter202 / API-mega-list

Star 7.7k

This GitHub repo is a powerhouse collection of APIs you can start using immediately to build everything from simple automations to full-scale applications. One of the most valuable API lists on GitHub—period. 💪

real-estate api open-source ecommerce automation social-media mcp artificial-intelligence web-scraping developer-tools awesome-list data-collection api-directory lead-generation apify public-apis api-list developer-resources ai-tools api-collection
  • Updated Jul 23, 2026
  • JavaScript
fredy

orangecoding / fredy

Sponsor
Star 1.6k

❤️ Fredy - [F]ind [R]eal [E]state [D]amn Eas[y] - Fredy keeps searching for new apartments, houses, and flats in Europe on platforms like ImmoScout24, Immowelt, eBay Kleinanzeigen and instantly delivers the results to you via Slack, Telegram, Email, Discord or ntfy, so you can focus on the more important things in life ;)

real-estate nodejs crawler self-hosted web-scraping wg-gesucht appartments immobilienscout24 puppeteer immowelt fredy ebay-kleinanzeigen immobilien telegrambots wohnungssuche appartment immoscout24 real-estate-search
  • Updated Sep 29, 2026
  • JavaScript

rebrowser / rebrowser-patches

Star 1.4k

Collection of patches for puppeteer and playwright to avoid automation detection and leaks. Helps to avoid Cloudflare and DataDome CAPTCHA pages. Easy to patch/unpatch, can be enabled/disabled on demand.

bot crawler chrome automation webdriver headless scraping crawling selenium cloudflare web-scraping chromedriver stealth bot-detection headless-chrome puppeteer datadome puppeteer-extra playwright rebrowser
  • Updated May 9, 2025
  • JavaScript
Followers
353 followers
Website
github.com/topics/web-scraping

Related topics

browser-automation headless-browser playwright puppeteer selenium