web-scrapping

AgentQL is a suite of tools for connecting your AI to the web. Featuring a query language and Playwright integrations for interacting with elements and extracting data quickly, precisely, and at scale. Includes REST API, Python and JavaScript SDKs, browser debugger.

The NSE has a website which displays the option chain in near real-time. This program retrieves this data from the NSE site and then generates useful analysis of the Option Chain for the specified Index or Stock. It also continuously refreshes the Option Chain and visually displays the trend in various indicators useful for Technical Analysis.

Adaptive Python web scraping toolkit + MCP server for AI agents. Self-healing selectors that survive site changes, TLS-fingerprint stealth to bypass anti-bot filters, CSS/XPath parsing, and 24 built-in scrapers, clean, structured, LLM-ready data from any URL.

Model Context Protocol server that integrates AgentQL's data extraction capabilities.

Proxy List Scrapper

PhantomCrawler is a Python-based web testing and research tool that simulates website interactions from multiple proxy IP addresses to analyze traffic behavior, access controls, and response patterns under different network conditions.

Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

OsintifyX: Powerful Open-source OSINT tool for extracting valuable information from Instagram profiles.

Unofficial python api for Erowid, PsychonautWiki and AskTheCaterpillar

2016: Builds a Tagalog dictionary by collecting Tagalog words from tagalog.pinoydictionary.com

🍀 Scraps data from Leafly and collects amazing weed-based meta-data.

Unoffical api to get comic books from various publishers. Published on NPM !

AgentQL's integrations with workflow automation tools and AI agent frameworks let you extract structured data from web pages using queries or natural language and interact with the web with Playwright. Resilient, fast, and AI-ready.

Evaluate JavaScript on a URL through headless Chrome browser.

This repository contains the code for completely scraping the yellow pages website to get the details of all the categories from all the cities in America and then turning all of the data into an api to be used later.

Get all the resources like Notes and Notebooks provided in CampusX Courses.

Library that retrieves public data from sololearn.com

This Is A Web Scraping Projects With Covid-19 Data From 2 Very Popular & Authentic Websites

This Python script automates interactions with LinkedIn Sales Navigator using Selenium WebDriver. It can log into LinkedIn, navigate through lists of prospects, send messages, and manage contacts based on specific criteria.

Navega pela API do Portal BNMP coletando dados