Technology & Engineering8 skills

Web Scraping skills for AI agents

8 practitioner-grade web scraping skills, each a focused Markdown document your agent loads into context on demand. Search them from Claude Desktop, Cursor or any MCP client, or pull one with the CLI.

$ npx skilldb search "web scraping"

All 8 skills

Anti Detection
Ethical techniques for handling CAPTCHAs, rate limiting, and bot detection while scraping responsibly
225 lines
Beautifulsoup
HTML and XML parsing with Beautiful Soup in Python for flexible data extraction
175 lines
Cheerio
Fast server-side HTML parsing and data extraction with Cheerio using jQuery-like syntax
157 lines
Data Pipeline
Patterns for building robust scraping data pipelines with validation, deduplication, storage, and monitoring
391 lines
Playwright Scraping
Cross-browser web scraping with Playwright, supporting Chromium, Firefox, and WebKit
172 lines
Puppeteer
Headless Chrome browser automation with Puppeteer for scraping dynamic, JavaScript-rendered pages
164 lines
Scrapy
Production-grade web scraping framework in Python with built-in crawling, pipelines, and middleware
217 lines
Selenium
Browser-based web scraping and automation with Selenium WebDriver across multiple languages
204 lines