Master Scraper

Complete ADE data pipeline - scrape, enrich, link, and analyze everything

Ready

Master Scraper Configuration

Configure what the master scraper should do

Sync Artists

Fetch all artists from ADE API

Sync Events

Fetch all events from ADE API

Parse Lineups

Extract artist lineups from event pages

Link Artists

Create artist-event connections

Scrape Artist Pages

Get Spotify URLs from ADE artist pages

Enrich Artists

Add Spotify data to artists

Check Missing

Find missing connections

Force Replace Existing

Overwrite artists/events/links instead of skipping when present

Batch Size

Rate Limit (ms)

Force Refresh

Refresh existing data

Data Sources

Processing

Analysis