setlist
← Back to Home

How SetList Works

Behind the scenes, SetList uses automated crawling and AI to keep you updated on live events.

The Daily Crawl

Every day, our automated crawler visits venue websites to find new events. We crawl dozens of venues across multiple cities, extracting:

  • Event titles and descriptions
  • Dates and times
  • Venue information
  • Ticket links
  • Artist names and genres

Smart Event Extraction

We use AI-powered crawling with Crawl4AI to understand venue websites, even when they're built with modern JavaScript frameworks. Our system can:

  • Extract events from dynamic, JavaScript-heavy pages
  • Parse dates in various formats
  • Identify artist names and genres
  • Categorize events (music, comedy, theater, etc.)
  • Detect and remove duplicate events

Data Quality

We take data quality seriously. Our system:

  • Validates dates: Only includes events with specific, parseable dates
  • Removes duplicates: Uses content hashing to prevent duplicate listings
  • Updates regularly: Re-crawls venues daily to catch new events and changes
  • Tracks errors: Monitors crawl failures and automatically retries

Technology Stack

Our platform is built with modern, scalable technologies:

Frontend

  • Next.js 15 + React 19
  • TypeScript
  • Tailwind CSS
  • Deployed on Vercel

Backend

  • Python crawler (Crawl4AI)
  • PostgreSQL (Neon)
  • Local worker execution
  • OpenAI GPT-4 for extraction

Privacy & Respect

We respect venue websites and follow best practices:

  • Rate limiting: We crawl responsibly with delays between requests
  • robots.txt compliance: We honor venue crawling preferences
  • Direct linking: We always link to venue pages and ticket sellers
  • No scraping of paid content: We only extract publicly available information

Future Improvements

We're constantly improving. Coming soon:

  • User accounts and favorite venues
  • Email notifications for new events
  • More cities and venues
  • Advanced filtering and search
  • Calendar exports and integrations

Technical FAQs

How does the crawler find events?

Our Python-based crawler visits venue websites daily using Crawl4AI, which can extract events from both static and JavaScript-rendered pages. The system uses AI to understand page structure and extract event details like dates, titles, and ticket links.

How accurate is the event information?

We strive for high accuracy by validating dates, removing duplicates, and linking directly to venue sources. However, since we aggregate from multiple sources, we recommend always checking the venue website for the most current information before attending.

What if an event is listed incorrectly?

While our system includes quality checks, errors can occur. Events are updated daily from venue sources. If you notice an issue, it will typically be corrected within 24 hours as we re-crawl venue pages.

Can I add my own events to the site?

Currently, we only list events that our crawler finds on venue websites. This ensures data quality and prevents spam. If you are a venue owner, contact us at aaron@findsomeshows.com to get your venue added to our crawler.

Do you use any third-party APIs?

We primarily crawl venue websites directly rather than relying on third-party APIs. This gives us more control over data quality and allows us to include venues that may not be on major ticketing platforms. We do use OpenAI GPT-4 to help with event extraction and categorization.

Want to Learn More?

Check out our open-source code or learn more about the project.

About UsView on GitHub
Enjoying SetList?
Buy me a coffee