A Python-based web scraper that fetches live and past match data from the Women's Premier League (WPL) website using Selenium, BeautifulSoup, and GitHub Actions. The script runs automatically every Hour and updates a JSON file with match details.
Important
This project is for educational purposes only. It does not store, redistribute, or claim ownership over any third-party data. Users are responsible for complying with website terms of service.
- Automated Web Scraping – Uses Selenium + BeautifulSoup to extract match details dynamically.
- GitHub Actions Integration – Runs automatically on schedule without manual execution.
- Web Development Ready – Data is stored in wpl_data.json, which can be used in web applications.
- Bypass Restrictions – Implements headless browsing, user-agent rotation, and dynamic content handling.
-
Clone the repository
git clone https://github.com/cu-sanjay/cricket-score-scraper cd cricket-score-scraper
-
Install dependencies
pip install -r requirements.txt
-
Run the script
python test.py
- The script is scheduled to run every 1 hour using GitHub Actions.
- It fetches live match data and commits changes automatically.
- No manual intervention is needed once set up.
- Handling Strict Websites
- Rotate user-agents to prevent detection.
- Use headless browsing for minimal footprint.
- Simulate human interactions (scrolling, waiting, retries).
- Extract data from network requests instead of the rendered page.
Tip
Web Development Integration
- Serve wpl_data.json via Flask/Django API.
- Fetch and display match data in React/Next.js frontend.
- Automate updates via Telegram/Reddit/Discord bot.