Skip to content

LanikSJ/ubo-filters

πŸ›‘οΈ uBlock Origin Filters

GitHub Repo Size GitHub Code Size in Bytes GitHub Last Commit GitHub Commit Activity

πŸ“‘ Table of Contents

🎯 Purpose

I got tired of losing my filters when I switched machines so I created this repo.

βš™οΈ Actions

Admiral Domains Dead Domains Check Fanboy Social

πŸ“Š Coverage

Codacy Badge codecov

πŸ“„ Lists

πŸ“„ Main Lists

If you'd like to use any of the filter lists in this repo click the bellow links for AdBlock / AdBlock Plus / AdGuard / uBlock Origin:

πŸ“„ Supplemental Lists

Some of these are old lists or not working lists for archiving purposes:

πŸ”§ Usage Instructions

This repository includes several scripts for managing and processing filter lists. Below are the usage instructions for the main tools.

πŸ“ Filter Processing Scripts

checksum-sort.sh - Enhanced Backup & Sorting Tool

The checksum-sort.sh script provides comprehensive filter list processing with advanced backup functionality.

Basic Usage:

./scripts/checksum-sort.sh filters/combined-filters.txt

Features:

  • Organized Backup Storage: Centralized backup directory (backup/) for better organization
  • Backup Integrity Verification: Automatic checksum verification using SHA256, MD5, or size comparison
  • Automatic Backup Rotation: Configurable maximum backup retention (default: 10 backups)
  • Enhanced Error Handling: Robust rollback mechanism with verification
  • Cross-Platform Compatibility: Works on both Linux and macOS

Advanced Usage:

# List available backups for a filter file
./scripts/checksum-sort.sh --list-backups filters/combined-filters.txt

# View backup history
ls -la backup/

Sample Output:

[INFO] 2024-01-15 10:30:15 Available backups for 'combined-filters.txt':
-rw-r--r-- 1 user staff 1.2M Jan 15 10:25 combined-filters.txt.backup.20240115_102530
-rw-r--r-- 1 user staff 1.1M Jan 15 09:15 combined-filters.txt.backup.20240115_091545

sorter.pl - Enhanced Fanboy Adblock Sorter v2.0

The sorter.pl script has been completely rewritten with modern Perl best practices and enhanced functionality.

Basic Usage (Backward Compatible):

perl scripts/sorter.pl filters/combined-filters.txt

Command Line Options:

  • --help, -h: Show comprehensive help
  • --verbose, -v: Enable detailed logging
  • --backup-suffix: Customize backup file suffix
  • --no-backup: Skip backup creation
  • --dry-run: Preview changes without modification

Advanced Usage Examples:

# Verbose processing with detailed statistics
perl scripts/sorter.pl --verbose filters/combined-filters.txt

# Custom backup suffix
perl scripts/sorter.pl --backup-suffix .original filters/combined-filters.txt

# Process multiple files without backups
perl scripts/sorter.pl --no-backup filters/*.txt

# Preview changes without making modifications
perl scripts/sorter.pl --dry-run --verbose filters/combined-filters.txt

Features:

  • Modern Perl Standards: Strict and warnings pragmas, UTF-8 support
  • Enhanced Error Handling: Comprehensive file validation and graceful error recovery
  • Statistics and Monitoring: File statistics, section counting, before/after comparison
  • Multi-file Support: Process multiple files with individual error handling
  • Safety Features: Atomic file operations, configurable backups, proper cleanup

πŸ› οΈ Other Utility Scripts

Domain-Specific Processing Scripts

The repository includes several specialized scripts for processing different types of filter lists:

  • admiral-domains.sh: Process Admiral anti-adblock domains
  • combined-filters.sh: Generate combined filter lists
  • fanboy-social.sh: Process Fanboy social media filters
  • malware-domains.sh: Process malware domain lists
  • remove-lines.sh: Remove specific lines from filter lists

Additional Tools

  • addChecksum.pl: Add checksums to filter lists
  • FOP.py: Filter Optimizer Python script

πŸ“‹ Configuration

Backup Configuration (checksum-sort.sh):

readonly BACKUP_DIR="${SCRIPT_DIR}/../backup"  # Backup storage location
readonly MAX_BACKUPS=10                        # Maximum backups to retain

Customization Options:

  • Modify BACKUP_DIR to change backup location
  • Adjust MAX_BACKUPS to change retention policy
  • Logging format can be customized in log functions

πŸ”„ Workflow Examples

Basic Filter Processing Workflow:

# 1. Process and sort a filter list with backup
./scripts/checksum-sort.sh filters/combined-filters.txt

# 2. Check processing results
./scripts/checksum-sort.sh --list-backups filters/combined-filters.txt

# 3. If needed, manually sort with verbose output
perl scripts/sorter.pl --verbose filters/combined-filters.txt

Development Workflow:

# 1. Test changes without modification
perl scripts/sorter.pl --dry-run --verbose filters/test-filter.txt

# 2. Apply changes with custom backup
perl scripts/sorter.pl --backup-suffix .dev filters/test-filter.txt

# 3. Process with full backup and verification
./scripts/checksum-sort.sh filters/test-filter.txt

⚠️ Important Notes

  • All scripts maintain backward compatibility with existing workflows
  • Backup files are automatically managed and rotated
  • UTF-8 encoding is supported for international filter lists
  • Cross-platform compatibility ensures consistent behavior on Linux and macOS
  • Process-safe temporary files prevent conflicts during concurrent operations

❓ FAQ

Q: Will these filter lists support any other adblockers such as AdGuard etc?

A: Yes, some of these lists do work with AdGuard, but USE AT YOUR OWN RISK as they haven't been tested thoroughly yet.

πŸ™Œ Attributions

Some scripts have been "borrowed" from Fanboy ©️ All rights reserved by the original authors.

πŸ› Bugs

Please report any bugs or issues you find. Thanks!

πŸ“ License

MIT License

πŸ’° Donate

Patreon