Add STUN filtering for Voice Chats (#266) and some cosmetic changes.

UPD: Some assets were manually updated. If you want to compare hashsums, also check https://github.com/Waujito/youtubeUnblock/actions/runs/16983809130

I just decided to drop off that release candidate inifinity counter and move to normal version system.

What's Changed

• Различные фиксы by @metrapoliten in #229
• Implement Aho-Corasick algorithm by @Waujito in #227

New Contributors

• @metrapoliten made their first contribution in #229

Full Changelog: v1.0.0-rc6...v1.0.0

What's Changed

• Add none strategy to udp-faking-strategy by @Waujito in #207
• Update trace logging by @Waujito in #212
• Conntrack support by @Waujito in #211
• Kmod fix #213 by @Waujito in #214
• Quic by @Waujito in #198
• Allow to specify sni domains as file by @Waujito in #223

Full Changelog: v1.0.0-rc5...v1.0.0-rc6

README updates

The readme updates grabbed with diff v1.0.0-rc5...v1.0.0-rc6

OpenWRT section

Since OpenWRT main branch switched to apk instead of opkg, but this is not released yet, here is not deploys for apk in Releases. But apk is supported in PR #196.

Flags

• --instaflush Used with tracing. Flushes the buffer instantly, without waiting for explicit new line. Highly useful for debugging crushes.
• --no-gso Disables support for TCP fat packets which uses GSO. This feature is well tested now, so this flag probably won't fix anything.
• --use-conntrack Enables support for conntrack in youtubeUnblock. Disabled by default. Enabled in kernel module.
• --connbytes-limit=<pkts> Kernel module only! Specify how much packets of connection should be processed by kyoutubeUnblock. Pass 0 if you want for each packet to be processed. This flag may be useful for UDP traffic since unlimited youtubeUnblock may lead to traffic flood and unexpected bans. Defaults to 19. In most cases you don't want to change it.
• --fake-custom-payload-file=<binary file containing TLS message> Same as --fake-custom-payload but binary file instead of hex. The file should contain raw binary TLS message (TCP payload).
• --sni-domains-file=<file contains comma or new-line separated list> Same as --sni-domains but accepts path to container file instead of inline domains list. The format is file may consist of both comma-separated domains list as well as new-line separated list.
• --exclude-domains-file=<file contains comma or new-line separated list> Same as --exclude-domains but accepts path to container file instead of inline domains list. The format is file may consist of both comma-separated domains list as well as new-line separated list.
• --udp-faking-strategy={checksum|ttl|none} Faking strategy for udp. checksum will fake UDP checksum, ttl won't fake but will make UDP content relatively small, none is no faking. Defaults to none.
• --udp-filter-quic={disabled|all|parse} Enables QUIC filtering for UDP handler. If disabled, quic won't be processed, if all, all quic initial packets will be handled. parse will decrypt and parse QUIC initial message and match it with --sni-domains. Defaults to disabled.
• --no-dport-filter By default, youtubeUnblock will filter for TLS and QUIC 443. If you want to disable it, pass this flag. (this does not affect --udp-dport-filter)

UDP/QUIC/Voice Chats

UDP is another communication protocol. Well-known technologies that use it are DNS, QUIC, voice chats. UDP does not provide reliable connection and its header is much simpler than TCP thus fragmentation is limited. The support provided primarily by faking.

For UDP faking in kernel module Make sure to decrease --connbytes-limit up to 5. This will allow not to process additional packets and prevent network flood.

Right now, QUIC faking may not work well, so use --udp-mode=drop option.

QUIC is enabled with --udp-filter-quic flag. The flag supports two modes: all will handle all the QUIC initial messages and parse will decrypt and parse the QUIC initial message, and then compare it with --sni-domains flag.

I recommend to use --udp-mode=drop --udp-filter-quic=parse.

For other UDP protocols I recommend to configure UDP support in the separate section from TCP, like --fbegin --udp-dport-filter=50000-50099 --tls=disabled. See more in flags related to udp and tickets tagged with udp label.

Conntrack troubleshooting

youtubeUnblock optionally depends on conntrack.
For kernel module, if conntrack breaks dependencies, compile it with make kmake EXTRA_CFLAGS="-DNO_CONNTRACK" to disable it completly.

If you want to be able to use connbytes in custom stack where conntrack is broken, check #220 and #213 for possible references.

NAT Hardware/Software offloading

youtubeUnblock will conflict with offloading. But hopefully youtubeUnblock need to process only a bunch of first packets in the connection. So, on some devices it is indeed possible to use youtubeUnblock alongside with offloading, especially on ones driven by nftables (OpenWRT 23+). Note, that this is not tested by me but reported as a workaround by users:

Edit /usr/share/firewall4/templates/ruleset.uc by replacing

meta l4proto { tcp, udp } flow offload @ft;

with

meta l4proto { tcp, udp } ct original packets ge 30 flow offload @ft;

And restart firewall with service firewall restart

What's Changed

• UDP faking support by @Waujito in #200
• New LUCI interface by @Waujito in #205
• Huge remaster of internal parameters logic, unification between kernel module and userspace (#200)

Full Changelog: v1.0.0-rc4...v1.0.0-rc5

README difference

Openwrt

LuCI configuration lives in Services->youtubeUnblock section. It is self descriptive, with description for each flag. Note, that after you push Save & Apply button, the configuration is applied automatically and the service is restarted.

UCI configuration is available in /etc/config/youtubeUnblock file, in section youtubeUnblock.youtubeUnblock. You may pass any args as a string to parameter args, but before it disable interactive flags (You can configurate with it but it is a way harder and I recommend to use it only with luci-app-youtubeUnblock):

uci set youtubeUnblock.youtubeUnblock.conf_strat="args"
uci set youtubeUnblock.youtubeUnblock.args="--queue-num=537 --threads=1"

To save the configs you should do uci commit and then reload_config to restart youtubeUnblock

You can check the logs in CLI mode with logread -l 200 | grep youtubeUnblock command.

In CLI mode you will use youtubeUnblock as a normal init.d service:
for example, you can enable it with /etc/init.d/youtubeUnblock enable.

Firewall

On nftables you should put next nftables rules:

nft add chain inet fw4 youtubeUnblock '{ type filter hook postrouting priority mangle - 1; policy accept; }'
nft add rule inet fw4 youtubeUnblock 'tcp dport 443 ct original packets < 20 counter queue num 537 bypass'
nft add rule inet fw4 youtubeUnblock 'meta l4proto udp ct original packets < 9 counter queue num 537 bypass'
nft insert rule inet fw4 output 'mark and 0x8000 == 0x8000 counter accept'

On iptables you should put next iptables rules:

iptables -t mangle -N YOUTUBEUNBLOCK
iptables -t mangle -A YOUTUBEUNBLOCK -p tcp --dport 443 -m connbytes --connbytes-dir original --connbytes-mode packets --connbytes 0:19 -j NFQUEUE --queue-num 537 --queue-bypass
iptables -t mangle -A YOUTUBEUNBLOCK -p udp -m connbytes --connbytes-dir original --connbytes-mode packets --connbytes 0:8 -j NFQUEUE --queue-num 537 --queue-bypass
iptables -t mangle -A POSTROUTING -j YOUTUBEUNBLOCK
iptables -I OUTPUT -m mark --mark 32768/32768 -j ACCEPT

For IPv6 on iptables you need to duplicate rules above for ip6tables:

ip6tables -t mangle -N YOUTUBEUNBLOCK
ip6tables -t mangle -A YOUTUBEUNBLOCK -p tcp --dport 443 -m connbytes --connbytes-dir original --connbytes-mode packets --connbytes 0:19 -j NFQUEUE --queue-num 537 --queue-bypass
ip6tables -t mangle -A YOUTUBEUNBLOCK -p udp -m connbytes --connbytes-dir original --connbytes-mode packets --connbytes 0:8 -j NFQUEUE --queue-num 537 --queue-bypass
ip6tables -t mangle -A POSTROUTING -j YOUTUBEUNBLOCK
ip6tables -I OUTPUT -m mark --mark 32768/32768 -j ACCEPT

Flags

• --udp-mode={drop|fake} This flag specifies udp handling strategy. If drop udp packets will be dropped (useful for quic when browser can fallback to tcp), if fake udp will be faked. Defaults to fake.

• --udp-fake-seq-len=<amount of faking packets sent> Specifies how much faking packets will be sent over the network. Defaults to 6.

• --udp-fake-len=<size of udp fake> Size of udp fake payload (typically payload is zeroes). Defaults to 64.

• --udp-dport-filter=<5,6,200-500> Filter the UDP destination ports. Defaults to no ports. Specifie the ports you want to be handled by youtubeUnblock.

• --udp-filter-quic={disabled|all} Enables QUIC filtering for UDP handler. If disabled, quic won't be processed, if all, all quic initial packets will be handled. Defaults to disabled.

• --quic-drop Drop all QUIC packets which goes to youtubeUnblock. Won't affect any other UDP packets. Just an alias for --udp-filter-quic=all --udp-mode=drop.

UDP

UDP is another communication protocol. Well-known technologies that use it are DNS, QUIC, voice chats. UDP does not provide reliable connection and its header is much simpler than TCP thus fragmentation is limited. The support provided primarily by faking. For QUIC faking may not work well, so use --quic-drop if you want to drop all quic traffic. For other technologies I recommend to configure UDP support in the separate section from TCP, like --fbegin --udp-dport-filter=50000-50099 --tls=disabled. See more in flags related to udp and issues tagged with udp label.

Kernel module

You can configure the module with its flags:

insmod kyoutubeUnblock.ko 
echo "--fake_sni=1 --exclude_domains=.ru --quic_drop" | sudo tee /sys/module/kyoutubeUnblock/parameters/parameters

You can also do

cat /sys/module/kyoutubeUnblock/parameters/parameters

and check all the parameters configured.

This release brings few but noticeable changes in the code base, few new flags.

What's Changed

• Padavan support by @renr4 in #163
• README.md & args.c: minor correction of typos by @MayanKoyote in #191
• Move workflow binaries builds to docker instead of cache by @Waujito in #193
• Daemonize youtubeUnblock with flags, without any overhead by @Waujito in #174

New Contributors

• @renr4 made their first contribution in #163
• @MayanKoyote made their first contribution in #191

Full Changelog: v1.0.0-rc3...v1.0.0-rc4

README difference

Flags

• --threads=<threads number> Specifies the amount of threads you want to be running for your program. This defaults to 1 and shouldn't be edited for normal use. But if you really want multiple queue instances of youtubeUnblock, note that you should change --queue-num to --queue balance. For example, with 4 threads, use --queue-balance 537:540 on iptables and queue num 537-540 on nftables.
• --daemonize Daemonizes the youtubeUnblock (forks and detaches it from the shell). Terminate the program with killall youtubeUnblock. If you want to track the logs of youtubeUnblock in logread or journalctl, use --syslog flag.
• --syslog Redirects logs to the system log. You can read it with journalctl or logread.
• --noclose Usable only with --daemonize. Will not redirect io streams to /dev/null.

Padavan

YoutubeUnblock may also run on Padavan. Check the manual here[rus]

Commits

• 55d359d: Padavan support (renr) #163
• 235bf82: Add issue templates (Vadim Vetrov)
• 0e9b3ff: Add notice about kmods in readme (Vadim Vetrov)
• f1ab2ac: Userspace youtubeUnblock: support kernel versions less than 3.8 (Vadim Vetrov) #163
• f344e52: Add armv7sf (Vadim Vetrov)
• better support for ALLOC_MALLOC (Vadim Vetrov)
• 93972f8: Update Padavan manual (#163) (Vadim Vetrov)
• handle --no-ipv6 for PF_INET6 UNBIND/BIND_PF (Vadim Vetrov)
• aa96769: Fix youtubeUnblock freeze when more than queue_size packets being sent (Vadim Vetrov)
• 0ee6d66: Fix starting logs (Vadim Vetrov)
• c9537ba: Add versioning (Vadim Vetrov)
• 56b6126: Mention --queue-balance in multithreading flag (Vadim Vetrov)
• 562f17e: Merge branch 'dev' (Vadim Vetrov)
• d87d3c9: Old openwrt (#168) (Vadim Vetrov)
• cb46c89: README.md & args.c: minor correction of typos (MayanKoyote) #191
• bb89925: args.c: minor correction of typos (MayanKoyote) #191
• 438a3c1: Update workflows (Vadim Vetrov)
• 79f47b2: Fix #192
• 3376860: Kyber on firefox (Vadim Vetrov)
• eb544c1: Entware binaries on docker instead of cache (Vadim Vetrov) #193
• 20ce078: Move test builders from cache to docker (Vadim Vetrov) #193
• a4975dc: Daemonize youtubeUnblock with flags, without any overhead (Vadim Vetrov) #174
• ef120ca: Update logging (Vadim Vetrov) #174
• 62551a9: Replace incrementing defines with one enum (Vadim Vetrov) #174
• 3a5dbff: Update kmod logging (Vadim Vetrov) #174
• cb138b9: Improve UDP payload logging (Vadim Vetrov) #174
• fd1ae1e: Move entware builders from entware branch to openwrt (Vadim Vetrov)
• aefabe7: Fix SIGSEGV/stacksmash in UDP logger. (Vadim Vetrov)
• f5a6c57: Do not duplicate default CFLAGS (Vadim Vetrov)

What's Changed

• Custom fakes by @Waujito in #132

Full Changelog: v1.0.0-rc1...v1.0.0-rc3

Commits

• 82c4911: Add more custom fake message, random fake message (Vadim Vetrov) #132
• 3ee979f: Enhance middle sni split (Vadim Vetrov) #132
• 98a3fd5: Add workflow for test build (Vadim Vetrov) #132
• 6b21e9b: Add checker for kernel module (Vadim Vetrov) #132
• 263a04b: Kernel module code cleanup (Vadim Vetrov) #132
• 666b357: Fix kmod fake, update kmod verbosity settings (Vadim Vetrov) #132
• 9dc40bb: Fix split on preset domain list (Vadim Vetrov) #132
• 30bc3a8: Update workflow for separate luci-app-youtubeUnblock (Vadim Vetrov) #132
• e9b033c: Update faking strategies (Vadim Vetrov) #132
• 05cc005: Fix getrandom on older versions (Vadim Vetrov) #132
• 58f4802: Update kernel module parameters (Vadim Vetrov) #132
• 4c7b63f: Add multiple sections with config sets for various domains (Vadim Vetrov) #132
• 96cf036: Fix possible errors on older compilers (Vadim Vetrov) #132
• ed08fea: Mention #148 in README Troubleshooting (Vadim Vetrov) #132
• 37a517e: Fix #155 synfake option deleted but present in openwrt (Vadim Vetrov)

README difference

OpenWRT section

For LuCI aka GUI aka web-interface of router you should install luci-app-youtubeUnblock package like you did it with the normal youtubeUnblock package. Note, that lists of official opkg feeds should be loaded (Do it with Update lists option).
LuCI configuration lives in Services->youtubeUnblock section. It is self descriptive, with description for each flag. Note, that after you push Save & Apply button, the configuration is applied automatically and the service is restarted.

Flags section

--fake-sni-type={default|custom|random} This flag specifies which faking message type should be used for fake packets. For random, the message of random length and with random payload will be sent. For default the default payload (sni=www.google.com) is used. And for the custom option, the payload from --fake-custom-payload section utilized. Defaults to default.
--fake-custom-payload=<payload> Useful with --fake-sni-type=custom. You should specify the payload for fake message manually. Use hex format: --fake-custom-payload=0001020304 mean that 5 bytes sequence: 0x00, 0x01, 0x02, 0x03, 0x04 used as fake.
--fbegin and --fend flags: youtubeUnblock supports multiple sets of strategies for specific filters. You may want to initiate a new set after the default one, like: --sni-domains=googlevideo.com --faking-strategy=md5sum --fbegin --sni-domains=youtube.com --faking-strategy=tcp_check --fend --fbegin --sni-domains=l.google.com --faking-strategy=pastseq --fend. Note, that the priority of these sets goes backwards: last is first, default (one that does not start with --fbegin) is last. If you start the new section, the default settings are implemented just like youtubeUnblock without any parameters. Note that the config above is just an example and won't work for you.

Troubleshooting section

Check up this issue for useful configs.

Kernel module section

Note that the flags names are different from ones used for the regular youtubeUnblock(right like in UCI configuration for OpenWRT): replace - with _ and no leading --. Also to configure togglers you should set them to 1 (quic_drop=1)
Also a good thig to mention is verbosity. The kernel module combines --trace and --silent option to the one parameter verbosity. This parameter accepts 3 arguments: trace, debug and silent. I highly don't recommend to enable trace mod on router because it may cause huge problems with performance and even freeze your device.

What's Changed

• Custom fakes by @Waujito in #132

Full Changelog: v1.0.0-rc1...v1.0.0-rc2

Commits

• 82c4911: Add more custom fake message, random fake message (Vadim Vetrov) #132
• 3ee979f: Enhance middle sni split (Vadim Vetrov) #132
• 98a3fd5: Add workflow for test build (Vadim Vetrov) #132
• 6b21e9b: Add checker for kernel module (Vadim Vetrov) #132
• 263a04b: Kernel module code cleanup (Vadim Vetrov) #132
• 666b357: Fix kmod fake, update kmod verbosity settings (Vadim Vetrov) #132
• 9dc40bb: Fix split on preset domain list (Vadim Vetrov) #132
• 30bc3a8: Update workflow for separate luci-app-youtubeUnblock (Vadim Vetrov) #132
• e9b033c: Update faking strategies (Vadim Vetrov) #132
• 05cc005: Fix getrandom on older versions (Vadim Vetrov) #132
• 58f4802: Update kernel module parameters (Vadim Vetrov) #132
• 4c7b63f: Add multiple sections with config sets for various domains (Vadim Vetrov) #132
• 96cf036: Fix possible errors on older compilers (Vadim Vetrov) #132
• ed08fea: Mention #148 in README Troubleshooting (Vadim Vetrov) #132

README difference

OpenWRT section

For LuCI aka GUI aka web-interface of router you should install luci-app-youtubeUnblock package like you did it with the normal youtubeUnblock package. Note, that lists of official opkg feeds should be loaded (Do it with Update lists option).
LuCI configuration lives in Services->youtubeUnblock section. It is self descriptive, with description for each flag. Note, that after you push Save & Apply button, the configuration is applied automatically and the service is restarted.

Flags section

--fake-sni-type={default|custom|random} This flag specifies which faking message type should be used for fake packets. For random, the message of random length and with random payload will be sent. For default the default payload (sni=www.google.com) is used. And for the custom option, the payload from --fake-custom-payload section utilized. Defaults to default.
--fake-custom-payload=<payload> Useful with --fake-sni-type=custom. You should specify the payload for fake message manually. Use hex format: --fake-custom-payload=0001020304 mean that 5 bytes sequence: 0x00, 0x01, 0x02, 0x03, 0x04 used as fake.
--fbegin and --fend flags: youtubeUnblock supports multiple sets of strategies for specific filters. You may want to initiate a new set after the default one, like: --sni-domains=googlevideo.com --faking-strategy=md5sum --fbegin --sni-domains=youtube.com --faking-strategy=tcp_check --fend --fbegin --sni-domains=l.google.com --faking-strategy=pastseq --fend. Note, that the priority of these sets goes backwards: last is first, default (one that does not start with --fbegin) is last. If you start the new section, the default settings are implemented just like youtubeUnblock without any parameters. Note that the config above is just an example and won't work for you.

Troubleshooting section

Check up this issue for useful configs.

Kernel module section

Note that the flags names are different from ones used for the regular youtubeUnblock(right like in UCI configuration for OpenWRT): replace - with _ and no leading --. Also to configure togglers you should set them to 1 (quic_drop=1)
Also a good thig to mention is verbosity. The kernel module combines --trace and --silent option to the one parameter verbosity. This parameter accepts 3 arguments: trace, debug and silent. I highly don't recommend to enable trace mod on router because it may cause huge problems with performance and even freeze your device.

This is a release candidate of the first major version of youtubeUnblock.

This release brings:

• CI: Add MIPS by @spvkgn in #108
• Openwrt LuCI support by @Waujito in #110 and #113
• Kernel module by @Waujito in #98
• Enhance scripts for entware
• Add support for Quic over IPv6
• Use firewall rules for udp traffic by default

Full Changelog: v0.3.2...v1.0.0-rc1

This version brings some default options improvements, entware packaging system improvements, support for IP version 6.

What's Changed

• Add options of choosing to use system libs by @denisstrizhkin in #82
• Support for ipv6 by @Waujito in #93

New Contributors

• @denisstrizhkin made their first contribution in #82

Full Changelog: v0.3.1...v0.3.2

Commits

• 24826f8: removed duplicated code in args.c (zabbius)
• 219062a: Fix segfault bufs, update coding style (Vadim Vetrov)
• aafa1a7: Merge #57 by @zabbius to dev (Vadim Vetrov)
• d5db8c1: Types to distinct file common for the entire program (Vadim Vetrov) #60
• 4a8f0d1: Skeleton for quic initial message parser (Vadim Vetrov) #60
• f3db464: Add initial support for QUIC, improve logging capabilities. (Vadim Vetrov) #60
• 0aef6a9: Merge branch 'main' into quic (Vadim Vetrov) #60
• bcdf181: Merge branch 'main' into dev (Vadim Vetrov)
• 727e909: Add documentation for QUIC (Vadim Vetrov) #60
• 7f340fb: Merge branch 'quic' into dev (Vadim Vetrov) #60
• 044801e: Add support for bruteforce mode of parsing SNI from Client Hello. (Vadim Vetrov)
• af6e993: Merge branch 'main' into dev (Vadim Vetrov)
• 51c21a8: Fix endian source (Vadim Vetrov)
• 1c5d4e6: Add few logs, minor improvements (Vadim Vetrov)
• a546e78: Add support for tcp_check and past sequence faking strategies (Vadim Vetrov)
• 6cf2ec5: Update README.md (Vadim Vetrov)
• b434ef4: Add compatibility with v0.2.2 (Vadim Vetrov)
• 551fb5d: Update README.md (Vadim Vetrov)
• e8d86b9: Do not delete all libraries on every clean (Vadim Vetrov)
• 78ed6a1: Merge branch 'dev' (Vadim Vetrov)

What's Changed

• Update Readme.md by @Viktor45 in #62
• CI workflow by @spvkgn in #66
• Quic support by @Waujito in #60
• minor - removed duplicated code in args.c by @zabbius in #57

New Contributors

• @Viktor45 made their first contribution in #62

Full Changelog: v0.3.0...v0.3.1