@vasild @jonatack Would you like to experiment with Yggdrasil too?

I set up an Yggdrasil router as well: [200:b9b6:8c66:f88d:7043:e30b:d803:c3e2]:8333 (it's not running this branch, but I don't think that matters).

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

• #25678 (p2p: skip querying dns seeds if -onlynet disables IPv4 and IPv6 by mzumsande)
• #25172 (refactor: use std:: prefix for std lib funcs by fanquake)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

Mind that this is a P2P BIP-level change, I think you'll want to write a (short) ancillary BIP to BIP155 to propose this new assigned number.

The addresses used by Yggdrasil are considered routable by Bitcoin Core. That is - just like any other normal IPv6 address, publicly routable on the global internet. They will be gossiped. This is in contrast to CJDNS which uses unroutable addresses from RFC 4193 and thus need special handling in Bitcoin Core.

I have to think more and review carefully this PR, but it may be that it is not necessary and Yggdrasil "just works" with current master. Edit: it works, but can work even better with some minimal changes, see comments below.

It uses an address range of 0200::/8

According to https://yggdrasil-network.github.io/faq.html#will-yggdrasil-conflict-with-my-network-routing it uses 0200::/7 (starts with 02 or 03).

it is hard to get CJDNS working on anything else than x86_64 Linux

Hmm, CJDNS works for me on FreeBSD/x86_64 and on my OpenWRT router - Linux/mips (there is also an yggdrasil package for OpenWRT).

I have to think more and review carefully this PR, but it may be that it is not necessary and Yggdrasil "just works" with current master.

That would great news actually!

If we add support for this, I do think we want to at least include logic to treat Yggdrasil's addresses separately for the purpose of group mapping (e.g. like Tor, treat all of Yggdrasil as 1 or a few network groups, rather than using the normal IPv6 logic). We could do that with or without treating it as a separate network:

• With a separate network, as this PR currently does, it's a somewhat cleaner separation, we get somewhat better failures (non-Yggdrasil-connected nodes would not even bother trying to connect, rather than trying and failing), and if we ever get a network extension to negotiate supported networks, also somewhat better gossip (by preferential relay among compatible nodes). The downside is that it requires a BIP or BIP addendum to reserve a BIP155 network id.
• Without a separate network, it's a smaller change, and we'd implement the group mapping directly on the 0200::/8 range in IPv6, essentially hardcoding the knowledge that that range belongs to Yggdrasil.

I have a slight preference for the first approach.

... treat Yggdrasil's addresses separately for the purpose of group mapping ...

Right!

I have a slight preference for the first approach.

Me too.

Interesting... Concept ACK

Concept ACK, will review

I think you'll want to write a (short) ancillary BIP to BIP155 to propose this new assigned number.

PR to bips repo in bitcoin/bips#1238

Concept ACK

Without this PR, master is able to connect to yggdrasil addresses and will gossip them (good). With minimal changes (this PR), we can get (thanks, @sipa):

• Proper reporting, like in getpeerinfo RPC (to print yggdrasil instead of ipv6)
• Proper network grouping - don't do nonsensical netmasks on those addresses
• Don't try to connect to such addresses if -someconfigoption is not set to indicate yggdrasil connectivity is available
• Selective gossip

@prayank23 Moving the discussion from bitcoin/bips#1238 (comment) here, because I think it's more a question about Bitcoin Core's implemention.

I'm not convinced we should add this either, because of the low maturity of the project. But I think the discussion is a bit more nuanced than just looking at that.

The name of this PR is a bit misleading, for starters. This isn't adding support for yggdrasil to Bitcoin Core in the sense of having the project depend on it, or add direct integration (as was the case for tor and i2p, where significant amount of special code were needed to let Bitcoin Core interact with the specific routing software for those networks). What it does add is a small amount of code to make Bitcoin Core's network stack aware of the fact that a particular subrange of the IPv6 space corresponds to Yggdrasil, and should be treated slightly differently. Interacting with the Yggdrasil network is done independently, by the user installing an Yggdrasil router, which integrates in the OS, and transparently routes yggdrasil-compliant IP addresses over the Yggdrasil network - and this happens (and can be used) whether or not Bitcoin Core has specific support for it.

The advantages are:

• Better behavior in case anyone starts rumouring such addresses on the network, regardless of whether the node's operator considers them using Yggdrasil to be a good idea or not.
• Better behavior for anyone actually using Yggdrasil. I personally think that more distinct ways to connect nodes in general is a good thing, as it contributes to partition resistance of the network.

The disadvantages are:

• Extra code to maintain, obviously.
• Connectivity issues in case there are problems/upgrades with yggdrasil, for those using it.

I think both of the disadvantages are actually tiny. The amount of code being added here is similar to what is needed for cjdns; just making the network stack aware of the existence of this separate range. Connectivity issues could be problem in case someone is exclusively connecting through yggdrasil, but I don't think anyone is arguing for that (or should; just like you shouldn't exclusively use automatic tor or i2p connections). If yggdrasil goes down (either due to a network-wide failure, or incompatible upgrades, or individual routers crashing)... bitcoin core will make new connections; interrupted connections are nothing special. Given that Bitcoin Core is in no way directly talking to the yggdrasil router, I'm not worried about it causing bugs/crashes in Bitcoin Core itself.

@sipa That makes sense. Thanks for explaining everything in detail.

I retract my NACK and have no opinion on this pull request. Will follow the progress, research more about yggdrasil and do tests though.

Hey everyone, I would not recommend adding Yggdrasil support, not on the basis of principle, but on the basis of the nature of the project itself, let me explain.

Yggdrasil is largely an experimental academic protocol/software, aimed to research routing at scale, the intention (of its author @Arceliar) is to - eventually - wrap it up, place a v1 on it, and write a paper, but there is sofar no intention to keep up support after that.

In the mean time, there is also no guarantees of minor version incompatibility, so this could be quite unstable to use.

Instead, i would like to direct everyone's attention to a spin-off project, pinecone, made with the same current routing methods, but intended to be supported for a long time. Pinecone also doesn't work via ipv6 addresses, but full public key hashes, and is intended to be embeddable.

However, as Bitcoin is a large project on the internet with many eyes on it, i would heavily recommend getting into dialogue with Pinecone's developers before proceeding with adding support.

TL;DR: Yggdrasil is experimental and academic, while a project derivative of it, Pinecone, is aiming to be actively developed in the future.

There are some caveats to my recommendation, the first one is the idea that yggdrasil wont be developed anymore after the paper has been written and published, that's an assumption, if anyone else (or bitcoin) could maybe like to pick up development after it's hit v1, then obviously this could be a good addition, but it depends on what it's authors think of the future, so i recommend conversing about that before adding this.

Most of my knowledge here comes from talking with the actual Yggdrasil devs, if anyone would like to ask for themselves, they can do so via the matrix room. (also listed on the site)

wrap it up, place a v1 on it, and write a paper, but there is sofar no intention to keep up support after that.

Can you please comment on that? @neilalexander @Arceliar

@prusnak please also read my disclaimer at the bottom, it's the intention i got from the authors while talking to them (especially @Arceliar), my point being that yggdrasil is more an academic project than a continuous software one. @neilalexander works on yggdrasil on the latter basis, but has split off efforts into Pinecone, primarily inspired by Yggdrasil.

🐙 This pull request conflicts with the target branch and needs rebase.

_{Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a "draft".}

i've had a full node reachable via yggdrasil for about a year ([201:4c36:45a0:4b59:d0d0:9d6a:311d:fca7]:8333) and I didn't know there was a PR for adding Yggdrasil awareness so i'll chuck in my 2 cents on this topic. There are quite a few overlay networks and the number of them seems likely to only increase in the future. Perhaps consolidating all of these kinds into a distinct network type for overlay networks would be a more future-proof approach. e.g. a network type just for overlay networks advertising a mapping with key of "insert-arbitrary-overlay-network-name-here" to ip-ranges-used/gtld-used, so peers can pick and choose which intersection they'd want to utilize. I think this would reduce the number of PRs for "hey add support for my special network layer please" everyone in bitcoin core would need to review and filter, which would likely be a win for everyone involved (especially bitcoin core reviewers).

I agree with the above comment.

Also I am closing this PR, but anyone feel free to take it as a base to for another PR which adds overlay networks in more generic way.

The blocking aspect here was a BIP or BIP update to allocate a BIP155 network number, to make sure there is no conflict with other networks trying to use the same number; apart from that I'd expect it to not be particularly hard to review here.

I'm not sure about a single network id for all overlay networks. That's already not the case for CJDNS, and wasn't the case for IPv4 and TorV2 before (which were specially-mapped subranges of IPv6 pre-BIP155). Having separate network ids for every network also means nodes can have visibility into what they're able to connect to. Having a variety of overlay networks means all overlay-enabled nodes will be relied upon to relay all overlay IP addresses, but depending on the selection of actual networks supported by them, most such connections would fail. Network ids are cheap (there are 256), I don't think we need to worry about allocating them conservatively.

Network ids are cheap (there are 256),

While networks ids are cheap, getting them allocated is quite hard: see bitcoin/bips#1238

While networks ids are cheap, getting them allocated is quite hard: see bitcoin/bips#1238

that is indeed the issue at hand. hence why suggesting a new network id as a catchall for "overlay network" with a peer address containing a mapping of "overlay-name" to "reachable-range/reachable-gtld"

it is hard to get CJDNS working on anything else than x86_64 Linux

Hmm, CJDNS works for me on FreeBSD/x86_64 and on my OpenWRT router - Linux/mips (there is also an yggdrasil package for OpenWRT).

I also found CJDNS to be easy to install on macOS (M1 Ventura in my testing) with brew install cjdns.

I also found CJDNS to be easy to install on macOS (M1 Ventura in my testing) with brew install cjdns.

cjdns has horrible build system - you need Python 3.9 (not 3.10 or newer), Rust and NodeJS just to compile something which is mostly a C99 code 🤮 It's notoriously hard to get it working on new platforms. Huge congrats to the person who got it working on M1. I spent at least 20 hours and failed.