-
Notifications
You must be signed in to change notification settings - Fork 37.7k
[RFC] bitcoin-asmap utility #18573
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[RFC] bitcoin-asmap utility #18573
Conversation
Concept ACK (leaving aside for now the question of inclusion in the codebase). |
Concept ACK. I think it makes sense for us to have this tool inside the repo, if we want to encourage using asmap, and improve the auditability/maintenance. Did some slight code review, will do more if/when we get enough Concept ACKs. Also, I will probably post this separately, but anyone who speaks rust is welcome to review our tool (https://github.com/i7i/asmap-rs) for aggregating routing info from various sources :) |
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
c75b786
to
cead708
Compare
Added lots of comments, and improved the encoder algorithm. I believe it will now always find the smallest encoding (in a few seconds, for realistic inputs). |
Concept ACK |
+0 for having it in the repo, Concept ACK |
Ideally it should be in another repo, but maybe we're not ready for that yet if it shares a lot of code. |
🐙 This pull request conflicts with the target branch and needs rebase. |
It might be worth building this on top of #19937 as part of Concept ACK on including something this into the codebase, ideally along with some instructions which are a bit sparse now. Unless you suspect this needs tons of tweaking in the future, in which case perhaps a separate repo is better. Or if the output can be used by other software than Bitcoin Core. |
@Sjors I had the same problem with the lack of clear instructions so I created a Vagrantfile that performs all the steps in one go. It's located at https://github.com/lestephane/vagrant-asmap You run |
There hasn't been much activity lately and the patch still needs rebase. What is the status here?
|
1 similar comment
There hasn't been much activity lately and the patch still needs rebase. What is the status here?
|
Concept ACK I'm working on some tooling to help with map comparisons between releases and some user experience improvements to Although certainly we'll have an encoded map included with every release when asmap is enabled by default, I think it still makes sense to include |
Closing. I'm working on external tooling for working with asmap files in https://github.com/sipa/asmap/tree/nextgen (encode, decode, diff, bottleneck, ...). |
This adds a new binary,
bitcoin-asmap
that can be used to construct, inspect, and query asmap files. It replaces the Python scripts I had at https://github.com/sipa/asmap with something more performant, efficient, and auditable.There are some rough edges still (in particular documenting the code and a number of edge cases), but I'd like to offer it for comments here. I'm not sure we want this code to be part of the Bitcoin Core codebase, but it certainly has advantages (in terms of code reuse and having consistent encoder/decoder). I think it may also be beneficial that users have a way to audit these files using code that's subject to our review and testing process.
This PR includes a fuzz test that verifies (for small, reduced inputs) that encoding produces asmap files consistent with their input, that invoking the interpreter on them works as expected, and that decoding and re-encoding does not change their behavior.
Given a text file
asmap.txt
with lines like these:You can run
bitcoin-asmap encode asmap.dat <asmap.txt
to construct an asmap file in a few seconds.bitcoin-asmap decode asmap.dat
will decode it back to the original format (note however that the result may look very different, as unassigned IP ranges get remapped for efficiency).bitcoin-asmap lookup asmap.dat 193.190.253.208
will look up one entry in it.There is code at https://github.com/rrybarczyk/asmap-rs (by @rrybarczyk and @naumenkogs) to build input for this tool using RIPE's dumps of Internet routing tables.