contrib: Check symbols in the bitcoinconsensus shared library
#25020
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Yes, very good idea, Concept ACK. |
laanwj
reviewed
May 3, 2022
| print(f'{filename}: symbol {symbol.name} from unsupported version {version}') | ||
| ok = False | ||
| return ok | ||
|
|
||
| def check_exported_symbols(binary) -> bool: | ||
| ok: bool = True | ||
| if binary.abstract.header.object_type == lief.OBJECT_TYPES.LIBRARY: | ||
| return ok |
There was a problem hiding this comment.
I think one of the most important things to check for shared libraries is that they export the right symbols (to make sure their entire API is available), and only the right symbols (extra and re-exported symbols can cause conflicts).
There was a problem hiding this comment.
Yea, that is what I assumed this would be doing when I first looked. Otherwise I think the checks currently being added are ~0.
There was a problem hiding this comment.
Exactly. It was the initial idea of this PR. When I faced with a crowd of exported symbols, I decided to make this PR focused on introducing a tool, and leave fighting with linker issues for follow ups.
There was a problem hiding this comment.
I think the PR should be renamed then? If it's not actually doing the thing in the PR title.
fanquake
reviewed
May 3, 2022
| @@ -176,76 +176,76 @@ def check_version(max_versions, version, arch) -> bool: | |||
| def check_imported_symbols(binary) -> bool: | |||
| ok: bool = True | |||
|
|
|||
| for symbol in binary.imported_symbols: | |||
| for symbol in binary.concrete.imported_symbols: | |||
There was a problem hiding this comment.
2012abe: contrib: Always explicitly cast lief.Binary object
Why?
There was a problem hiding this comment.
I also wonder, it's a bit ugly to have .concrete everywhere.
laanwj
reviewed
May 31, 2022
| print(f'{split[-1]} is not in ALLOWED_LIBRARIES!') | ||
| for dylib in binary.concrete.libraries: | ||
| library = dylib.name.split('/')[-1] | ||
| if binary.abstract.header.object_type == lief.OBJECT_TYPES.LIBRARY and library == 'libbitcoinconsensus.0.dylib': |
There was a problem hiding this comment.
Let's not hardcode this name here. If we need to skip this, please define another constant for the list of libraries to skip.
There was a problem hiding this comment.
In general, I don't really like how these changes are mixing up the binary and library checks.
|
Speaking of checking exported symbols, I checked Most are related to C++ standard library (like vector), it seems? I wonder if there's a way to avoid this. |
It appears, #24994 fixes it: |
That's beautiful! |
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
|
GUIX hashes x86: arm64: |
fanquake
reviewed
Dec 5, 2022
There was a problem hiding this comment.
Is this change (going to be) dependant on #24994, or not? If so, maybe mark as a draft for now, and add that to the PR description?
Looks like there's still review comments outstanding, i.e here: https://github.com/bitcoin/bitcoin/pull/25020/files#r885641952 and here: https://github.com/bitcoin/bitcoin/pull/25020/files#r885637804.
| @echo "Running symbol and dynamic library checks..." | ||
| $(AM_V_at) $(PYTHON) $(top_srcdir)/contrib/devtools/symbol-check.py $(bin_PROGRAMS) | ||
| $(AM_V_at) $(PYTHON) $(top_srcdir)/contrib/devtools/symbol-check.py $(bin_PROGRAMS) $(builddir)/.libs/$$(echo $(LIBBITCOINCONSENSUS) | sed "s/.la//")$(LIBEXT) |
There was a problem hiding this comment.
Is there a nicer way to do this? Would rather not re-introduce sed.
| @@ -117,7 +117,7 @@ def test_MACHO(self): | |||
| ''') | |||
|
|
|||
| self.assertEqual(call_symbol_check(cc, source, executable, ['-lexpat', '-Wl,-platform_version','-Wl,macos', '-Wl,11.4', '-Wl,11.4']), | |||
| (1, 'libexpat.1.dylib is not in ALLOWED_LIBRARIES!\n' + | |||
| (1, f'{executable}: libexpat.1.dylib is not in ALLOWED_LIBRARIES!\n' + | |||
There was a problem hiding this comment.
I don't think adding the executable name to every line is an improvement. It's already printed in the failure.
| print(f'{split[-1]} is not in ALLOWED_LIBRARIES!') | ||
| for dylib in binary.concrete.libraries: | ||
| library = dylib.name.split('/')[-1] | ||
| if binary.abstract.header.object_type == lief.OBJECT_TYPES.LIBRARY and library == 'libbitcoinconsensus.0.dylib': |
There was a problem hiding this comment.
In general, I don't really like how these changes are mixing up the binary and library checks.
| print(f'{filename}: symbol {symbol.name} from unsupported version {version}') | ||
| ok = False | ||
| return ok | ||
|
|
||
| def check_exported_symbols(binary) -> bool: | ||
| ok: bool = True | ||
| if binary.abstract.header.object_type == lief.OBJECT_TYPES.LIBRARY: | ||
| return ok |
There was a problem hiding this comment.
I think the PR should be renamed then? If it's not actually doing the thing in the PR title.
Drafted. |
fanquake
reviewed
Feb 14, 2023
| 'KERNEL32.dll', # win32 base APIs | ||
| 'msvcrt.dll', # C standard library for MSVC | ||
| 'libssp-0.dll', # mingw-w64 stack smashing protection |
There was a problem hiding this comment.
NACK. This is accomodating a bug (in the DLL), and doing so in such a way, that it could leak into the actual binaries.
There was a problem hiding this comment.
This is accomodating a bug...
Which one?
Closing. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In the light of a new shared library it looks reasonable to start checking shared library symbols.
During testing this PR, it was pointed that
libbitcoinconsensus.sohas "a ton of unnecessary symbols being exported". Fortunately, it could be fixed in #24994.Therefore, it makes sense to draft this PR until #24994 is merged.
Guix builds on
x86_64: