Skip to content

Conversation

hebasto
Copy link
Member

@hebasto hebasto commented Apr 29, 2022

After #24322, libraries, which were promoted from EXTRA_LIBRARIES to noinst_LTLIBRARIES, are built regardless whether they are actually required as dependencies.

This PR makes build system to create such libraries conditionally.

Some other improvements of configure.ac are incorporated in this PR.

A use case example:

./configure \
  --disable-tests \
  --disable-bench \
  --without-libs \
  --without-daemon \
  --without-gui \
  --disable-fuzz-binary \
  --without-utils \
  --enable-util-util
...
configure: error: No targets! Please specify at least one of: --with-utils --with-libs --with-daemon --with-gui --enable-fuzz(-binary) --enable-bench or --enable-tests
  • with this PR the configure script finishes successfully, and the bitcoin-util binary can be built.

@DrahtBot
Copy link
Contributor

DrahtBot commented Apr 30, 2022

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.
A summary of reviews will appear here.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #26824 (build: fix configuring with only bitcoin-util by fanquake)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

hebasto added 4 commits July 27, 2022 11:37
This change fixes detection of situation when a user provides no targets
to build. For example, now it is possible to build `bitcoin-util` only.
Otherwise, they are built regardless whether they are required as
dependencies.
@hebasto
Copy link
Member Author

hebasto commented Jul 27, 2022

Rebased 6434339 -> 6f77d03 (pr25037.01 -> pr25037.02) due to the conflict with #24291.

@DrahtBot
Copy link
Contributor

Guix builds

File commit 207a228
(master)
commit 0f8a36e
(master and this pull)
SHA256SUMS.part 6b3102ab53e0b793... 8f92726f91679db3...
*-aarch64-linux-gnu-debug.tar.gz 8ec54e2536417257... cb0958694be0a860...
*-aarch64-linux-gnu.tar.gz 0dda4b9ddb6e3723... 394965e6747a1224...
*-arm-linux-gnueabihf-debug.tar.gz 0ab423a33c337179... 9e2908b225f7c1b3...
*-arm-linux-gnueabihf.tar.gz 406cbba0aaa2f1da... 8e1e93dc390de9ec...
*-arm64-apple-darwin-unsigned.dmg d27c92781e3ed97d... 6662b4228a1a1e36...
*-arm64-apple-darwin-unsigned.tar.gz 793c142ecbf1232e... 18ef21fe71a943e5...
*-arm64-apple-darwin.tar.gz 3d4d8ab65302a535... e35f98d68300ff01...
*-powerpc64-linux-gnu-debug.tar.gz a50ac273219e388a... 4bb488f0f822ac72...
*-powerpc64-linux-gnu.tar.gz 3422cd46c6173a46... 43451d474edac71c...
*-powerpc64le-linux-gnu-debug.tar.gz 0868d8e2e8fdd9a7... 9ed6be06f75ada1a...
*-powerpc64le-linux-gnu.tar.gz 23e33691640c7ac8... e164b61b99b72d2e...
*-riscv64-linux-gnu-debug.tar.gz 2f5bb7e276245b55... c23050cd67727af8...
*-riscv64-linux-gnu.tar.gz 4ee836f63bf92751... 5f0f0441198448ae...
*-win64-debug.zip e147a82769c2e3ba... 6bdf01d1c14f41e2...
*-win64-setup-unsigned.exe 9051cb9870b5330e... 3583542e8871012d...
*-win64-unsigned.tar.gz 3561608217c7f504... b00259331245f0cc...
*-win64.zip 37ba8b813661cec2... dc8b64207e5a0ed8...
*-x86_64-apple-darwin-unsigned.dmg 80891b5bbbe61d46... 34836371903353f4...
*-x86_64-apple-darwin-unsigned.tar.gz 716c1ed695d78460... 3c86da170a5b05dd...
*-x86_64-apple-darwin.tar.gz fccf2ccded08c6cf... f783a4d5c14d6c25...
*-x86_64-linux-gnu-debug.tar.gz ad1d6eeed71d8a6a... 9723e0e0a1199d62...
*-x86_64-linux-gnu.tar.gz 2154cfe4a1459ffc... dfbc407817503f90...
*.tar.gz df32b01715124731... 4af6f2809addb1a6...
guix_build.log be2f97f8040e337a... 1a5e81ad25b57d42...
guix_build.log.diff 1361bd92fbebaa1b...

@jarolrod
Copy link
Member

jarolrod commented Nov 21, 2022

GUIX hashes

x86:

cfa84024dae05de0c5f0330bb85c38c7bb10a6e70232fc13277b729d97924b37  guix-build-6f77d03c5a40/output/aarch64-linux-gnu/SHA256SUMS.part
3b5616e879009fc55f3c9952083cb5372ad360f997ac4c0e1adb1c08c0cb18c0  guix-build-6f77d03c5a40/output/aarch64-linux-gnu/bitcoin-6f77d03c5a40-aarch64-linux-gnu-debug.tar.gz
1f5732b5c46ec950472392b272c70aad3929b51537d1c9e17e5c97771726f20f  guix-build-6f77d03c5a40/output/aarch64-linux-gnu/bitcoin-6f77d03c5a40-aarch64-linux-gnu.tar.gz
c7eed7f5ab292c17c35468b008593edddb5eb79878d791df5e60aa86ab45be1b  guix-build-6f77d03c5a40/output/arm-linux-gnueabihf/SHA256SUMS.part
e8f44f3440833580078c03b8b2f3f1cc7e3da5b427c04260403a438b47c9f0db  guix-build-6f77d03c5a40/output/arm-linux-gnueabihf/bitcoin-6f77d03c5a40-arm-linux-gnueabihf-debug.tar.gz
47892f328189b72e9fce4ea6b924629433f44bfb09be1af841604e9866ad265a  guix-build-6f77d03c5a40/output/arm-linux-gnueabihf/bitcoin-6f77d03c5a40-arm-linux-gnueabihf.tar.gz
0c62238cc0bc039c9ae491b6d284424dd934ac34689490fbc19f41bf71aae3a6  guix-build-6f77d03c5a40/output/arm64-apple-darwin/SHA256SUMS.part
2cafdef84531ec6ad90b7e05f138e6d72d67d3960f58555f9dba9469bf759f74  guix-build-6f77d03c5a40/output/arm64-apple-darwin/bitcoin-6f77d03c5a40-arm64-apple-darwin-unsigned.dmg
f95b9969128cec7ce4e4be194f07a97b4e46bc79d454598a5fecb97ba95e0e18  guix-build-6f77d03c5a40/output/arm64-apple-darwin/bitcoin-6f77d03c5a40-arm64-apple-darwin-unsigned.tar.gz
bfd351156e265dfe6afcfdd078072108a4207be002f746a13774047bf26f9a26  guix-build-6f77d03c5a40/output/arm64-apple-darwin/bitcoin-6f77d03c5a40-arm64-apple-darwin.tar.gz
cdadd8db8791f31b1e6308eabeb5cb71c2ee9928e22abc543ec22f9d1af79262  guix-build-6f77d03c5a40/output/dist-archive/bitcoin-6f77d03c5a40.tar.gz
13d5a1867ad8267ea3086adce221141ab9cb61a1c03ffdcceec1f48225ad7ae6  guix-build-6f77d03c5a40/output/powerpc64-linux-gnu/SHA256SUMS.part
f762f2fdf2f7d7df0a24eb327c1b7670addfd1eb9565e456b8df8204bf0be875  guix-build-6f77d03c5a40/output/powerpc64-linux-gnu/bitcoin-6f77d03c5a40-powerpc64-linux-gnu-debug.tar.gz
dc93dea45e57dc773980b0859caf5dbc2413fc4f11750fb3e0c2f7a26bba6480  guix-build-6f77d03c5a40/output/powerpc64-linux-gnu/bitcoin-6f77d03c5a40-powerpc64-linux-gnu.tar.gz
284eb5e3ed87ffad15da865d2db63a8a6bffa737668c0c9e3ea269e5addf3e96  guix-build-6f77d03c5a40/output/powerpc64le-linux-gnu/SHA256SUMS.part
33e700e00c40d2b3eb1fc154e7aa05c660d2b82b5e50eca2046f7ab2b8394723  guix-build-6f77d03c5a40/output/powerpc64le-linux-gnu/bitcoin-6f77d03c5a40-powerpc64le-linux-gnu-debug.tar.gz
2d84ddea89b3992ed98411cb3393ab88f5f7e1aa1de71bc4e366cb72a9a735e3  guix-build-6f77d03c5a40/output/powerpc64le-linux-gnu/bitcoin-6f77d03c5a40-powerpc64le-linux-gnu.tar.gz
24ac795340185f9692414cc60547e5a6ddd836878fba1fb7fabe5dd09a11ec73  guix-build-6f77d03c5a40/output/riscv64-linux-gnu/SHA256SUMS.part
afa9bd56dca6fd4724f2e724be3b385df33d71819ff711be806b1da69bb69276  guix-build-6f77d03c5a40/output/riscv64-linux-gnu/bitcoin-6f77d03c5a40-riscv64-linux-gnu-debug.tar.gz
6b147f44e7f6bc020e9c1d2e1b1bb1d0570678e7f79d7951412dc286f387b86b  guix-build-6f77d03c5a40/output/riscv64-linux-gnu/bitcoin-6f77d03c5a40-riscv64-linux-gnu.tar.gz
93bfe759fd41b673fcf777fb8549334265aa1186f92ef07005a4915ec9a90d8f  guix-build-6f77d03c5a40/output/x86_64-apple-darwin/SHA256SUMS.part
68e5ed0e70e57a519a144f7a402bd2dc6c4d8da33c242ec9271c93fcfd5b6c72  guix-build-6f77d03c5a40/output/x86_64-apple-darwin/bitcoin-6f77d03c5a40-x86_64-apple-darwin-unsigned.dmg
4e6a886612ce09efecfa7f05bd7dcc54d700943f627fd2df0f965451688645f3  guix-build-6f77d03c5a40/output/x86_64-apple-darwin/bitcoin-6f77d03c5a40-x86_64-apple-darwin-unsigned.tar.gz
57586a51eb8eafeb654a9e771c2f5bfb1a6495a5cf956bc6584c2d274ab164fa  guix-build-6f77d03c5a40/output/x86_64-apple-darwin/bitcoin-6f77d03c5a40-x86_64-apple-darwin.tar.gz
9049d487adc08aa0d767fb5b9b0f55e0f980db1587c6d609fc8b60d9c40ea98b  guix-build-6f77d03c5a40/output/x86_64-linux-gnu/SHA256SUMS.part
d8b3f30bf6e181f7bba63788ca7dde297c0b197877ab9b43b97e5c1b0b53b3dd  guix-build-6f77d03c5a40/output/x86_64-linux-gnu/bitcoin-6f77d03c5a40-x86_64-linux-gnu-debug.tar.gz
abacdbb32354a1d5e99736d04fa756aa572bef6592e13ed8d89e623a6f6c10eb  guix-build-6f77d03c5a40/output/x86_64-linux-gnu/bitcoin-6f77d03c5a40-x86_64-linux-gnu.tar.gz
5926d8d97cf8f60bad6324da44712107bced947686432ca029bdcad69a2cbe26  guix-build-6f77d03c5a40/output/x86_64-w64-mingw32/SHA256SUMS.part
2ea8976e6d756fbaa1d2b16004621a271c664c804dcf3ef620830411f0a00841  guix-build-6f77d03c5a40/output/x86_64-w64-mingw32/bitcoin-6f77d03c5a40-win64-debug.zip
cbf4e3affc3ee6dedaf4ec9fc8810abcbba33a1933a83a65e74669b790f792e1  guix-build-6f77d03c5a40/output/x86_64-w64-mingw32/bitcoin-6f77d03c5a40-win64-setup-unsigned.exe
819f773b43ab5ae13efda7a1caed44894d1a268d08959bc0b497a9d7a24d32cd  guix-build-6f77d03c5a40/output/x86_64-w64-mingw32/bitcoin-6f77d03c5a40-win64-unsigned.tar.gz
361d404499403f318970ed6b1706cecb827a986053b0607c7c5eb2617302f2fd  guix-build-6f77d03c5a40/output/x86_64-w64-mingw32/bitcoin-6f77d03c5a40-win64.zip

@fanquake
Copy link
Member

fanquake commented Dec 5, 2022

~0. I don't think introducing the macro, and tying it into the boost usage check is much of an improvement. It also means it becomes harder to remove targets from that list when they become Boost-free?

The bitcoin-util bug (i.e configure failing with the invocation from the PR description), can be fixed with this diff:

diff --git a/configure.ac b/configure.ac
index 541f31e3f0..d00b2dbdba 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1870,7 +1870,7 @@ else
   AC_MSG_RESULT([no])
 fi
 
-if test "$build_bitcoin_wallet$build_bitcoin_cli$build_bitcoin_tx$build_bitcoin_libs$build_bitcoind$bitcoin_enable_qt$enable_fuzz_binary$use_bench$use_tests" = "nonononononononono"; then
+if test "$build_bitcoin_wallet$build_bitcoin_cli$build_bitcoin_tx$build_bitcoin_util$build_bitcoin_libs$build_bitcoind$bitcoin_enable_qt$enable_fuzz_binary$use_bench$use_tests" = "nononononononononono"; then
   AC_MSG_ERROR([No targets! Please specify at least one of: --with-utils --with-libs --with-daemon --with-gui --enable-fuzz(-binary) --enable-bench or --enable-tests])
 fi

fanquake added a commit to fanquake/bitcoin that referenced this pull request Jan 5, 2023
Fixes the issue presented in bitcoin#25037 in a single (easily backportable)
diff, with no additional refactoring/changes.

Can be tested with:
```bash
./configure \
  --disable-tests \
  --disable-bench \
  --without-libs \
  --without-daemon \
  --without-gui \
  --disable-fuzz-binary \
  --without-utils \
  --enable-util-util
```
@hebasto
Copy link
Member Author

hebasto commented Jan 13, 2023

Closing in favour of #26824.

@hebasto hebasto closed this Jan 13, 2023
fanquake added a commit that referenced this pull request Jan 13, 2023
0f883df build: fix configuring with only bitcoin-util (fanquake)

Pull request description:

  Fixes the issue presented in #25037 in a single (easily backportable) diff, with no additional refactoring/changes.

  Can be tested with:
  ```bash
  ./configure \
    --disable-tests \
    --disable-bench \
    --without-libs \
    --without-daemon \
    --without-gui \
    --disable-fuzz-binary \
    --without-utils \
    --enable-util-util
  ```

ACKs for top commit:
  TheCharlatan:
    tACK 0f883df
  hebasto:
    ACK 0f883df, tested on Ubuntu 22.04.

Tree-SHA512: 3682712405c360852c4edd90c171e21302154bf8789252c64083974a5c873cf04d97e8721c7916d5b2dafa6acd2b8dc32deecf550e90e03bcbbabbbbf75ce959
aguycalled added a commit to nav-io/navio-core that referenced this pull request Jan 27, 2023
* script, test: fix python linter E275 errors with flake8 5.0.4

* script: update python linter dependencies

* wallet: simplify ListCoins implementation

Can remove the locked coins lookup if we include them directly
inside the AvailableCoins result

* wallet: unify FindNonChangeParentOutput functions

The function is only used in ListCoins.

* Use same Python executable for subprocesses as for all-lint.py

* tests: Use unique port for ZMQ tests

The ZMQ interface tests should use unique ports as we do for the p2p and
rpc ports so that multiple instances of the test can be run at the same
time.

* test: add coverage for cross-chain wallet restore

* refactor: use braced init for integer constants instead of c style casts

* rpc: Sanitize label name in various RPCs

- importprivkey
- importaddress
- importpubkey
- listtransactions
- listsinceblock
- importmulti
- importdescriptors

* compat: use STDIN_FILENO over 0

This is already used throughout this file, and is self-documenting.

* debian: remove nonexistent files from copyright

* refactor: Add sanity checks in LabelFromValue

* test: Invalid label name coverage

* refactor: use convenience fn to auto parse non-string parameters

Minimizes code duplication and improves function naming by having
a single (overloaded) convenience function that both checks if
the parameter is a non-string parameter and automatically parses the
value if so.

* refactor: add kernel/cs_main.*

Co-authored-by: Anthony Towns <aj@erisian.com.au>

* test: Fix feature_startupnotify intermittent issue

* refactor: Make `ThreadHTTP` return void

The `bool` return value was introduced in 755aa05.

It has been not used since 8d3f46e.

No behavior change.

* build: remove already tested headers from AC_CHECK_HEADERS

These headers are already included in a default set which are checked
early during configure.

We already use at least sys/types.h and unistd.h unconditionally in
configure.

* refactor: Work around Werror=free-nonheap-object in AssumeCalculateMemPoolAncestors

* build: fix configuring with only bitcoin-util

Fixes the issue presented in bitcoin#25037 in a single (easily backportable)
diff, with no additional refactoring/changes.

Can be tested with:
```bash
./configure \
  --disable-tests \
  --disable-bench \
  --without-libs \
  --without-daemon \
  --without-gui \
  --disable-fuzz-binary \
  --without-utils \
  --enable-util-util
```

* random: remove windows-only compat.h include in randomenv

Note that this was probably only here to indirectly receive windows.h
via another include in compat.h (windows.h or winreg.h aren't included
there).

Also note that compat.h is already pulled in here for everyone via
util/time.h, so including inside a windows only ifdef is secondarily
redundant.

* randomenv: consolidate WIN32 #ifdefs

Order includes.
Remove // for xyz comments

* test: Add signs P2TR and RAWSCRIPT to MiniWallet sign_tx

* test: Add "include immature coinbase" flag to MiniWallet get_utxos

* test: Add "include mempool" flag to MiniWallet rescan_utxos

* test: Run feature_bip68_sequence.py with MiniWallet

* doc: Update Boost version in doc/dependencies.md

* scripts: add PE Canary check to security-check

* rpc: Return accurate results for scanblocks

This makes use of undo data to accurately verify results
from blockfilters.

* p2p, rpc: don't allow past absolute timestamp in `setban`

* test: add coverage for absolute timestamp in `setban`

* [validation] when quitting early in AcceptPackage, set package_state and tx result

Bug: not setting package_state means package_state.IsValid() == true and
the caller does not know that this failed.

We won't be validating this transaction again, so it makes sense to return this
failure to the caller.

Rename package_state to package_state_quit_early to make it more clear
what this variable is used for and what its scope is.

Co-authored-by: Greg Sanders <gsanders87@gmail.com>

* [test] package validation quits early due to non-policy, non-missing-inputs failure

* [validation] return effective feerate from mempool validation

* ci: Fix ci_native_fuzz_msan CONTAINER_NAME

* ci: Create named symbol for BINS_SCRATCH_DIR

Also, create the dir a bit earlier, right after it was put in the PATH.

* ci: Remove unused busybox workaround

The find workaround is no longer needed after commit
d3d547c

* ci: Run one task with all tests on credits

* doc: clarify -i2pacceptincoming help documentation

and also hoist the default setting to a constexpr and
remove unused f-string operators in a related functional test.

* doc: update bandwidth section of I2P documentation

* doc: update/clarify/de-emphasize I2P transient address section

* doc: remove recommended I2P router versions

as these go stale and users will generally install the current versions available.

* [validation] return wtxids of other transactions whose fees were used

* [rpc] return effective-feerate in testmempoolaccept and submitpackage

* [rpc] return effective-includes in testmempoolaccept and submitpackage

* [validation] remove PackageMempoolAcceptResult::m_package_feerate

This value creates an extremely confusing interface as its existence is
dependent upon implementation details (whether something was submitted
on its own, etc). MempoolAcceptResult::m_effective_feerate is much more
helpful, as it always exists for submitted transactions.

* [doc] release note effective-feerate and effective-includes RPC results

No release note for submitpackage because it is regtest-only.

* [refactor] rename variables in AcceptPackage for clarity

* [validation] return MempoolAcceptResult for every tx on PCKG_TX failure

This makes the interface more predictable and useful. The caller
understands one or more transactions failed, and can learn what happened
with each transaction. We already have this information, so we might as
well return it.

It doesn't make sense to do this for other PackageValidationResult
values because:
- PCKG_RESULT_UNSET: this means everything succeeded, so the individual
  failures are no longer accurate.
- PCKG_MEMPOOL_ERROR: something went wrong with the mempool logic;
  transaction failures might not be meaningful.
- PCKG_POLICY: this means something was wrong with the package as a
  whole. The caller should use the PackageValidationState to find the
  error, rather than looking at individual MempoolAcceptResults.

* doc: net: fix link to onion address encoding scheme [ONIONADDRESS]

Instead of referring to a fixed line number to a file in master (which
is obviously always quickly outdated), use a permalink tied to the
latest commit.

* doc: fix up -netinfo relaytxes help

Co-authored-by: "MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>"

* scripted-diff: ci: Rework docker naming

DOCKER in names is confusingly used as synonym for "image", "container",
and "ci". Fix the confusion by picking the term that fits the context.

-BEGIN VERIFY SCRIPT-
 ren() { sed -i "s:$1:$2:g" $( git grep -l "$1" ) ; }

 ren DOCKER_PACKAGES CI_BASE_PACKAGES
 # This better reflects that they are the common base for all CI
 # containers.

 ren DOCKER_ID CI_CONTAINER_ID
 # This is according to the documentation of "--detach , -d: Run
 # container in background and print container ID".

 ren DOCKER_NAME_TAG CI_IMAGE_NAME_TAG
 # This avoids confusing with CONTAINER_NAME and clarifies that it is an
 # image.

 ren DOCKER_ADMIN CI_CONTAINER_CAP
 # This clarifies that it is a capability added to the container.

 ren DOCKER_CI_CMD_PREFIX CI_EXEC_CMD_PREFIX
 # This brings it in line with the CI_EXEC naming.

-END VERIFY SCRIPT-

* ci: Stop and remove CI container

* test: Fix intermittent timeout in p2p_permissions.py

* test: Fix wrong types passed to RPCs

* rpc: Run type check against RPCArgs

* doc: add databases/py-sqlite3 to FreeBSD test suite deps

* Wallet/RPC: Allow specifying min & max chain depth for inputs used by fund calls

Enables users to craft BIP-125 replacements with changes to the output
list, ensuring that if additional funds are needed they will be added.

* rpc: add minconf and maxconf options to sendall

* doc: move errant release note to doc/

* test: skip sqlite3 tests if it isn't available

Fixes bitcoin#26819. Related too bitcoin#26873.

* doc: remove usages of C++11

Now it's just the standard library.

* Update to mention restoring wallet via GUI

* Change dots to an ellipsis and fix capitalization

Matches ellipsis usage in the "Restore" section.

* test: remove `-spendzeroconfchange` setting from mempool_limit.py

Since this test was changed to use MiniWallet instead of the Bitcoin
Core wallet (see commit d447ded),
the setting doesn't have any effect and hence can be removed.

* ci: Bump vcpkg to the latest version 2023.01.09

Dependency changes (2022.09.27 - 2023.01.09):
 - boost 1.80.0#0 -> 1.81.0#0
 - libevent 2.1.12#6 -> libevent 2.1.12#7
 - sqlite3 3.39.2#0 -> 3.40.0#1

* ci: Bump `ccache` version to the latest 4.7.4 in "Win64 native" task

* test: wallet: add coverage for `-spendzeroconfchange` setting

* build: allow NO_BOOST=1 in depends

* build: allow NO_LIBEVENT=1 in depends

* test: refactor: simplify p2p_permissions.py by using MiniWallet

Also, use the pre-mined chain of the test framework rather than mining
100 blocks manually on each run.

* test: Remove redundant function call

* ci: Use pyenv's `python-build` to install Python in lint task

* ci: Bump lint task image to Ubuntu Jammy

* test: add `rescan_utxos` in MiniWallet's initialization

this simplifies usage when MiniWallet is used with a pre-mined chain.

* test: remove redundant blocks generation logic

those tests already have enough mature utxos from the pre-mined chain.

* test: simplify tests by using the pre-mined chain

* test: Return wtxid from create_self_transfer_multi

This is not used right now, but may be in the future. Also, it
simplifies the create_self_transfer return logic

* test: Refactor MiniWallet sign_tx

To make the code less verbose and easier to read.

* test: Return chain of MiniWallet txs from MiniWallet chain method

* test: Return fee from MiniWallet

* test: Run mempool_packages.py with MiniWallet

* build: move rpc/request from util lib to common

This is JSON RPC request code that doesn't need to be in util, and
should not be required by the kernel.

* test: Add test for missing and omitted required arg

* doc: Fix incorrect sendmany RPC doc

This enables the type check and fixes the wrong docs.

Otherwise the enabled check would lead to test errors, such as:

> "wallet_labels.py", line 96, in run_test
>     node.sendmany(
>
> test_framework.authproxy.JSONRPCException:
>  JSON value of type null is not of expected type string (-3)

* refactor: Remove const to fix performance-move-const-arg clang-tidy errors

The warnings look like:

src/rpc/util.h:192:19: error: std::move of the const variable 'name' has no effect; remove std::move() or make the variable non-const [performance-move-const-arg,-warnings-as-errors]
        : m_names{std::move(name)},
                  ^~~~~~~~~~    ~

* refactor: Introduce is_top_level_arg

* doc: Properly report optional RPC args

* refactor: Remove duplication of clang-tidy's check names

* ci: Add missing lint dependency

* hash: add HashedSourceWriter

This class is the counterpart to CHashVerifier, in that it
writes data to an underlying source stream,
while keeping a hash of the written data.

* addrdb: Only call Serialize() once

The previous logic would call it once for serializing into the filestream,
and then again for serializing into the hasher. If AddrMan was changed
in between these calls by another thread, the resulting peers.dat would
be corrupt with non-matching checksum and data.
Fix this by using HashedSourceWriter, which writes the data
to the underlying stream and keeps track of the hash in one go.

* ci: Bump --combinedlogslen to debug intermittent issues

* doc: Clarify debian copyright comment

* Bump minimum python version to 3.7

* Revert "contrib: Fix capture_output in getcoins.py"

This reverts commit be59bd1
because the changes are no longer needed.

* scripted-diff: Use new python 3.7 keywords

-BEGIN VERIFY SCRIPT-
 sed -i 's/universal_newlines/text/g' $(git grep -l universal_newlines)
-END VERIFY SCRIPT-

* test: add an easy way to run linters locally

Adds a Dockerfile configuration
that allows straightforward running of linters with compatible versions
locally. This removes a ton of annoyance when trying to appease CI,
because many of the linter versions are quite old and difficult to
maintain locally.

I realize that people may not be thrilled to more ancillary tooling to
the repo, but I think this makes a lot of sense given the linter
versions listed in this container configuration are dictated by this
repo (within the CI configuration), so having these things live in
two separate places is a recipe for version mismatches.

Eventually we can likely just use this container on CI directly to avoid
any chance of inconsistencies between local dev experience and CI.

* lint: specify the right commit range when running locally

When running lints on Cirrus, a special envvar is set ($CIRRUS_PR);
emulate this when running linters locally by setting $LOCAL_BRANCH
to any value.

* clang-tidy: Fix `performance-move-const-arg` in headers

See https://clang.llvm.org/extra/clang-tidy/checks/performance/move-const-arg.html

* clang-tidy: Fix `performance-no-automatic-move` in headers

See https://clang.llvm.org/extra/clang-tidy/checks/performance/no-automatic-move.html

* doc: add release note for bitcoin#25957 (fast wallet rescan)

* test: refactor: simplify p2p_tx_download.py by using MiniWallet

* test: refactor: simplify p2p_eviction.py by using MiniWallet

Also, use the pre-mined chain of the test framework rather than
mining 100 blocks manually on each run.

* Add missing includes to fix gcc-13 compile error

* RPC: make RPCResult::MatchesType return useful errors

* [fuzz] Actually use mocked mempool in tx_pool target

* rpc: Throw more user friendly arg type check error

* mempool: Don't share mempool with dbcache in blocksonly

When -blockonly is set, reduce mempool size to 5MB unless -maxmempool
is also set.

See bitcoin#9569

* doc: Update blocksonly behaviour in reduce-memory

Changes to the default mempool allocation size now documented.

Provides users with guidance on the mempool implications of -blocksonly
mode, along with instructions on how to re-enable old behaviour.

* doc: release note on mempool size in -blocksonly

Adds a release note detailing the new mempool sizing behaviour when
running in blocksonly mode, and instruction on how to override the new
defaults.

* Add unit test for ComputeTapleafHash

* test: Fix intermittent feature_rbf issue

* scripted-diff: use RPCArg::Optional::OMITTED over OMITTED_NAMED_ARG

-BEGIN VERIFY SCRIPT-
sed -i -e "/Deprecated alias for OMITTED, can be removed/d" src/rpc/util.h src/rpc/util.cpp
sed -i -e "s/OMITTED_NAMED_ARG/OMITTED/g" $(git grep -l "OMITTED_NAMED_ARG" src/)
-END VERIFY SCRIPT-

* doc: improve doc for RPCArg::Optional::OMITTED

* depends: fix systemtap download URL

* depends: systemtap 4.8

* build: use more recommended sqlite3 compile options

See https://www.sqlite.org/compile.html.

DSQLITE_DQS
> This setting disables the double-quoted string literal misfeature.

DSQLITE_DEFAULT_MEMSTATUS
> This setting causes the sqlite3_status() interfaces that track
> memory usage to be disabled.
> This helps the sqlite3_malloc() routines run much faster, and since
> SQLite uses sqlite3_malloc() internally, this helps to make the
> entire library faster.

DSQLITE_OMIT_DEPRECATED
> Omitting deprecated interfaces and features will not help SQLite
> to run any faster.
> It will reduce the library footprint, however. And it is the
> right thing to do.

DSQLITE_OMIT_SHARED_CACHE
> Omitting the possibility of using shared cache allows many
> conditionals in performance-critical sections of the code to be
> eliminated. This can give a noticeable improvement in performance.

Also: https://www.sqlite.org/sharedcache.html
> Shared-cache mode is an obsolete feature.
> The use of shared-cache mode is discouraged.
> Most use cases for shared-cache are better served by WAL mode.
> Applications that build their own copy of SQLite from source code
> are encouraged to use the -DSQLITE_OMIT_SHARED_CACHE compile-time
> option, as the resulting binary will be both smaller and faster.

DSQLITE_OMIT_JSON
Starting with sqlite 3.38.0 the JSON extension became opt-out rather
than opt-in, so we disable it here.

--disable-rtree
> An R-Tree is a special index that is designed for doing range queries.
> R-Trees are most commonly used in geospatial systems...
https://www.sqlite.org/rtree.html

--disable-fts4 --disable-fts5
> FTS5 is an SQLite virtual table module that provides full-text
> search functionality to database applications.

DSQLITE_LIKE_DOESNT_MATCH_BLOBS
> simplifies the implementation of the LIKE optimization and allows
> queries that use the LIKE optimization to run faster.

DSQLITE_OMIT_DECLTYPE
> By omitting the (seldom-needed) ability to return the declared type of
> columns from the result set of query, prepared statements can be made
> to consume less memory.

DSQLITE_OMIT_PROGRESS_CALLBACK
> By omitting this interface, a single conditional is removed from the
> inner loop of the bytecode engine, helping SQL statements to run slightly
> faster.

DSQLITE_OMIT_AUTOINIT
> with the SQLITE_OMIT_AUTOINIT option, the automatic initialization is omitted.
> This helps many API calls to run a little faster
> it also means that the application must call sqlite3_initialize()
manually.

* build: pass --enable-debug to sqlite when DEBUG=1

* wallet: permit mintxfee=0

Fixes bitcoin#26797

Permit nodes to use a mintxfee of `0` if they choose.
Values below 0 are handled by the ParseMoney() check.

* test: Avoid rpc timeout in p2p_headers_sync_with_minchainwork

* [block encodings] Make CheckBlock mockable for PartiallyDownloadedBlock

* [block encodings] Avoid fuzz blocking asserts in PartiallyDownloadedBlock

* [fuzz] Add PartiallyDownloadedBlock target

* [fuzz] Assert that omitting missing transactions always fails block reconstruction

* build: fix usage of -Wloop-analysis

Looks like I introduced this in
5ced925.

* depends: systemtap: remove variadic params that trigger compiler warnings

* init: Remove sensitive flag from rpcbind

* refactor: Remove c_str from util/check

* streams: Add DataStream without ser-type and ser-version

The moved parts can be reviewed with "--color-moved=dimmed-zebra".
The one-char changes can be reviewed with "--word-diff-regex=.".

* ci: Fix APPEND_APT_SOURCES_LIST trying to modify the host system

* txorphange: Drop redundant originator arg from GetTxToReconsider

* net_processing: only process orphans before messages

Previously, when we processed a new tx we would attempt to ATMP any
orphans that considered the new tx a parent immediately, but would only
accept at most one such tx, leaving any others to be considered on a
future run of ProcessMessages(). With this patch, we don't attempt any
orphan processing immediately after receiving a tx, instead deferring
all of them until the next call to ProcessMessages().

* net_processing: Don't process tx after processing orphans

If we made progress on orphans, consider that enough work for this peer
for this round of ProcessMessages. This also allows cleaning up the api
for TxOrphange:GetTxToReconsider().

* net_processing: indicate more work to do when orphans are ready to reconsider

When PR#15644 made orphan processing interruptible, it also introduced a
potential 100ms delay between processing of the first and second newly
reconsiderable orphan, because it didn't check if the orphan work set
was non-empty after invoking ProcessMessage(). This adds that check, so
that ProcessMessages() will return true if there are orphans to process,
usually avoiding the 100ms delay in CConnman::ThreadMessageHandler().

* Use DataStream where possible

* Remove unused CDataStream::SetType

The last use was removed in the previous commit.

* Comment mcl/bls libs

* fix linter

* fix linter

* fix Makefile.am

---------

Co-authored-by: Andrew Chow <github@achow101.com>
Co-authored-by: Jon Atack <jon@atack.com>
Co-authored-by: furszy <matiasfurszyfer@protonmail.com>
Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
Co-authored-by: Kristaps Kaupe <kristaps@blogiem.lv>
Co-authored-by: Sebastian Falbesoner <sebastian.falbesoner@gmail.com>
Co-authored-by: Pasta <pasta@dashboost.org>
Co-authored-by: glozow <gloriajzhao@gmail.com>
Co-authored-by: fanquake <fanquake@gmail.com>
Co-authored-by: Aurèle Oulès <aurele@oules.com>
Co-authored-by: stickies-v <stickies-v@protonmail.com>
Co-authored-by: Anthony Towns <aj@erisian.com.au>
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
Co-authored-by: Miles Liu <miles@bung.cc>
Co-authored-by: brunoerg <brunoely.gc@gmail.com>
Co-authored-by: Greg Sanders <gsanders87@gmail.com>
Co-authored-by: Juan Pablo Civile <elementohb@gmail.com>
Co-authored-by: ishaanam <ishaana.misra@gmail.com>
Co-authored-by: John Moffett <john.moff@gmail.com>
Co-authored-by: Kolby ML <31669092+KolbyML@users.noreply.github.com>
Co-authored-by: kouloumos <kouloumosa@gmail.com>
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
Co-authored-by: James O'Beirne <james.obeirne@pm.me>
Co-authored-by: dergoegge <n.goeggi@gmail.com>
Co-authored-by: willcl-ark <will@256k1.dev>
Co-authored-by: Cory Fields <cory-nospam-@coryfields.com>
Co-authored-by: alex v <alex@nav.community>
@bitcoin bitcoin locked and limited conversation to collaborators Jan 13, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants