Skip to content

Conversation

martinus
Copy link
Contributor

@martinus martinus commented Nov 19, 2023

The class CTxOut has a member CAmount which is an int64_t, and on ARM 32bit int64_t are 8 byte aligned, which is larger than the pointer alignment of 4 bytes.

So for CCoinsMap to be able to use the pool, we need to use the alignment of the member instead of just alignof(void*).

This fixes #28906 (first noted in #28718 (comment)) and #28440.

@DrahtBot
Copy link
Contributor

DrahtBot commented Nov 19, 2023

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK pinheadmz, hebasto, theStack
Concept ACK pablomartin4btc
Stale ACK sipa

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Copy link
Contributor

@mzumsande mzumsande left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pattern of alignof(void*) was used in 3 places in #25325 (also bench and tests) - looks like those other spots could be changed as well? (just a superficial observation, I didn't review that PR)

@fanquake fanquake added this to the 26.0 milestone Nov 19, 2023
@hebasto
Copy link
Member

hebasto commented Nov 19, 2023

The pattern of alignof(void*) was used in 3 places in #25325 (also bench and tests) - looks like those other spots could be changed as well? (just a superficial observation, I didn't review that PR)

Test doesn't fail now. Maybe replace int with int64_t?

@martinus
Copy link
Contributor Author

martinus commented Nov 19, 2023

The pattern of alignof(void*) was used in 3 places

Thanks, I forgot about these... I'll change the PoolAllocator to default to the alignment of the given type, that makes much more sense and reduces the code.

Test doesn't fail now. Maybe replace int with int64_t?

Good idea, I'll change that too

This changes the PoolAllocator to default the alignment to the given type. This makes the code simpler, and most importantly
fixes a bug on ARM 32bit that caused OOM: The class CTxOut has a member CAmount which is an int64_t and on ARM 32bit int64_t
are 8 byte aligned which is larger than the pointer alignment of 4 bytes. So for CCoinsMap to be able to use the pool, we
need to use the alignment of the member instead of just alignof(void*).
@martinus martinus force-pushed the 2023-11-fix-pool-allocation-alignment-on-ARM branch from 72918e0 to 486c291 Compare November 19, 2023 17:48
@hebasto
Copy link
Member

hebasto commented Nov 19, 2023

Concept ACK.

@pinheadmz
Copy link
Member

concept ACK
running this branch on my RPi same as #28718 (comment) will update

@sipa
Copy link
Member

sipa commented Nov 19, 2023

utACK 486c291

@pinheadmz
Copy link
Member

Up to height 376554 now which is way further than I crashed at on 26rc2. htop says RAM is only about 890MB used. Are there any RPCs I should check?

Copy link
Member

@hebasto hebasto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 486c291, tested on ARM 32-bit (ODROID-HC1 + Armbian 23.8.3 jammy) both native and Guix binaries.

The first commit should not introduce any behaviour change in any other release platforms as they are 64-bit and it holds that alignof(T) == alignof(void*) == 8 for all of them.

I agree with the second commit diff. It indeed handles the case where "int64_t is aligned to 8 bytes but void* is aligned to 4 bytes". But, due do its design flaw, the test still not failing in such a case, i.e., on ARM 32-bit, if the first commit were reverted. I believe that the reason of that is the test does not cover cases where IsFreeListUsable returns false only due to the alignment <= ELEM_ALIGN_BYTES condition fails while the bytes <= MAX_BLOCK_SIZE_BYTES condition still holds. It can be improved in a follow up.

nit: The 486c291 commit message has a stray substring in the end.

Suggesting for backporting to the 26.x branch. EDIT: nm, already tagged :)

@martinus
Copy link
Contributor Author

@pinheadmz I don't think there's a need to test any any RPC calls, if sync doesn't go OOM it means it is now working :)

@pinheadmz
Copy link
Member

confirmed unit passes on arm32 but also passes without the actual bugfix commit. so effectively this on top of d752349:

commit 2c7f49e83a5d05a5ba8b29d52108c14bf4d65af7
Author: Martin Leitner-Ankerl <martin.ankerl@gmail.com>
Date:   Sun Nov 19 18:43:15 2023 +0100

    pool: change memusage_test to use int64_t
    
    If alignment of the PoolAllocator would be insufficient, then the test would fail. This also catches the issue with ARM 32bit, where int64_t is aligned to 8 bytes but void* is aligned to 4 bytes.catches cause the ARM 32bit tests to fail when a

diff --git a/src/test/pool_tests.cpp b/src/test/pool_tests.cpp
index 8a07e09a44..9590f80a33 100644
--- a/src/test/pool_tests.cpp
+++ b/src/test/pool_tests.cpp
@@ -156,21 +156,21 @@ BOOST_AUTO_TEST_CASE(random_allocations)
 
 BOOST_AUTO_TEST_CASE(memusage_test)
 {
-    auto std_map = std::unordered_map<int, int>{};
-
-    using Map = std::unordered_map<int,
-                                   int,
-                                   std::hash<int>,
-                                   std::equal_to<int>,
-                                   PoolAllocator<std::pair<const int, int>,
-                                                 sizeof(std::pair<const int, int>) + sizeof(void*) * 4,
+    auto std_map = std::unordered_map<int64_t, int64_t>{};
+
+    using Map = std::unordered_map<int64_t,
+                                   int64_t,
+                                   std::hash<int64_t>,
+                                   std::equal_to<int64_t>,
+                                   PoolAllocator<std::pair<const int64_t, int64_t>,
+                                                 sizeof(std::pair<const int64_t, int64_t>) + sizeof(void*) * 4,
                                                  alignof(void*)>>;
     auto resource = Map::allocator_type::ResourceType(1024);
 
     PoolResourceTester::CheckAllDataAccountedFor(resource);
 
     {
-        auto resource_map = Map{0, std::hash<int>{}, std::equal_to<int>{}, &resource};
+        auto resource_map = Map{0, std::hash<int64_t>{}, std::equal_to<int64_t>{}, &resource};
 
         // can't have the same resource usage
         BOOST_TEST(memusage::DynamicUsage(std_map) != memusage::DynamicUsage(resource_map));

@hebasto
Copy link
Member

hebasto commented Nov 20, 2023

If anyone consider the second commit as a blocker, it might be dropped from this PR, no?

If alignment of the PoolAllocator would be insufficient, then the test would fail. This also catches the issue with ARM 32bit,
where int64_t is aligned to 8 bytes but void* is aligned to 4 bytes. The test adds a check to ensure the pool has allocated
a minimum number of chunks
@martinus martinus force-pushed the 2023-11-fix-pool-allocation-alignment-on-ARM branch from 486c291 to d5b4c0b Compare November 20, 2023 16:11
@martinus
Copy link
Contributor Author

I've updated the test in the PR, this time I think it should have caught the issue. It should be reproducible by adding an alignof(void*) back as the PoolAllocator's last template argument

@hebasto
Copy link
Member

hebasto commented Nov 20, 2023

A failure in the "Win64 native, VS 2022" CI job is unrelated. See: #28905.

Copy link
Contributor

@theStack theStack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK

Due to a lack of real hardware I've set up an emulated 32-bit ARM system using qemu, maybe someone finds this useful: https://gist.github.com/theStack/6eaeadd3fa1e521ed038b1ed93c101d6

(Of course, the obvious drawback is that it's very sloooow.)

Copy link
Member

@pinheadmz pinheadmz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK d5b4c0b

Synced up to height 472600 on ARM 32 no issues, < 1 GB RAM used.
Also confirmed the test fails without the patch:

test/pool_tests.cpp(189): error: in "pool_tests/memusage_test": check resource.NumAllocatedChunks() >= min_num_allocated_chunks has failed [1 < 157]

Great catch, great work!

Show Signature
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

ACK d5b4c0b69e543de51bb37d602d488ee0949ba185
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEE5hdzzW4BBA4vG9eM5+KYS2KJyToFAmVbpSoACgkQ5+KYS2KJ
yTrHlw/9ERNwDT6jLISX7vGjXTY2/wxzItPlJbUBk6ieYHzAny0jvXm/NdxNMLpi
j5HUWX/gPAHOPz5csX7lKA0VAgIEfeBd5llMSI1H8nvQOwiyKFKHE8ijhyJh0hoS
oBW+lLbZwxp3tBAl6QHeiVYD4NG0j2BX66qvmfmAzPPYIEvqS/4x1JzUpLztRRgv
nukx4XjPIAecQO1bK1aQZoQFBgHziQYeUNEwsvdJ6eiWHlJx2iUgZPNikGP43pGr
cEPra1rQEz9MaRePX0FQvgky66ldn+MdoAL1xxnKhuINpX1xR0xI2M2FLYfPDxDG
IYz8gxmFAgWksc0gQt7OD0fIXrLVADWwc4QwBB+pCEereZC5by456/V6VCe8NXeD
4DeEyvFygaMVVrv1CeliJ08Tu2tFbf9uDOpkhWliEpEyf62cCJaOK1hT1iVDEOZC
ZadYowUeTSObRv+brITAqOBaVIWhufGU4PfKgifj6ndB/Q8UD+p8qMKN6x+fcVvs
UxWctlnNsgMcXX20cL54Kbspd44RiKvpA5i8JZXWh8QCsO5+k9L/IQV3vf27duMs
jH8gOQN8tK9+W89e9M1nTtIWf/M/RPQdgDZfQjkJwBLg9x2daPv6RcSkUyS8VPz1
po4PF2mVKvzq9xa07Q7PnblISl7Eb8MDgH/TJ/g+cSAw62/h0gM=
=OBRK
-----END PGP SIGNATURE-----

pinheadmz's public key is on keybase

Copy link
Member

@hebasto hebasto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re-ACK d5b4c0b, the only change since my recent review is an updated test.

I've tried to break the updated test with the diff as follows:

--- a/src/test/pool_tests.cpp
+++ b/src/test/pool_tests.cpp
@@ -163,7 +163,8 @@ BOOST_AUTO_TEST_CASE(memusage_test)
                                    std::hash<int64_t>,
                                    std::equal_to<int64_t>,
                                    PoolAllocator<std::pair<const int64_t, int64_t>,
-                                                 sizeof(std::pair<const int64_t, int64_t>) + sizeof(void*) * 4>>;
+                                                 sizeof(std::pair<const int64_t, int64_t>) + sizeof(void*) * 4,
+                                                 alignof(void*)>>;
     auto resource = Map::allocator_type::ResourceType(1024);
 
     PoolResourceTester::CheckAllDataAccountedFor(resource);

It passes on x86_64 and fails on ARM 32-bit as expected.

Copy link
Member

@pablomartin4btc pablomartin4btc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK

Copy link
Contributor

@theStack theStack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested ACK d5b4c0b

Using the emulated 32-bit ARM machine using qemu and Debian bookworm for armhf,
I could both reproduce issue #28906 on master and verify the fix in this PR. Having 2GB of RAM available without swap memory, bitcoind failed due to OOM on block height 322923 on the master branch, as the coins flush never happened up to that height due to an underestimation of the cache size. On the PR branch, the flushes happen regularly before that height as expected, and the RAM usage never exceeds 0.9 GB.

The code looks correct and as mentioned by hebasto above, shouldn't have any effect on the other architectures (all of them being 64-bit) that we support.

I also verified that the modified memusage_test succeeds on the PR branch and fails if cherry-picked on master.

Thanks for the fix!

@fanquake fanquake merged commit e9beaa7 into bitcoin:master Nov 22, 2023
@fanquake
Copy link
Member

Will be backported in #28872.

fanquake pushed a commit to fanquake/bitcoin that referenced this pull request Nov 22, 2023
This changes the PoolAllocator to default the alignment to the given type. This makes the code simpler, and most importantly
fixes a bug on ARM 32bit that caused OOM: The class CTxOut has a member CAmount which is an int64_t and on ARM 32bit int64_t
are 8 byte aligned which is larger than the pointer alignment of 4 bytes. So for CCoinsMap to be able to use the pool, we
need to use the alignment of the member instead of just alignof(void*).

Github-Pull: bitcoin#28913
Rebased-From: ce881bf
fanquake pushed a commit to fanquake/bitcoin that referenced this pull request Nov 22, 2023
If alignment of the PoolAllocator would be insufficient, then the test would fail. This also catches the issue with ARM 32bit,
where int64_t is aligned to 8 bytes but void* is aligned to 4 bytes. The test adds a check to ensure the pool has allocated
a minimum number of chunks

Github-Pull: bitcoin#28913
Rebased-From: d5b4c0b
Copy link
Member

@pablomartin4btc pablomartin4btc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested ACK d5b4c0b

Reproduced the issue on master using emulated ARM 32 getting the OOM at height 318617.
2023-11-22T08:24:03Z UpdateTip: new best=00000000000000001771de0c2dc0ff95d619943d4160308c93f9184fcee8d5f8 height=318617 version=0x00000002 log2_work=80.467750 tx=45766401 date='2014-09-01T14:44:48Z' progress=0.049383 cache=107.7MiB(14296037txo)
[44951.075682] b-addcon invoked oom-killer: gfp_mask=0x140cca(GFP_HIGHUSER_MOVABLE|__GFP_COMP), order=0, oom_score_adj=0
[44951.078750] CPU: 0 PID: 15497 Comm: b-addcon Not tainted 6.1.0-13-armmp-lpae #1  Debian 6.1.55-1
[44951.078960] Hardware name: Generic DT based system
[44951.079486]  unwind_backtrace from show_stack+0x18/0x1c
[44951.080028]  show_stack from dump_stack_lvl+0x40/0x4c
[44951.080112]  dump_stack_lvl from dump_header+0x50/0x1f4
[44951.080193]  dump_header from oom_kill_process+0x22c/0x238
[44951.080260]  oom_kill_process from out_of_memory+0x218/0x34c
[44951.080327]  out_of_memory from __alloc_pages+0xe64/0x1088
[44951.080394]  __alloc_pages from __folio_alloc+0x18/0x3c
[44951.080576]  __folio_alloc from __filemap_get_folio+0x160/0x40c
[44951.080654]  __filemap_get_folio from filemap_fault+0x25c/0x9c4
[44951.080723]  filemap_fault from __do_fault+0x44/0x158
[44951.080814]  __do_fault from handle_mm_fault+0xd38/0x12f4
[44951.080906]  handle_mm_fault from do_page_fault+0x17c/0x324
[44951.080998]  do_page_fault from do_PrefetchAbort+0x38/0x88
[44951.081087]  do_PrefetchAbort from ret_from_exception+0x0/0x28
[44951.081213] Exception stack(0xf0a89fb0 to 0xf0a89ff8)
[44951.081415] 9fa0:                                     01c70ed8 00000081 00000001 00000000
[44951.081536] 9fc0: 00000000 00000001 af3bdb88 00000013 6dc0dff0 00000013 6dc0e008 af3bdc6c
[44951.081656] 9fe0: 000000f0 af3bda9c b6bf378f b6bb7610 60030030 ffffffff
[44951.081995] Mem-Info:
[44951.082176] active_anon:57 inactive_anon:497856 isolated_anon:0
[44951.082176]  active_file:14 inactive_file:33 isolated_file:0
[44951.082176]  unevictable:64 dirty:0 writeback:0
[44951.082176]  slab_reclaimable:2360 slab_unreclaimable:2143
[44951.082176]  mapped:3 shmem:94 pagetables:1168
[44951.082176]  sec_pagetables:0 bounce:0
[44951.082176]  kernel_misc_reclaimable:0
[44951.082176]  free:7475 free_pcp:372 free_cma:0
[44951.082844] Node 0 active_anon:228kB inactive_anon:1991424kB active_file:56kB inactive_file:132kB unevictable:256kB isolated(anon):0kB isolated(file):0kB mapped:12kB dirty:0kB writeback:0kB shmem:376kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 1402880kB writeback_tmp:0kB kernel_stack:584kB pagetables:4672kB sec_pagetables:0kB all_unreclaimable? no
[44951.083418] DMA free:29468kB boost:0kB min:22528kB low:28160kB high:33792kB reserved_highatomic:2048KB active_anon:0kB inactive_anon:689528kB active_file:80kB inactive_file:132kB unevictable:0kB writepending:0kB present:786432kB managed:747580kB mlocked:0kB bounce:0kB free_pcp:724kB local_pcp:724kB free_cma:0kB
[44951.083926] lowmem_reserve[]: 0 0 1280 1280
[44951.084301] HighMem free:432kB boost:0kB min:512kB low:10724kB high:20936kB reserved_highatomic:0KB active_anon:228kB inactive_anon:1301896kB active_file:20kB inactive_file:40kB unevictable:256kB writepending:0kB present:1310720kB managed:1310720kB mlocked:256kB bounce:0kB free_pcp:764kB local_pcp:764kB free_cma:0kB
[44951.084665] lowmem_reserve[]: 0 0 0 0
[44951.084788] DMA: 139*4kB (UMEH) 226*8kB (UME) 174*16kB (UME) 100*32kB (ME) 98*64kB (UME) 62*128kB (ME) 27*256kB (M) 0*512kB 0*1024kB 0*2048kB 0*4096kB 0*8192kB = 29468kB
[44951.085242] HighMem: 0*4kB 0*8kB 11*16kB (U) 8*32kB (U) 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB 0*8192kB = 432kB
[44951.085518] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[44951.085660] 150 total pagecache pages
[44951.085742] 0 pages in swap cache
[44951.085802] Free swap  = 0kB
[44951.085858] Total swap = 0kB
[44951.085946] 524288 pages RAM
[44951.085997] 327680 pages HighMem/MovableOnly
[44951.086059] 9713 pages reserved
[44951.086117] 4096 pages cma reserved
[44951.086203] Tasks state (memory values in pages):
[44951.086277] [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
[44951.086484] [    212]     0   212     5251      182    53248        0         -1000 systemd-udevd
[44951.086657] [    267]   997   267     4934      150    49152        0             0 systemd-timesyn
[44951.086788] [    290]     0   290      946      146    32768        0             0 dhclient
[44951.086912] [    292]     0   292     1766       46    45056        0             0 cron
[44951.087031] [    293]   100   293     1551      113    36864        0          -900 dbus-daemon
[44951.087160] [    298]     0   298     2752      177    45056        0             0 systemd-logind
[44951.087286] [    319]     0   319     1001       14    32768        0             0 agetty
[44951.087415] [    327]     0   327     2624      177    49152        0         -1000 sshd
[44951.087540] [   1790]     0  1790     2152      107    40960        0             0 login
[44951.087661] [   1797]  1000  1797     3053      242    49152        0           100 systemd
[44951.087839] [   1799]  1000  1799     7818      490    57344        0           100 (sd-pam)
[44951.087982] [   1814]  1000  1814     1970      242    40960        0             0 bash
[44951.088089] [  15486]  1000 15486   530244   495348  4116480        0             0 bitcoind-master
[44951.088208] [  15657]     0 15657     4699      157    57344        0          -250 systemd-journal
[44951.088338] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/session-4.scope,task=bitcoind-master,pid=15486,uid=1000
[44951.089813] Out of memory: Killed process 15486 (bitcoind-master) total-vm:2120976kB, anon-rss:1981392kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:4020kB oom_score_adj:0
Killed

I coudn't reproduce the issue using this PR branch.

Verifying the issue is not happening in x86 64bits even forcing the modified test as mentioned above by other reviewers.

fanquake added a commit that referenced this pull request Nov 22, 2023
2f86d30 doc: update release notes for v26.0rc3 (fanquake)
3b6c7f2 doc: update manual pages for v26.0rc3 (fanquake)
3db4d1c build: bump version to v26.0rc3 (fanquake)
6045f38 build: Fix regression in "ARMv8 CRC32 intrinsics" test (Hennadii Stepanov)
5eaa179 ci: Avoid toolset ambiguity that MSVC can't handle (Hennadii Stepanov)
55af112 p2p: do not make automatic outbound connections to addnode peers (Jon Atack)
5e0bcc1 ci: Switch from `apt` to `apt-get` (Hennadii Stepanov)
437a531 ci: Update apt cache (Hennadii Stepanov)
1488648 pool: change memusage_test to use int64_t, add allocation check (Martin Leitner-Ankerl)
bcc183c pool: make sure PoolAllocator uses the correct alignment (Martin Leitner-Ankerl)
7dda499 doc: regenerate example bitcoin.conf (fanquake)
5845331 doc: rewrite explanation for -par= (fanquake)

Pull request description:

  Currently backports:
  * #28858
  * #28895 (partial)
  * #28913
  * #28905
  * #28919
  * #28925

  Also includes changes for rc3, and reintegrating the release-notes.

ACKs for top commit:
  hebasto:
    re-ACK 2f86d30, only #28919 backported since my [recent](#28872 (review)) review.
  TheCharlatan:
    ACK 2f86d30

Tree-SHA512: 43c91b344d37f582081ac184ac59cf76c741317b2b69a24fcd4287eefa8333e20c545e150798f4057d6f4ac8e70ed9cba1c8dd9777b11c1cf8992cce09108727
Fabcien pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request May 20, 2024
…cator

Summary:
Extracts the resource from a PoolAllocator and uses it for
calculation of the node's memory usage.

This is a partial backport of [[bitcoin/bitcoin#25325 | core#25325]]
bitcoin/bitcoin@e19943f

with an ARM fix and additional test to make sure the pool is actually used by the nodes from [[bitcoin/bitcoin#28913 | core#28913]]

Depends on D16175

Test Plan: `ninja check`

Reviewers: #bitcoin_abc, Fabien

Reviewed By: #bitcoin_abc, Fabien

Differential Revision: https://reviews.bitcoinabc.org/D16176
Fabcien pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request May 20, 2024
Summary:
> In my benchmarks, using this pool allocator for CCoinsMap gives about 20% faster `-reindex-chainstate` with -dbcache=5000 with practically the same memory  usage. The change in max RSS changed was 0.3%.
>
> The `validation_flush_tests` tests need to be updated because memory allocation is now done in large pools instead of one node at a time, so the limits need to be updated accordingly.

This is a partial backport of [[bitcoin/bitcoin#25325 | core#25325]]
bitcoin/bitcoin@9f947fc

with a fix from [[bitcoin/bitcoin#28913 | core#28913]]

Depends on D16177

Benchmark on a 12 cores AMD CPU, with the data on a spinning harddrive, using the default dbcache (1024 MB):
```
time src/bitcoind -reindex-chainstate -stopatheight=400000 -assumevalid=000000000000000004ec466ce4732fe6f1ed1cddc2ed4b328fff5224276e3f6f
```

Before:
```
real 96m2s
user 27m27s
sys 2min58s
```

After:
```
real 74m22s
user 23m39s
sys 2min23s
```

Test Plan:
`ninja all check-all`

Reviewers: #bitcoin_abc, Fabien

Reviewed By: #bitcoin_abc, Fabien

Subscribers: Fabien

Differential Revision: https://reviews.bitcoinabc.org/D16178
PastaPastaPasta pushed a commit to PastaPastaPasta/dash that referenced this pull request Oct 24, 2024
…alignment

d5b4c0b pool: change memusage_test to use int64_t, add allocation check (Martin Leitner-Ankerl)
ce881bf pool: make sure PoolAllocator uses the correct alignment (Martin Leitner-Ankerl)

Pull request description:

  The class `CTxOut` has a member `CAmount` which is an int64_t, and on ARM 32bit int64_t are 8 byte aligned, which is larger than the pointer alignment of 4 bytes.

  So for `CCoinsMap` to be able to use the pool, we need to use the alignment of the member instead of just `alignof(void*)`.

  This fixes bitcoin#28906 (first noted in bitcoin#28718 (comment)) and bitcoin#28440.

ACKs for top commit:
  pinheadmz:
    ACK d5b4c0b
  hebasto:
    re-ACK d5b4c0b, the only change since my recent [review](bitcoin#28913 (review)) is an updated test.
  theStack:
    Tested ACK d5b4c0b

Tree-SHA512: 4446793fad6d56f0fe22e09ac9ade051e86de11ac039cd61c0f6b7f79874242878a6a46a2c76ac3b8f1d53464872620d39139f54b1471daccad26d6bb1ae8ca1
PastaPastaPasta pushed a commit to PastaPastaPasta/dash that referenced this pull request Oct 24, 2024
…alignment

d5b4c0b pool: change memusage_test to use int64_t, add allocation check (Martin Leitner-Ankerl)
ce881bf pool: make sure PoolAllocator uses the correct alignment (Martin Leitner-Ankerl)

Pull request description:

  The class `CTxOut` has a member `CAmount` which is an int64_t, and on ARM 32bit int64_t are 8 byte aligned, which is larger than the pointer alignment of 4 bytes.

  So for `CCoinsMap` to be able to use the pool, we need to use the alignment of the member instead of just `alignof(void*)`.

  This fixes bitcoin#28906 (first noted in bitcoin#28718 (comment)) and bitcoin#28440.

ACKs for top commit:
  pinheadmz:
    ACK d5b4c0b
  hebasto:
    re-ACK d5b4c0b, the only change since my recent [review](bitcoin#28913 (review)) is an updated test.
  theStack:
    Tested ACK d5b4c0b

Tree-SHA512: 4446793fad6d56f0fe22e09ac9ade051e86de11ac039cd61c0f6b7f79874242878a6a46a2c76ac3b8f1d53464872620d39139f54b1471daccad26d6bb1ae8ca1
PastaPastaPasta pushed a commit to PastaPastaPasta/dash that referenced this pull request Oct 24, 2024
…alignment

d5b4c0b pool: change memusage_test to use int64_t, add allocation check (Martin Leitner-Ankerl)
ce881bf pool: make sure PoolAllocator uses the correct alignment (Martin Leitner-Ankerl)

Pull request description:

  The class `CTxOut` has a member `CAmount` which is an int64_t, and on ARM 32bit int64_t are 8 byte aligned, which is larger than the pointer alignment of 4 bytes.

  So for `CCoinsMap` to be able to use the pool, we need to use the alignment of the member instead of just `alignof(void*)`.

  This fixes bitcoin#28906 (first noted in bitcoin#28718 (comment)) and bitcoin#28440.

ACKs for top commit:
  pinheadmz:
    ACK d5b4c0b
  hebasto:
    re-ACK d5b4c0b, the only change since my recent [review](bitcoin#28913 (review)) is an updated test.
  theStack:
    Tested ACK d5b4c0b

Tree-SHA512: 4446793fad6d56f0fe22e09ac9ade051e86de11ac039cd61c0f6b7f79874242878a6a46a2c76ac3b8f1d53464872620d39139f54b1471daccad26d6bb1ae8ca1
PastaPastaPasta added a commit to dashpay/dash that referenced this pull request Oct 24, 2024
8bf1d06 Merge bitcoin#29308: doc: update `BroadcastTransaction` comment (glozow)
2a77808 Merge bitcoin-core/gui#789: Avoid non-self-contained Windows header (Hennadii Stepanov)
da371b8 Merge bitcoin#28870: depends: Include `config.guess` and `config.sub` into `meta_depends` (fanquake)
2e41562 Merge bitcoin#29219: fuzz: Improve fuzzing stability for ellswift_roundtrip harness (fanquake)
b091329 Merge bitcoin#29211: fuzz: fix `connman` initialization (Ava Chow)
df42d41 Merge bitcoin#29200: net: create I2P sessions using both ECIES-X25519 and ElGamal encryption (fanquake)
4cdd1a8 Merge bitcoin#29172: fuzz: set `nMaxOutboundLimit` in connman target (fanquake)
97012ea Merge bitcoin#28962: doc: Rework guix docs after 1.4 release (fanquake)
c70ff5d Merge bitcoin#28844: contrib: drop GCC MAX_VERSION to 4.3.0 in symbol-check (fanquake)
e6f19e7 Merge bitcoin#29068: test: Actually fail when a python unit test fails (fanquake)
75e0334 Merge bitcoin#28989: test: Fix test by checking the actual exception instance (Andrew Chow)
8cd85d3 Merge bitcoin#28852: script, assumeutxo: Enhance validations in utxo_snapshot.sh (Ryan Ofsky)
fd2e88d Merge bitcoin#26077: guix: switch from `guix environment` to `guix shell` (fanquake)
02741a7 Merge bitcoin#28913: coins: make sure PoolAllocator uses the correct alignment (fanquake)
dfd53da Merge bitcoin#28902: doc: Simplify guix install doc, after 1.4 release (fanquake)

Pull request description:

  ## Issue being fixed or feature implemented
  Batch of trivial backports

  ## What was done?
  See commits

  ## How Has This Been Tested?
  built locally; large combined merge passed tests locally

  ## Breaking Changes
  Should be none

  ## Checklist:
    _Go over all the following points, and put an `x` in all the boxes that apply._
  - [ ] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  UdjinM6:
    utACK 8bf1d06
  knst:
    utACK 8bf1d06

Tree-SHA512: 506273e5a188f9ca74edf656e3cd338992192e6e97f68c89fc43e34be20fb7f211b48e4dfa8693727839a7920da8284509413c722f55774a428939c296dad517
@bitcoin bitcoin locked and limited conversation to collaborators Nov 21, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RAM usage regression in 26.x and master on ARM 32-bit
10 participants