Skip to content

Conversation

jamesob
Copy link
Contributor

@jamesob jamesob commented Jan 31, 2023

Make use of #17487.

Benchmark results available here:
#17487 (comment)

@DrahtBot
Copy link
Contributor

DrahtBot commented Jan 31, 2023

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK Sjors

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #26045 (rpc: Optimize serialization disk space of dumptxoutset by aureleoules)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

Copy link
Member

@Sjors Sjors left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK d6f54da

Based on testing #17487 I think it's safe. I'd like to take this for a spin on a rebase of #15606 (the bits that enable loadtxoutset), to see if I can reproduce the performance improvement. But imo that doesn't have to hold back this PR.

@jamesob jamesob force-pushed the 2023-01-au-flush-optimize branch from d6f54da to 2834617 Compare February 6, 2023 14:23
@jamesob
Copy link
Contributor Author

jamesob commented Feb 6, 2023

Thanks for the look @Sjors. I also realized that there were two outlier calls where we weren't asserting that Flush()/Sync() has succeeded, so I've fixed those.

@jamesob jamesob requested a review from Sjors February 6, 2023 14:34
We've historically done this check in most places, so make sure we're
doing it uniformly.
Copy link
Contributor Author

@jamesob jamesob left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedback @Sjors.

@jamesob jamesob force-pushed the 2023-01-au-flush-optimize branch from 2834617 to 937016b Compare February 22, 2023 17:05
@Sjors
Copy link
Member

Sjors commented Feb 23, 2023

re-utACK 937016b

@jamesob jamesob mentioned this pull request Feb 24, 2023
18 tasks
@Sjors
Copy link
Member

Sjors commented Mar 10, 2023

From my testing of #15606 - without this PR - I get the impression that flushing isn't working, specifically that RAM is not freed up. It could just be confusion on my end from rebase hell, but I'd like to make sure we're not obscuring a bug by merging this. (I initially assumed the main assumeutxo PR already contained this PR because RAM didn't go down)

@jamesob
Copy link
Contributor Author

jamesob commented Apr 19, 2023

@Sjors how did you detect that RAM wasn't being freed? Did you verify that the flush did not happen based on an absence of logs?

Measures of memory can be deceptive; I remember bitcoinperf spuriously detecting an increase in memory usage because once claimed, the process' resident set size (RSS) doesn't necessarily go down upon free.

@Sjors
Copy link
Member

Sjors commented Apr 27, 2023

@jamesob using the system monitor desktop app in Ubuntu, and I believe also based on the cache size log messages. I'll double check this next time I run it.

@jamesob
Copy link
Contributor Author

jamesob commented May 6, 2023

Actually this is broken at the moment, since although we don't flush after snapshot load, we do flush (and not sync) in the MaybeRebalanceCaches() -> FlushStateToDisk() call right after activating the chainstate snapshot. Will close this and think about how to rework it.

@jamesob jamesob closed this May 6, 2023
@bitcoin bitcoin locked and limited conversation to collaborators May 5, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants