Skip to content

Conversation

fjahr
Copy link
Contributor

@fjahr fjahr commented Aug 24, 2020

Requires #19521 (this PR only adds the last commit).

The rpc dumpcoinstats dumps the content of the coinstats index into a CSV file for auditing purposes.

@DrahtBot
Copy link
Contributor

DrahtBot commented Aug 24, 2020

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type Reviewers
Concept ACK laanwj, promag
Approach ACK Zero-1729
Stale ACK PierreRochard, w0xlt, Sjors

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #27607 (init: verify blocks data existence only once for all the indexers by furszy)
  • #20892 (tests: Run both descriptor and legacy tests within a single test invocation by achow101)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@fjahr
Copy link
Contributor Author

fjahr commented Aug 25, 2020

Taking this out of draft status since I am interested in conceptual feedback on something like this but of course, #19521 still has a long way to go before this can be seriously considered.

@PierreRochard
Copy link
Contributor

Tested ACK 21c42c2

@laanwj
Copy link
Member

laanwj commented Aug 27, 2020

Concept ACK, I wish we had gotten streaming to work though so that the file can be downloaded (see my experiment in #7759). RPCs that creates a server-local file, although there are already some others, are not that great from a security and usability point of view.

@fjahr
Copy link
Contributor Author

fjahr commented Sep 22, 2020

Rebased on top of the latest changes in #19521

@fjahr fjahr force-pushed the dumpcoinstats branch 2 times, most recently from c8b5d0a to 42763a2 Compare June 11, 2022 18:50
Copy link
Contributor

@aureleoules aureleoules left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the index has not fully been synced yet, the error thrown would be:

Failed to get coins stats at height X. You may have to remove an incomplete dump file before running this command again.

It's not very clear that the index has not finished syncing.
Also, the csv file would be created before this error is thrown, so the file would have to be removed manually.

I would suggest checking if the index is synced before dumping and create the file only if the index is ready+enabled, with something like this:

Diff
diff --git a/src/rpc/blockchain.cpp b/src/rpc/blockchain.cpp
index 74877d7da..7bb6e2927 100644
--- a/src/rpc/blockchain.cpp
+++ b/src/rpc/blockchain.cpp
@@ -1039,16 +1039,21 @@ static RPCHelpMan dumpcoinstats()
                 throw JSONRPCError(RPC_INVALID_PARAMETER, filepath.u8string() + " already exists. If you are sure this is what you want, move it out of the way first");
             }
 
+            if (!g_coin_stats_index) {
+                throw JSONRPCError(RPC_MISC_ERROR, "Coinstats index is not enabled");
+            }
+
+            const auto summary{g_coin_stats_index->GetSummary()};
+            if (!summary.synced) {
+                throw JSONRPCError(RPC_MISC_ERROR, strprintf("coinstatsindex is not synced. Current best block height: %d", summary.best_block_height));
+            }
+
             std::ofstream file;
             file.open(filepath);
             if (!file.is_open()) {
                 throw JSONRPCError(RPC_INVALID_PARAMETER, "Cannot open dump file");
             }
 
-            if (!g_coin_stats_index) {
-                throw JSONRPCError(RPC_MISC_ERROR, "Coinstats index is not enabled");
-            }
-
             const bool cumulative{request.params[1].isNull() ? false : request.params[1].get_bool()};
 
             file << strprintf("height,"

@fjahr
Copy link
Contributor Author

fjahr commented Dec 22, 2022

Addressed comments from @aureleoules , thanks for the review!

Further note: I can't change the status back it seems, but please treat this as a draft for now. This PR will need to be updated and rebased soon when I have resolved the overflow issue noted in #26362.

EDIT: Rebased as well since there was a silent merge conflict.

@fanquake fanquake marked this pull request as draft December 22, 2022 15:45
@fjahr
Copy link
Contributor Author

fjahr commented Dec 22, 2022

Thanks @fanquake , lightning speed :)

fjahr added 3 commits May 3, 2023 14:04
This RPC dumps the full content of the coinstats index together with some other statistics into a CSV file.
@DrahtBot
Copy link
Contributor

🐙 This pull request conflicts with the target branch and needs rebase.

@DrahtBot
Copy link
Contributor

There hasn't been much activity lately and the patch still needs rebase. What is the status here?

  • Is it still relevant? ➡️ Please solve the conflicts to make it ready for review and to ensure the CI passes.
  • Is it no longer relevant? ➡️ Please close.
  • Did the author lose interest or time to work on this? ➡️ Please close it and mark it 'Up for grabs' with the label, so that it can be picked up in the future.

@fjahr
Copy link
Contributor Author

fjahr commented Aug 15, 2023

There doesn't seem to be much interest in this feature currently. If someone needs it, feel free to ping me here and I will reorg/rebase for you.

@fjahr fjahr closed this Aug 15, 2023
@bitcoin bitcoin locked and limited conversation to collaborators Aug 14, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.