-
Notifications
You must be signed in to change notification settings - Fork 97
indexer: Reduce required on-chain logs #6739
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
📝 WalkthroughWalkthroughThis pull request applies extensive modifications across multiple modules. It removes deprecated type aliases, streamlines import paths, and updates method signatures for blockchain interactions. New asynchronous methods for retrieving token allowances and processing logs have been introduced, and log filtering has been refactored to use a new filter collection structure. Additionally, feature flags and dependencies have been updated to support multiple asynchronous runtimes. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant HoprChain
participant RpcOperations
participant Blockchain
Client->>HoprChain: get_safe_hopr_allowance()
HoprChain->>RpcOperations: get_allowance(safe_address, contract)
RpcOperations->>Blockchain: Query allowance(contract, safe_address)
Blockchain-->>RpcOperations: Return Balance
RpcOperations-->>HoprChain: Return Balance
HoprChain-->>Client: Return Balance
sequenceDiagram
participant Block
participant Indexer
participant EventHandler
participant RpcOperations
participant Blockchain
Block->>Indexer: New block with logs
Indexer->>EventHandler: collect_log_event(log)
EventHandler->>RpcOperations: get_balance/get_allowance query (if token event)
RpcOperations->>Blockchain: Fetch token data
Blockchain-->>RpcOperations: Return data
RpcOperations-->>EventHandler: Return data
EventHandler-->>Indexer: Process event & update state
Possibly related PRs
Suggested labels
Suggested reviewers
✨ Finishing Touches
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
🧹 Nitpick comments (4)
chain/rpc/src/lib.rs (2)
167-176
: Introducing FilterSet for refined log filtering.
Consider adding helper methods (e.g.,is_empty()
) or cohesive merges of filters if needed.
345-347
: Added get_allowance to HoprRpcOperations.
Ensure test coverage for typical and edge cases (e.g., zero allowance).Would you like assistance generating tests for this method?
chain/indexer/src/handlers.rs (1)
61-62
: Doc comment on struct fieldDeclaring doc comments (
///
) on a struct field is unconventional but still valid Rust. Ensure the doc clearly explains the field for readers and generated documentation.chain/rpc/src/indexer.rs (1)
80-125
: Refined logic for log subrange fetching and sortingThis block:
- Iterates over subrange filters
- Fetches each filter’s logs in parallel
- Flattens all logs into a single vector
- Sorts logs by block number
The approach is sensible for chunked retrieval, but watch out for potential performance issues with large vectors. Also confirm block reordering is correct if logs come in partial sets.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (12)
chain/api/src/lib.rs
(4 hunks)chain/indexer/Cargo.toml
(1 hunks)chain/indexer/src/block.rs
(21 hunks)chain/indexer/src/constants.rs
(0 hunks)chain/indexer/src/handlers.rs
(16 hunks)chain/indexer/src/traits.rs
(4 hunks)chain/rpc/src/indexer.rs
(6 hunks)chain/rpc/src/lib.rs
(4 hunks)chain/rpc/src/rpc.rs
(2 hunks)chain/rpc/src/types.rs
(1 hunks)chain/types/src/lib.rs
(2 hunks)hopli/src/environment_config.rs
(1 hunks)
💤 Files with no reviewable changes (1)
- chain/indexer/src/constants.rs
✅ Files skipped from review due to trivial changes (1)
- chain/types/src/lib.rs
🧰 Additional context used
🧠 Learnings (1)
chain/rpc/src/indexer.rs (1)
Learnt from: NumberFour8
PR: hoprnet/hoprnet#6226
File: chain/rpc/src/indexer.rs:35-60
Timestamp: 2024-11-12T10:55:50.205Z
Learning: The `split_range` function in `chain/rpc/src/indexer.rs` requires cloning the `LogFilter` because each iteration potentially modifies the filter for a specific block range.
⏰ Context from checks skipped due to timeout of 90000ms (8)
- GitHub Check: hoprd / docker
- GitHub Check: hopli / docker
- GitHub Check: Docs / Rust docs
- GitHub Check: tests-smoke-websocket
- GitHub Check: tests-smart-contracts
- GitHub Check: tests-unit
- GitHub Check: tests-unit-nightly
- GitHub Check: Linter
🔇 Additional comments (59)
chain/indexer/Cargo.toml (2)
16-18
: New dependencies added correctlyThe addition of the
hopr-chain-rpc/runtime-async-std
feature dependency is properly aligned with the other runtime features.
21-23
: New dependencies added correctlyThe addition of the
hopr-chain-rpc/runtime-tokio
feature dependency is properly aligned with the other runtime features.chain/rpc/src/types.rs (3)
7-11
: Good conditional compilationThe code correctly handles the case where both runtime features could be enabled during testing by using
all(feature = "runtime-tokio", not(feature = "runtime-async-std"))
for the tokio feature.
12-15
: Type alias improves code organizationThe
JsonRpcClient
type alias helps centralize the definition which can be imported by other modules, reducing duplication and improving maintainability.
1-6
:✅ Verification successful
Address the TODO comment in a future PR
The TODO comment indicates that
DefaultHttpPostRequestor
should be an internal type, and thathopr_lib::chain
must be moved to this package. Consider creating a follow-up task to address this technical debt.
🏁 Script executed:
#!/bin/bash # Check for references to hopr_lib::chain in the codebase rg "hopr_lib::chain" --type rustLength of output: 141
Action: Create a follow-up task for migrating the HTTP request implementation.
The current code in
chain/rpc/src/types.rs
definesDefaultHttpPostRequestor
and includes a TODO comment indicating that this type should be made internal, with thehopr_lib::chain
functionality moved into this package. A search confirms that the only reference tohopr_lib::chain
exists in this comment. No other issues were detected, so it’s appropriate to document the technical debt and address it in a separate future PR.
- Location:
chain/rpc/src/types.rs
(lines 1-6)- Next Step: Create a follow-up issue to track the migration of
hopr_lib::chain
into the package.hopli/src/environment_config.rs (2)
29-32
: Import reorganization improves module structureThe import statements have been reorganized to better structure the dependencies, separating the imports from
hopr_chain_rpc
and improving readability.
37-37
: Type alias simplificationReplacing the verbose type alias definition with a direct
pub use
statement from the newhopr_chain_rpc::types
module simplifies the code and ensures type consistency across the codebase.chain/rpc/src/rpc.rs (2)
113-114
: Mockable struct enhances testabilityAdding the
#[cfg_attr(test, mockall::automock)]
attribute enables automatic mock generation for theRpcOperations
struct during testing, which improves testability.
191-206
: Well-implemented allowance retrieval methodThe new
get_allowance
method is well-implemented with proper error handling and follows the same pattern as the existingget_balance
method. This addition enhances the RPC operations interface by allowing token allowance queries.This method aligns with the PR objective to support the indexer in filtering out most wxHOPR token events, allowing it to only process events that are relevant to a specific node.
chain/indexer/src/traits.rs (9)
3-3
: Imported Arc usage is appropriate for shared data handling.
No issues found with the new import; it cleanly supports returning references in the trait methods.
6-6
: New ContractAddresses import aligns well with usage.
This import is consistent with the updated trait returning Arc.
15-16
: New method returning Arc is clear and consistent.
This provides a thread-safe reference to contract addresses, which may improve concurrent access patterns.
19-20
: Introducing a safe_address method completes contract metadata retrieval.
This addition complements existing contract address getters nicely.
21-21
: Asynchronous log-event collection method is well-scoped.
Returning Option is flexible for logs that may not produce an event.
35-36
: Updated mock documentation snippet matches the new collect_log_event method.
This helps clarify usage in testing environments.
47-47
: Mock method matches the real contract_addresses_map signature.
Maintains consistency with the trait's new method signature.
49-49
: Mock method for safe_address is consistent with the trait.
No concerns here; it aligns with the real method.
50-50
: Mock method for collect_log_event aligns with async trait usage.
Ensures the mock can fully mimic the real method’s expected behavior.chain/api/src/lib.rs (4)
24-24
: Reintroducing DefaultHttpPostRequestor and JsonRpcClient usage seems correct.
These imports appear to be used consistently in RPC client initialization.
109-109
: Refined ActionQueueType to use RpcEthereumClient<RpcOperations>.
This centralizes the RPC usage, ensuring a single consistent client type.
245-245
: Passing cloned rpc_operations into ContractEventHandlers respects ownership rules.
This approach avoids lifecycle or borrow issues when sharing the same RPC object.
322-328
: New get_safe_hopr_allowance method is neatly implemented.
It delegates directly to get_allowance on the channels contract, returning a Balance.chain/rpc/src/lib.rs (4)
36-36
: Re-exporting types module.
No conflicts or issues; consolidates RPC-related type definitions appropriately.
412-413
: Indexer also provides get_allowance, mirroring HoprRpcOperations.
Consistency across the traits is beneficial for maintainability.
416-416
: Providing get_balance in HoprIndexerRpcOperations broadens indexer capabilities.
This parallels the existing HoprRpcOperations method.
425-427
: try_stream_logs now takes FilterSet and is_synced flag.
The updated signature accommodates more granular filtering logic.chain/indexer/src/handlers.rs (17)
19-21
: Imports fromhopr_chain_rpc
addedThese imports provide the ability to perform RPC operations and work with
JsonRpcClient
/HoprRpcOperations
. The addition appears consistent with the revised design where handlers can directly call the RPC methods.
65-65
: Debug implementation noteImplementing a custom
Debug
is fine as it selectively displays fields. No immediate issues.
79-85
: Constructor extended withrpc_operations
parameterThis constructor now requires an
RpcOperations<JsonRpcClient>
instance to enable direct chain RPC calls (e.g., fetching balance/allowance). The signature and usage appear consistent with the rest of the file.Also applies to: 91-91
187-188
: New match arm for channel eventsThe
match event { HoprChannelsEvents::ChannelBalanceDecreasedFilter(...) => ... }
block is unchanged except for the leading line. No specific concern here.
194-198
: Added trace logging for decreased balance eventsThis
trace!
call improves debugging by capturing channel ID and channel existence. The usage of trace-level logging is appropriate for detailed diagnostics.
221-226
: Trace logging for increased balance eventsSimilarly, these trace statements help confirm channel ID and existence. This log is valuable for debugging deposit events.
249-253
: Trace logging for closed channel eventsAgain, a consistent trace log describing the channel ID and whether the channel is in the database. This aids debugging channel termination scenarios.
517-520
: Error logging for irrelevant transfer eventsThis path logs an error if the token transfer does not involve the known safe address. This helps spot potential filter misconfiguration early.
522-534
: Updating safe balance on incoming transfersWhen tokens arrive at the safe, the code fetches the new on-chain balance. The match clause properly handles any RPC failure. This is a good approach for reliability.
555-557
: Storing safe allowance on successful fetchOn success, the code updates the DB with the new allowance. Looks correct.
559-560
: Error handling for allowance fetch failuresThe design logs the error without retry. Confirm whether a retry or fallback is needed.
563-566
: Misconfiguration error for approvals not involving the safeTreating approvals that do not match the safe/channels combination as an error is consistent with your filtering logic.
835-835
: Additional trait bounds forDb
Expanding the trait bounds (including
Debug
) suggests you might log theDb
type or need deeper introspection. No immediate concerns.
850-852
: New helper method:contract_addresses_map
This method returns a clone of the internal
Arc<ContractAddresses>
. It improves readability but watch for potential overhead if called frequently in tight loops.
854-856
: New helper method:safe_address
Returning the safe address directly is straightforward and likely improves code clarity.
879-904
: Enhanced log processing and event creationThis chunk inside
collect_log_event
handles the newly stored states (tx_hash
,log_index
, etc.) and constructs aSignificantChainEvent
. The approach to capturing more metadata (e.g.,block_id
,log_id
) in debug logs is beneficial for troubleshooting. Ensure all usage sites align with the new data structures.
913-917
: Additional test importsBringing in
anyhow
,Context
, and refinements fromethers
and the standard library is consistent with test expansions.chain/rpc/src/indexer.rs (8)
18-19
: Add import forhopr_primitive_types::prelude::*
Bringing these types is coherent if references to
Address
,Balance
, etc. are used extensively.
22-22
: Importing additional core traits/typesImporting
BlockWithLogs
,FilterSet
,HoprIndexerRpcOperations
,Log
from the crate clarifies usage within this file.
66-71
: Function signature adjustments instream_logs
The method now accepts multiple
filters: Vec<ethers::types::Filter>
and a block range. Ensure consistent usage of these arguments across the code.
134-137
:get_allowance
default forwardingThis method delegates directly to the same-named method in
RpcOperations
. Straight pass-through with no additional logic is fine.
138-141
:get_balance
default forwardingSimilarly, forwarding to the underlying
RpcOperations
is correct if no local transformations are needed.
145-146
: Updatedtry_stream_logs
parametersAdding
filters: FilterSet
andis_synced: bool
clarifies the difference between ignoring token events when unsynced vs. including them once fully synced. This addresses log volume concerns and is logically sound.
153-158
: Filtering out token logs if not syncedPreventing the indexing of token-related events until the node is caught up significantly reduces log overhead. This approach looks aligned with the PR’s objective.
194-194
: Storing logs fetch inretrieved_logs
Using
self.stream_logs(log_filters.clone(), from_block, latest_block)
is a straightforward composition with the new subrange logic.chain/indexer/src/block.rs (8)
1-2
: Newly importedethers
referencesIntroducing
EthEvent
,Filter
,H256
suggests more direct interaction with Ethers-based events and filter building.
8-8
: Import token filtersPulling in
TransferFilter
andApprovalFilter
fromhopr_token
aligns with the new token log filtering approach.
11-11
: Importinghopr_chain_rpc
items
BlockWithLogs
,FilterSet
, andHoprIndexerRpcOperations
integration signals that the indexer is now using these more structured constructs (rather than a singleLogFilter
).
108-108
: Generating log filters before streaming logsStoring
let log_filters = Self::generate_log_filters(&logs_handler);
clarifies that the indexing logic chooses which addresses/topics to track. Good to see a separate function.
329-335
: Comment on grouping big filters logicThese comment lines outline a shift from building a single
LogFilter
to constructing a more sophisticated set. Reading them is helpful for explaining the rationale.
336-386
:generate_log_filters
functionThis method:
- Collects addresses from the
logs_handler
, excluding the token contract- Builds a base filter for those addresses
- Builds specialized filters for safe-specific transfers and approvals
It addresses the PR’s goals of filtering out most token events except those explicitly involving the safe. Well-structured and clear.
463-505
: Revised block-level log processingHere, logs are:
- Converted to a
stream::iter(...)
- Collected into events via
logs_handler.collect_log_event(...)
- Marked processed only upon successful event extraction
This ensures consistent handling of partial failures. The panic on error is strong but aligns with your design principle of preventing data corruption.
541-550
: Extended parameters incalculate_sync_process
Including
db: Db
,safe_address: Option<Address>
, andchannels_address: Option<Address>
allows the indexer to refresh balances or allowances post-sync. This matches the PR’s objective to update the node’s safe data once fully synced.Also applies to: 553-553
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
🧹 Nitpick comments (8)
chain/rpc/src/types.rs (1)
1-10
: Well-structured conditional HTTP requestor implementationThe conditional compilation ensures that the appropriate HTTP requestor is used based on the enabled runtime, which is a clean approach to handle multiple runtime support.
Consider addressing the TODO comment in a future PR. Moving
hopr_lib::chain
to this package would help with encapsulation.chain/indexer/src/block.rs (3)
329-386
: Good approach to centralize filter creation.
Defining one function to build theFilterSet
improves readability. Consider adding safer defaults or warnings if no addresses/topics are found to avoid an empty filter situation.
464-505
: Panicking on log processing failure might halt the entire node.
Marking logs as processed only ifcollect_log_event
succeeds prevents data inconsistency, but panics can disrupt indexing in production. Checking whether to use a recovery or retry mechanism instead of panicking could improve resilience.
622-624
: Graceful send-failure handling.
It’s acceptable to log the error fromtry_send
and continue. If lost synchronization signals are detrimental, consider adding a fallback mechanism.chain/rpc/src/indexer.rs (2)
40-45
: Flexible chunked splitting for multiple filters.
Parameterizing the split by aVec<Filter>
supports concurrent filter application. Ensure potential large ranges are handled well in production.
53-57
: Applying subrange blocks in each filter.
Chaining.from_block()
and.to_block()
systematically is a neat approach. Keep an eye on potential performance overhead for large filter sets.chain/indexer/src/handlers.rs (2)
65-65
: Consider includingrpc_operations
in Debug output.
Theimpl Debug
currently omits therpc_operations
field, which might be helpful for diagnostics or debugging.
517-534
: Safe balance update logic.
The approach of verifying the transfer is relevant to the safe and then querying on-chain is correct. However, consider lowering the log level fromerror!
towarn!
when a transfer is irrelevant to the safe, unless it's always considered a genuine misconfiguration.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
Cargo.lock
is excluded by!**/*.lock
📒 Files selected for processing (13)
chain/api/src/lib.rs
(4 hunks)chain/indexer/Cargo.toml
(1 hunks)chain/indexer/src/block.rs
(21 hunks)chain/indexer/src/constants.rs
(0 hunks)chain/indexer/src/handlers.rs
(16 hunks)chain/indexer/src/traits.rs
(4 hunks)chain/rpc/Cargo.toml
(1 hunks)chain/rpc/src/indexer.rs
(6 hunks)chain/rpc/src/lib.rs
(4 hunks)chain/rpc/src/rpc.rs
(2 hunks)chain/rpc/src/types.rs
(1 hunks)chain/types/src/lib.rs
(2 hunks)hopli/src/environment_config.rs
(1 hunks)
💤 Files with no reviewable changes (1)
- chain/indexer/src/constants.rs
✅ Files skipped from review due to trivial changes (1)
- chain/types/src/lib.rs
⏰ Context from checks skipped due to timeout of 90000ms (8)
- GitHub Check: tests-smoke-websocket
- GitHub Check: tests-smart-contracts
- GitHub Check: tests-unit-nightly
- GitHub Check: tests-unit
- GitHub Check: hoprd / docker
- GitHub Check: hopli / docker
- GitHub Check: Docs / Rust docs
- GitHub Check: Linter
🔇 Additional comments (48)
chain/rpc/Cargo.toml (1)
44-44
: Adding new dependency for concurrent stream processingThe addition of
rust-stream-ext-concurrent
is well aligned with the PR's objective to improve indexer performance by managing concurrent stream operations more efficiently.chain/indexer/Cargo.toml (1)
17-17
: Good propagation of RPC runtime featuresAdding the RPC runtime features for both async-std and tokio ensures that the indexer can utilize the RPC functionality with either async runtime, providing better flexibility and coherence with the rest of the system.
Also applies to: 22-22
chain/rpc/src/types.rs (1)
12-14
: Clear JsonRpcClient type definitionThis type alias standardizes how JSON RPC calls are made, which is important for the PR's goal of improving error handling in the indexer.
chain/rpc/src/rpc.rs (2)
113-113
: Good testing support with mockallAdding the mockall automock attribute enables proper mocking in tests, which is essential for reliable unit testing of the RPC operations.
191-206
:❓ Verification inconclusive
Well-implemented token allowance retrieval method
The new
get_allowance
method follows the established pattern for RPC operations and includes proper error handling. This method will help with focusing on relevant token events as mentioned in the PR objectives.Make sure all places that need to check token allowances are updated to use this new method. This is particularly important for the indexer logic that filters logs.
🏁 Script executed:
#!/bin/bash # Looking for potential callers of the token's allowance function directly rg -A 2 -B 2 "\.allowance\(" --exclude "rpc.rs"Length of output: 459
Token allowance retrieval confirmed.
The newget_allowance
method inchain/rpc/src/rpc.rs
is correctly implemented with appropriate error handling and follows the established RPC pattern. However, please verify that all parts of the code—particularly the indexer logic—no longer call the token’sallowance
function directly but instead use this new method.
- Confirm that there are no direct invocations of
.allowance(
in the codebase (excludingrpc.rs
). For example, you can run:rg -A2 -B2 '\.allowance\(' -g '!rpc.rs'- Manually inspect any indexer-related logic to ensure it now references
get_allowance
.hopli/src/environment_config.rs (2)
30-31
: Imports appear correctly updated.These new imports from
hopr_chain_rpc::types
replace the previous nested imports. This streamlines the code and aligns with the recentchain/rpc
refactors.
37-37
: Updated type alias usage.Switching to
pub use hopr_chain_rpc::types::JsonRpcClient;
instead of defining a local type alias helps maintain consistency across the codebase and reduces redundancy.chain/indexer/src/traits.rs (5)
3-3
: Arc import for concurrency.Introducing
std::sync::Arc
is necessary to return thread-safe references in subsequent trait methods. This is a sound approach for sharing read-only data.
5-6
: New imports for event and addresses.Adding
SignificantChainEvent
andContractAddresses
references indicates a shift toward more granular log event handling and storing contract data. This looks consistent with the rest of the trait changes.
15-21
: Additional trait methods to handle log events more granularly.
contract_addresses_map
returns anArc<ContractAddresses>
for shared data without copying.safe_address
provides direct retrieval of the safe’s address.collect_log_event
processes a single log, returning an optionalSignificantChainEvent
, aligning with the new approach of handling logs individually.
35-36
: Mock usage for single event processing.This mock setup for
expect_collect_log_event
is aligned with the new per-log approach. It cleanly returnsOk(None)
by default, ensuring consistent testing.
47-50
: Mock trait updates match new signatures.Defining mock implementations for the new methods (
contract_addresses_map
,safe_address
, andcollect_log_event
) ensures test coverage matches the trait changes.chain/api/src/lib.rs (4)
24-24
: Imports align with unified RPC approach.Bringing in
DefaultHttpPostRequestor
andJsonRpcClient
fromhopr_chain_rpc::types
consolidates HTTP request handling. The removal of older local aliases is consistent with the overall refactoring.
109-109
: Refined generic type for RPC client.Using
RpcEthereumClient<RpcOperations<JsonRpcClient>>
clarifies the layering of RPC, ensuring the separation of concerns between Ethereum transactions and the JSON RPC logic.
245-245
: Passingrpc_operations
to event handlers.Adding
self.rpc_operations.clone()
toContractEventHandlers::new()
provides direct access to RPC operations for log processing, likely crucial for retrieving additional on-chain data.
322-327
:❓ Verification inconclusive
New method to retrieve Safe’s HOPR allowance.
This addition checks the allowance between the Safe address and the
channels
contract. Confirm thatchannels
is the expected contract for token allowance, especially if WXHOPR or other tokens are involved.
🏁 Script executed:
#!/bin/bash # Verify where and how get_allowance is used across the codebase to confirm correctness: rg "get_allowance\s*\("Length of output: 1094
Verify the token contract used for HOPR allowance
The new method in
chain/api/src/lib.rs
(lines 322–327) retrieves the allowance by callingget_allowance(self.safe_address, self.contract_addresses.channels)
, implying that the Safe address is the owner and thechannels
contract is the expected spender. Please confirm that this is indeed the intended behavior—namely, that thechannels
contract is the correct destination for HOPR allowances. Note that similar allowance calls elsewhere in the codebase (e.g., inchain/indexer/src/block.rs
) pass addresses in a different order, so double-check that this usage aligns with the intended functionality, especially in scenarios where WXHOPR or other tokens may be involved.chain/indexer/src/block.rs (5)
1-2
: New imports look fine.
The added imports fromethers::contract::EthEvent
andethers::types::{Filter, H256}
provide event filtering capabilities and are used in the subsequent code.
8-8
: Import for token event filters is appropriate.
TheApprovalFilter
andTransferFilter
are needed to properly detect token-related events.
11-11
: No issues with these trait imports.
Bringing inFilterSet
andHoprIndexerRpcOperations
aligns with the new filtering logic.
128-128
: Generate filters prior to streaming.
Callinggenerate_log_filters
early ensures consistent filter configurations.
543-620
: Safe balance & allowance updates post-sync.
The additional parameters (next_block_to_process
,safe_address
,channels_address
) extend the indexing logic to update on-chain data after finishing sync. The approach is sound as long as reorgs are rare.chain/rpc/src/lib.rs (5)
36-36
: Exporting a newtypes
module is fine.
Ensure that this module is properly documented if extended further.
167-176
: Introduction ofFilterSet
for comprehensive filtering.
This struct neatly separates token vs non-token filters. Verify correct usage across all call sites to avoid unexpected filter overlaps.
345-346
: Newget_allowance
method forHoprRpcOperations
.
Allows retrieving HOPR token allowance, which is critical for checking spending permissions. Looks consistent with the rest of the trait.
412-417
: Indexer-specific allowance & balance retrieval.
Adding these toHoprIndexerRpcOperations
extends the indexer’s capabilities to manage token states. No immediate concerns.
425-427
: Extendedtry_stream_logs
signature.
UsingFilterSet
plus anis_synced
flag is a logical technique for tailoring filters during nodal catch-up vs. real-time streaming.chain/rpc/src/indexer.rs (7)
15-15
: Concurrent stream extension import is appropriate.
Thethen_concurrent
usage reduces retrieval time by processing multiple filters in parallel.
19-20
: Primitive types import is unobtrusive.
Ensures easier usage ofAddress
,Balance
, etc.
23-23
: Bringing inFilterSet
and others from this crate.
Centralizing key types helps keep the indexing logic cohesive.
67-72
: Renamed signature with multiple filters.
Transitioning tostream_logs(filters: Vec<Filter>, ...)
properly aligns with chunk-based retrieval.
81-96
: Concurrent retrieval for subrange filters.
Utilizing concurrency can significantly reduce the fetching time. Confirm that the provider’s rate limit or concurrency constraints are respected.
129-148
: Revisedtry_stream_logs
to useFilterSet
andis_synced
.
Conditionally excluding token contract logs until the indexer is synced helps reduce overhead. It’s a well-structured approach.
153-159
: Empty filter guard.
Returning an error whenFilterSet::all
is empty avoids ambiguous scenarios in the stream. This is a good safeguard.chain/indexer/src/handlers.rs (15)
5-5
: No issues found.
This import is necessary for theDebug
trait implementation.
19-21
: Imports look good.
These imports cleanly introduce the new RPC types needed for enhanced on-chain interactions.
61-62
: Field injection for RPC operations.
Addingrpc_operations
to the struct is sensible for centralizing chain interaction logic. Make sure to avoid unintended concurrency issues when using it in async flows.
79-91
: Constructor update for RPC injection.
The updated signature properly injectsrpc_operations
so the handlers can fetch on-chain data. This design should improve maintainability and testability.
194-199
: Trace logging for balance decrease.
These trace logs are effective for debugging channel balance updates.
221-226
: Trace logging for balance increase.
Similar to the prior addition, these logs will aid in diagnosing channel funding issues.
249-251
: Trace logging for channel closure.
This mirrors other trace statements and keeps event visibility consistent.
549-566
: Safe allowance update logic.
Fetching the allowance from the chain upon approval and logging a misconfiguration if irrelevant are both appropriate. Again, you might consider adjusting the log severity for non-critical mismatches.
835-835
: Database trait constraints
AddingDebug + Send + Sync + 'static
is consistent with usage in async contexts and debugging.
850-856
: New trait methods for addresses
Exposingcontract_addresses_map
andsafe_address
is clear and maintains standard accessor patterns.
885-904
: Improved error handling for log processing.
Wrapping the processing in a transaction and preserving errors ensures logs aren’t marked as processed prematurely, which aligns well with the PR objective of avoiding missed retries.
913-913
: Test import usage
Bringing in the module withuse super::*;
is a standard approach for test scopes.
915-916
: Additional test imports
These imports (anyhow, ethers, hex_literal, multiaddr, mock RPC) appear routine and pose no apparent issues.Also applies to: 918-919, 921-922, 945-945
975-975
: Mock RPC initialization in tests
Creating the mock RPC object helps isolate and validate chain interactions during testing.
993-993
: Passing mock RPC into the handler
This ensures consistent, testable RPC usage in the test environment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
🧹 Nitpick comments (1)
chain/indexer/src/handlers.rs (1)
65-73
: Debug implementation doesn't include rpc_operations field.The Debug implementation doesn't display the
rpc_operations
field, which could be intentional but might hide useful information during debugging.Consider including the
rpc_operations
field in the debug output:- f.debug_struct("ContractEventHandler") - .field("addresses", &self.addresses) - .field("safe_address", &self.safe_address) - .field("chain_key", &self.chain_key) - .finish_non_exhaustive() + f.debug_struct("ContractEventHandler") + .field("addresses", &self.addresses) + .field("safe_address", &self.safe_address) + .field("chain_key", &self.chain_key) + .field("rpc_operations", &"<RpcOperations>") + .finish_non_exhaustive()
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
Cargo.lock
is excluded by!**/*.lock
📒 Files selected for processing (2)
chain/indexer/Cargo.toml
(1 hunks)chain/indexer/src/handlers.rs
(16 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- chain/indexer/Cargo.toml
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: tests-smoke-websocket
- GitHub Check: Linter
🔇 Additional comments (18)
chain/indexer/src/handlers.rs (18)
5-5
: Appropriate Debug implementation imports.This addition properly supports the Debug trait implementation for the struct.
19-21
: Good addition of RPC operation imports.These imports support the new RPC interaction capabilities added to the handlers, aligning with the PR objective to reduce required logs by focusing on relevant events.
61-62
: Appropriate new field for blockchain interaction.Adding the
rpc_operations
field enables direct blockchain interactions, eliminating the need to process and store all token events locally.
79-92
: Constructor updated correctly with new parameter.The
new
constructor has been properly updated to accept and initialize therpc_operations
field.
194-198
: Improved trace logging format.The trace logging has been restructured for better readability and clarity.
221-225
: Consistent logging structure.This change maintains consistency with the improved trace logging format used elsewhere.
249-252
: Improved logging with proper field formatting.The trace logging has been updated to use a more consistent and readable format.
530-533
: Enhanced error message clarity.The error message has been improved to better explain the filter misconfiguration issue.
534-546
: Implemented fetching balance from blockchain.Instead of relying on the event data, the handler now fetches the actual balance from the blockchain, which is more accurate and resilient. This also helps reduce the required logs by only processing what's needed.
561-573
: Improved allowance handling.Similar to the balance handling, the code now fetches the current allowance from the blockchain instead of relying on event data. This improves accuracy and reduces dependency on processing all token events.
575-578
: Improved error message for token approval events.The error message now more clearly indicates the misconfiguration issue with approval events not involving the safe and channels contract.
890-919
: Significant refactoring of log event collection.The
collect_log_event
method has been refactored to process a single log event rather than a batch. This simplifies control flow and improves error handling by:
- Properly tracking transaction hash, log ID, and block ID
- Adding clear debug and error logging
- Ensuring that logs aren't marked as processed if an error occurs during processing
This implementation aligns with the PR objectives to enhance error handling and ensure logs aren't marked as processed on errors.
847-847
: Added Debug trait bound to Db parameter.The addition of the Debug trait bound is necessary to support the updated Debug implementation of ContractEventHandlers.
862-868
: Added helpful accessor methods.These methods provide convenient access to contract addresses map and safe address, improving the API's usability.
927-936
: Test import updates.The imports have been properly updated to align with the implementation changes.
957-958
: Added RPC mock for testing.The test now uses a MockRpcOperations, which allows proper testing of the RPC operation functionality.
987-992
: Test initialization correctly updated.The test initialization now includes creating a MockRpcOperations instance for the handler.
1005-1005
: Updated struct initialization in tests.Properly initializes the new rpc_operations field in the test handlers.
Retarget on |
Replaced by #7213 |
This change drastically reduces the amount of logs which the indexer needs to process by excluding most of the wxHOPR token events and only process the ones which are relevant to the particular node.
Moreover, the indexer has been made error-safe as in that it does not mark logs as processed if errors occurred and panicking. Subsequently, the node will then try again to process a log.
Fixes #6792
Fixes #6881