Skip to content

Conversation

tolbrino
Copy link
Contributor

@tolbrino tolbrino commented Dec 12, 2024

This change drastically reduces the amount of logs which the indexer needs to process by excluding most of the wxHOPR token events and only process the ones which are relevant to the particular node.
Moreover, the indexer has been made error-safe as in that it does not mark logs as processed if errors occurred and panicking. Subsequently, the node will then try again to process a log.

Fixes #6792
Fixes #6881

@tolbrino tolbrino self-assigned this Dec 12, 2024
Copy link
Contributor

coderabbitai bot commented Dec 12, 2024

📝 Walkthrough

Walkthrough

This pull request applies extensive modifications across multiple modules. It removes deprecated type aliases, streamlines import paths, and updates method signatures for blockchain interactions. New asynchronous methods for retrieving token allowances and processing logs have been introduced, and log filtering has been refactored to use a new filter collection structure. Additionally, feature flags and dependencies have been updated to support multiple asynchronous runtimes.

Changes

File(s) Change Summary
chain/api/src/lib.rs Removed type aliases (DefaultHttpPostRequestor, JsonRpcClient); added async fn get_safe_hopr_allowance(&self) in HoprChain.
chain/indexer/(Cargo.toml, src/block.rs, src/constants.rs, src/handlers.rs, src/traits.rs) Added features "hopr-chain-rpc/runtime-async-std" and "hopr-chain-rpc/runtime-tokio"; refactored log filtering via a new FilterSet, removed token filters, and updated event handler and trait method signatures.
chain/rpc/(Cargo.toml, src/indexer.rs, src/lib.rs, src/rpc.rs, src/types.rs) Introduced dependency rust-stream-ext-concurrent; replaced LogFilter with FilterSet; updated streaming functions; added asynchronous get_allowance methods and new type aliases (DefaultHttpPostRequestor, JsonRpcClient).
chain/types/src/lib.rs Streamlined import statements by consolidating dependencies and removing redundancies.
hopli/src/environment_config.rs Updated import paths for DefaultHttpPostRequestor and re-exported JsonRpcClient directly to simplify the client configuration.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant HoprChain
    participant RpcOperations
    participant Blockchain

    Client->>HoprChain: get_safe_hopr_allowance()
    HoprChain->>RpcOperations: get_allowance(safe_address, contract)
    RpcOperations->>Blockchain: Query allowance(contract, safe_address)
    Blockchain-->>RpcOperations: Return Balance
    RpcOperations-->>HoprChain: Return Balance
    HoprChain-->>Client: Return Balance
Loading
sequenceDiagram
    participant Block
    participant Indexer
    participant EventHandler
    participant RpcOperations
    participant Blockchain

    Block->>Indexer: New block with logs
    Indexer->>EventHandler: collect_log_event(log)
    EventHandler->>RpcOperations: get_balance/get_allowance query (if token event)
    RpcOperations->>Blockchain: Fetch token data
    Blockchain-->>RpcOperations: Return data
    RpcOperations-->>EventHandler: Return data
    EventHandler-->>Indexer: Process event & update state
Loading

Possibly related PRs

Suggested labels

bug, improvement, crate:hopr-lib, crate:hopr-crypto-types

Suggested reviewers

  • Teebor-Choka
✨ Finishing Touches
  • 📝 Generate Docstrings (Beta)

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@tolbrino tolbrino changed the title Tb/20241127 slim indexer rebased indexer: Reduce required on-chain logs Dec 12, 2024
@Teebor-Choka Teebor-Choka added this to the 3.0.0 milestone Jan 22, 2025
@Teebor-Choka Teebor-Choka changed the base branch from master to release/singapore January 24, 2025 08:50
@NumberFour8 NumberFour8 modified the milestones: 3.0.0, 2.2.2 Feb 2, 2025
@Teebor-Choka Teebor-Choka modified the milestones: 2.2.2, 2.2.3 Feb 4, 2025
@Teebor-Choka Teebor-Choka linked an issue Feb 25, 2025 that may be closed by this pull request
@tolbrino tolbrino marked this pull request as ready for review February 25, 2025 07:29
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (4)
chain/rpc/src/lib.rs (2)

167-176: Introducing FilterSet for refined log filtering.
Consider adding helper methods (e.g., is_empty()) or cohesive merges of filters if needed.


345-347: Added get_allowance to HoprRpcOperations.
Ensure test coverage for typical and edge cases (e.g., zero allowance).

Would you like assistance generating tests for this method?

chain/indexer/src/handlers.rs (1)

61-62: Doc comment on struct field

Declaring doc comments (///) on a struct field is unconventional but still valid Rust. Ensure the doc clearly explains the field for readers and generated documentation.

chain/rpc/src/indexer.rs (1)

80-125: Refined logic for log subrange fetching and sorting

This block:

  • Iterates over subrange filters
  • Fetches each filter’s logs in parallel
  • Flattens all logs into a single vector
  • Sorts logs by block number

The approach is sensible for chunked retrieval, but watch out for potential performance issues with large vectors. Also confirm block reordering is correct if logs come in partial sets.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 096d423 and 4ffe5a3.

📒 Files selected for processing (12)
  • chain/api/src/lib.rs (4 hunks)
  • chain/indexer/Cargo.toml (1 hunks)
  • chain/indexer/src/block.rs (21 hunks)
  • chain/indexer/src/constants.rs (0 hunks)
  • chain/indexer/src/handlers.rs (16 hunks)
  • chain/indexer/src/traits.rs (4 hunks)
  • chain/rpc/src/indexer.rs (6 hunks)
  • chain/rpc/src/lib.rs (4 hunks)
  • chain/rpc/src/rpc.rs (2 hunks)
  • chain/rpc/src/types.rs (1 hunks)
  • chain/types/src/lib.rs (2 hunks)
  • hopli/src/environment_config.rs (1 hunks)
💤 Files with no reviewable changes (1)
  • chain/indexer/src/constants.rs
✅ Files skipped from review due to trivial changes (1)
  • chain/types/src/lib.rs
🧰 Additional context used
🧠 Learnings (1)
chain/rpc/src/indexer.rs (1)
Learnt from: NumberFour8
PR: hoprnet/hoprnet#6226
File: chain/rpc/src/indexer.rs:35-60
Timestamp: 2024-11-12T10:55:50.205Z
Learning: The `split_range` function in `chain/rpc/src/indexer.rs` requires cloning the `LogFilter` because each iteration potentially modifies the filter for a specific block range.
⏰ Context from checks skipped due to timeout of 90000ms (8)
  • GitHub Check: hoprd / docker
  • GitHub Check: hopli / docker
  • GitHub Check: Docs / Rust docs
  • GitHub Check: tests-smoke-websocket
  • GitHub Check: tests-smart-contracts
  • GitHub Check: tests-unit
  • GitHub Check: tests-unit-nightly
  • GitHub Check: Linter
🔇 Additional comments (59)
chain/indexer/Cargo.toml (2)

16-18: New dependencies added correctly

The addition of the hopr-chain-rpc/runtime-async-std feature dependency is properly aligned with the other runtime features.


21-23: New dependencies added correctly

The addition of the hopr-chain-rpc/runtime-tokio feature dependency is properly aligned with the other runtime features.

chain/rpc/src/types.rs (3)

7-11: Good conditional compilation

The code correctly handles the case where both runtime features could be enabled during testing by using all(feature = "runtime-tokio", not(feature = "runtime-async-std")) for the tokio feature.


12-15: Type alias improves code organization

The JsonRpcClient type alias helps centralize the definition which can be imported by other modules, reducing duplication and improving maintainability.


1-6:

✅ Verification successful

Address the TODO comment in a future PR

The TODO comment indicates that DefaultHttpPostRequestor should be an internal type, and that hopr_lib::chain must be moved to this package. Consider creating a follow-up task to address this technical debt.


🏁 Script executed:

#!/bin/bash
# Check for references to hopr_lib::chain in the codebase
rg "hopr_lib::chain" --type rust

Length of output: 141


Action: Create a follow-up task for migrating the HTTP request implementation.

The current code in chain/rpc/src/types.rs defines DefaultHttpPostRequestor and includes a TODO comment indicating that this type should be made internal, with the hopr_lib::chain functionality moved into this package. A search confirms that the only reference to hopr_lib::chain exists in this comment. No other issues were detected, so it’s appropriate to document the technical debt and address it in a separate future PR.

  • Location: chain/rpc/src/types.rs (lines 1-6)
  • Next Step: Create a follow-up issue to track the migration of hopr_lib::chain into the package.
hopli/src/environment_config.rs (2)

29-32: Import reorganization improves module structure

The import statements have been reorganized to better structure the dependencies, separating the imports from hopr_chain_rpc and improving readability.


37-37: Type alias simplification

Replacing the verbose type alias definition with a direct pub use statement from the new hopr_chain_rpc::types module simplifies the code and ensures type consistency across the codebase.

chain/rpc/src/rpc.rs (2)

113-114: Mockable struct enhances testability

Adding the #[cfg_attr(test, mockall::automock)] attribute enables automatic mock generation for the RpcOperations struct during testing, which improves testability.


191-206: Well-implemented allowance retrieval method

The new get_allowance method is well-implemented with proper error handling and follows the same pattern as the existing get_balance method. This addition enhances the RPC operations interface by allowing token allowance queries.

This method aligns with the PR objective to support the indexer in filtering out most wxHOPR token events, allowing it to only process events that are relevant to a specific node.

chain/indexer/src/traits.rs (9)

3-3: Imported Arc usage is appropriate for shared data handling.
No issues found with the new import; it cleanly supports returning references in the trait methods.


6-6: New ContractAddresses import aligns well with usage.
This import is consistent with the updated trait returning Arc.


15-16: New method returning Arc is clear and consistent.
This provides a thread-safe reference to contract addresses, which may improve concurrent access patterns.


19-20: Introducing a safe_address method completes contract metadata retrieval.
This addition complements existing contract address getters nicely.


21-21: Asynchronous log-event collection method is well-scoped.
Returning Option is flexible for logs that may not produce an event.


35-36: Updated mock documentation snippet matches the new collect_log_event method.
This helps clarify usage in testing environments.


47-47: Mock method matches the real contract_addresses_map signature.
Maintains consistency with the trait's new method signature.


49-49: Mock method for safe_address is consistent with the trait.
No concerns here; it aligns with the real method.


50-50: Mock method for collect_log_event aligns with async trait usage.
Ensures the mock can fully mimic the real method’s expected behavior.

chain/api/src/lib.rs (4)

24-24: Reintroducing DefaultHttpPostRequestor and JsonRpcClient usage seems correct.
These imports appear to be used consistently in RPC client initialization.


109-109: Refined ActionQueueType to use RpcEthereumClient<RpcOperations>.
This centralizes the RPC usage, ensuring a single consistent client type.


245-245: Passing cloned rpc_operations into ContractEventHandlers respects ownership rules.
This approach avoids lifecycle or borrow issues when sharing the same RPC object.


322-328: New get_safe_hopr_allowance method is neatly implemented.
It delegates directly to get_allowance on the channels contract, returning a Balance.

chain/rpc/src/lib.rs (4)

36-36: Re-exporting types module.
No conflicts or issues; consolidates RPC-related type definitions appropriately.


412-413: Indexer also provides get_allowance, mirroring HoprRpcOperations.
Consistency across the traits is beneficial for maintainability.


416-416: Providing get_balance in HoprIndexerRpcOperations broadens indexer capabilities.
This parallels the existing HoprRpcOperations method.


425-427: try_stream_logs now takes FilterSet and is_synced flag.
The updated signature accommodates more granular filtering logic.

chain/indexer/src/handlers.rs (17)

19-21: Imports from hopr_chain_rpc added

These imports provide the ability to perform RPC operations and work with JsonRpcClient/HoprRpcOperations. The addition appears consistent with the revised design where handlers can directly call the RPC methods.


65-65: Debug implementation note

Implementing a custom Debug is fine as it selectively displays fields. No immediate issues.


79-85: Constructor extended with rpc_operations parameter

This constructor now requires an RpcOperations<JsonRpcClient> instance to enable direct chain RPC calls (e.g., fetching balance/allowance). The signature and usage appear consistent with the rest of the file.

Also applies to: 91-91


187-188: New match arm for channel events

The match event { HoprChannelsEvents::ChannelBalanceDecreasedFilter(...) => ... } block is unchanged except for the leading line. No specific concern here.


194-198: Added trace logging for decreased balance events

This trace! call improves debugging by capturing channel ID and channel existence. The usage of trace-level logging is appropriate for detailed diagnostics.


221-226: Trace logging for increased balance events

Similarly, these trace statements help confirm channel ID and existence. This log is valuable for debugging deposit events.


249-253: Trace logging for closed channel events

Again, a consistent trace log describing the channel ID and whether the channel is in the database. This aids debugging channel termination scenarios.


517-520: Error logging for irrelevant transfer events

This path logs an error if the token transfer does not involve the known safe address. This helps spot potential filter misconfiguration early.


522-534: Updating safe balance on incoming transfers

When tokens arrive at the safe, the code fetches the new on-chain balance. The match clause properly handles any RPC failure. This is a good approach for reliability.


555-557: Storing safe allowance on successful fetch

On success, the code updates the DB with the new allowance. Looks correct.


559-560: Error handling for allowance fetch failures

The design logs the error without retry. Confirm whether a retry or fallback is needed.


563-566: Misconfiguration error for approvals not involving the safe

Treating approvals that do not match the safe/channels combination as an error is consistent with your filtering logic.


835-835: Additional trait bounds for Db

Expanding the trait bounds (including Debug) suggests you might log the Db type or need deeper introspection. No immediate concerns.


850-852: New helper method: contract_addresses_map

This method returns a clone of the internal Arc<ContractAddresses>. It improves readability but watch for potential overhead if called frequently in tight loops.


854-856: New helper method: safe_address

Returning the safe address directly is straightforward and likely improves code clarity.


879-904: Enhanced log processing and event creation

This chunk inside collect_log_event handles the newly stored states (tx_hash, log_index, etc.) and constructs a SignificantChainEvent. The approach to capturing more metadata (e.g., block_id, log_id) in debug logs is beneficial for troubleshooting. Ensure all usage sites align with the new data structures.


913-917: Additional test imports

Bringing in anyhow, Context, and refinements from ethers and the standard library is consistent with test expansions.

chain/rpc/src/indexer.rs (8)

18-19: Add import for hopr_primitive_types::prelude::*

Bringing these types is coherent if references to Address, Balance, etc. are used extensively.


22-22: Importing additional core traits/types

Importing BlockWithLogs, FilterSet, HoprIndexerRpcOperations, Log from the crate clarifies usage within this file.


66-71: Function signature adjustments in stream_logs

The method now accepts multiple filters: Vec<ethers::types::Filter> and a block range. Ensure consistent usage of these arguments across the code.


134-137: get_allowance default forwarding

This method delegates directly to the same-named method in RpcOperations. Straight pass-through with no additional logic is fine.


138-141: get_balance default forwarding

Similarly, forwarding to the underlying RpcOperations is correct if no local transformations are needed.


145-146: Updated try_stream_logs parameters

Adding filters: FilterSet and is_synced: bool clarifies the difference between ignoring token events when unsynced vs. including them once fully synced. This addresses log volume concerns and is logically sound.


153-158: Filtering out token logs if not synced

Preventing the indexing of token-related events until the node is caught up significantly reduces log overhead. This approach looks aligned with the PR’s objective.


194-194: Storing logs fetch in retrieved_logs

Using self.stream_logs(log_filters.clone(), from_block, latest_block) is a straightforward composition with the new subrange logic.

chain/indexer/src/block.rs (8)

1-2: Newly imported ethers references

Introducing EthEvent, Filter, H256 suggests more direct interaction with Ethers-based events and filter building.


8-8: Import token filters

Pulling in TransferFilter and ApprovalFilter from hopr_token aligns with the new token log filtering approach.


11-11: Importing hopr_chain_rpc items

BlockWithLogs, FilterSet, and HoprIndexerRpcOperations integration signals that the indexer is now using these more structured constructs (rather than a single LogFilter).


108-108: Generating log filters before streaming logs

Storing let log_filters = Self::generate_log_filters(&logs_handler); clarifies that the indexing logic chooses which addresses/topics to track. Good to see a separate function.


329-335: Comment on grouping big filters logic

These comment lines outline a shift from building a single LogFilter to constructing a more sophisticated set. Reading them is helpful for explaining the rationale.


336-386: generate_log_filters function

This method:

  • Collects addresses from the logs_handler, excluding the token contract
  • Builds a base filter for those addresses
  • Builds specialized filters for safe-specific transfers and approvals

It addresses the PR’s goals of filtering out most token events except those explicitly involving the safe. Well-structured and clear.


463-505: Revised block-level log processing

Here, logs are:

  1. Converted to a stream::iter(...)
  2. Collected into events via logs_handler.collect_log_event(...)
  3. Marked processed only upon successful event extraction

This ensures consistent handling of partial failures. The panic on error is strong but aligns with your design principle of preventing data corruption.


541-550: Extended parameters in calculate_sync_process

Including db: Db, safe_address: Option<Address>, and channels_address: Option<Address> allows the indexer to refresh balances or allowances post-sync. This matches the PR’s objective to update the node’s safe data once fully synced.

Also applies to: 553-553

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (8)
chain/rpc/src/types.rs (1)

1-10: Well-structured conditional HTTP requestor implementation

The conditional compilation ensures that the appropriate HTTP requestor is used based on the enabled runtime, which is a clean approach to handle multiple runtime support.

Consider addressing the TODO comment in a future PR. Moving hopr_lib::chain to this package would help with encapsulation.

chain/indexer/src/block.rs (3)

329-386: Good approach to centralize filter creation.
Defining one function to build the FilterSet improves readability. Consider adding safer defaults or warnings if no addresses/topics are found to avoid an empty filter situation.


464-505: Panicking on log processing failure might halt the entire node.
Marking logs as processed only if collect_log_event succeeds prevents data inconsistency, but panics can disrupt indexing in production. Checking whether to use a recovery or retry mechanism instead of panicking could improve resilience.


622-624: Graceful send-failure handling.
It’s acceptable to log the error from try_send and continue. If lost synchronization signals are detrimental, consider adding a fallback mechanism.

chain/rpc/src/indexer.rs (2)

40-45: Flexible chunked splitting for multiple filters.
Parameterizing the split by a Vec<Filter> supports concurrent filter application. Ensure potential large ranges are handled well in production.


53-57: Applying subrange blocks in each filter.
Chaining .from_block() and .to_block() systematically is a neat approach. Keep an eye on potential performance overhead for large filter sets.

chain/indexer/src/handlers.rs (2)

65-65: Consider including rpc_operations in Debug output.
The impl Debug currently omits the rpc_operations field, which might be helpful for diagnostics or debugging.


517-534: Safe balance update logic.
The approach of verifying the transfer is relevant to the safe and then querying on-chain is correct. However, consider lowering the log level from error! to warn! when a transfer is irrelevant to the safe, unless it's always considered a genuine misconfiguration.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 096d423 and ebfdfb2.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (13)
  • chain/api/src/lib.rs (4 hunks)
  • chain/indexer/Cargo.toml (1 hunks)
  • chain/indexer/src/block.rs (21 hunks)
  • chain/indexer/src/constants.rs (0 hunks)
  • chain/indexer/src/handlers.rs (16 hunks)
  • chain/indexer/src/traits.rs (4 hunks)
  • chain/rpc/Cargo.toml (1 hunks)
  • chain/rpc/src/indexer.rs (6 hunks)
  • chain/rpc/src/lib.rs (4 hunks)
  • chain/rpc/src/rpc.rs (2 hunks)
  • chain/rpc/src/types.rs (1 hunks)
  • chain/types/src/lib.rs (2 hunks)
  • hopli/src/environment_config.rs (1 hunks)
💤 Files with no reviewable changes (1)
  • chain/indexer/src/constants.rs
✅ Files skipped from review due to trivial changes (1)
  • chain/types/src/lib.rs
⏰ Context from checks skipped due to timeout of 90000ms (8)
  • GitHub Check: tests-smoke-websocket
  • GitHub Check: tests-smart-contracts
  • GitHub Check: tests-unit-nightly
  • GitHub Check: tests-unit
  • GitHub Check: hoprd / docker
  • GitHub Check: hopli / docker
  • GitHub Check: Docs / Rust docs
  • GitHub Check: Linter
🔇 Additional comments (48)
chain/rpc/Cargo.toml (1)

44-44: Adding new dependency for concurrent stream processing

The addition of rust-stream-ext-concurrent is well aligned with the PR's objective to improve indexer performance by managing concurrent stream operations more efficiently.

chain/indexer/Cargo.toml (1)

17-17: Good propagation of RPC runtime features

Adding the RPC runtime features for both async-std and tokio ensures that the indexer can utilize the RPC functionality with either async runtime, providing better flexibility and coherence with the rest of the system.

Also applies to: 22-22

chain/rpc/src/types.rs (1)

12-14: Clear JsonRpcClient type definition

This type alias standardizes how JSON RPC calls are made, which is important for the PR's goal of improving error handling in the indexer.

chain/rpc/src/rpc.rs (2)

113-113: Good testing support with mockall

Adding the mockall automock attribute enables proper mocking in tests, which is essential for reliable unit testing of the RPC operations.


191-206:

❓ Verification inconclusive

Well-implemented token allowance retrieval method

The new get_allowance method follows the established pattern for RPC operations and includes proper error handling. This method will help with focusing on relevant token events as mentioned in the PR objectives.

Make sure all places that need to check token allowances are updated to use this new method. This is particularly important for the indexer logic that filters logs.


🏁 Script executed:

#!/bin/bash
# Looking for potential callers of the token's allowance function directly
rg -A 2 -B 2 "\.allowance\(" --exclude "rpc.rs"

Length of output: 459


Token allowance retrieval confirmed.
The new get_allowance method in chain/rpc/src/rpc.rs is correctly implemented with appropriate error handling and follows the established RPC pattern. However, please verify that all parts of the code—particularly the indexer logic—no longer call the token’s allowance function directly but instead use this new method.

  • Confirm that there are no direct invocations of .allowance( in the codebase (excluding rpc.rs). For example, you can run:
    rg -A2 -B2 '\.allowance\(' -g '!rpc.rs'
  • Manually inspect any indexer-related logic to ensure it now references get_allowance.
hopli/src/environment_config.rs (2)

30-31: Imports appear correctly updated.

These new imports from hopr_chain_rpc::types replace the previous nested imports. This streamlines the code and aligns with the recent chain/rpc refactors.


37-37: Updated type alias usage.

Switching to pub use hopr_chain_rpc::types::JsonRpcClient; instead of defining a local type alias helps maintain consistency across the codebase and reduces redundancy.

chain/indexer/src/traits.rs (5)

3-3: Arc import for concurrency.

Introducing std::sync::Arc is necessary to return thread-safe references in subsequent trait methods. This is a sound approach for sharing read-only data.


5-6: New imports for event and addresses.

Adding SignificantChainEvent and ContractAddresses references indicates a shift toward more granular log event handling and storing contract data. This looks consistent with the rest of the trait changes.


15-21: Additional trait methods to handle log events more granularly.

  1. contract_addresses_map returns an Arc<ContractAddresses> for shared data without copying.
  2. safe_address provides direct retrieval of the safe’s address.
  3. collect_log_event processes a single log, returning an optional SignificantChainEvent, aligning with the new approach of handling logs individually.

35-36: Mock usage for single event processing.

This mock setup for expect_collect_log_event is aligned with the new per-log approach. It cleanly returns Ok(None) by default, ensuring consistent testing.


47-50: Mock trait updates match new signatures.

Defining mock implementations for the new methods (contract_addresses_map, safe_address, and collect_log_event) ensures test coverage matches the trait changes.

chain/api/src/lib.rs (4)

24-24: Imports align with unified RPC approach.

Bringing in DefaultHttpPostRequestor and JsonRpcClient from hopr_chain_rpc::types consolidates HTTP request handling. The removal of older local aliases is consistent with the overall refactoring.


109-109: Refined generic type for RPC client.

Using RpcEthereumClient<RpcOperations<JsonRpcClient>> clarifies the layering of RPC, ensuring the separation of concerns between Ethereum transactions and the JSON RPC logic.


245-245: Passing rpc_operations to event handlers.

Adding self.rpc_operations.clone() to ContractEventHandlers::new() provides direct access to RPC operations for log processing, likely crucial for retrieving additional on-chain data.


322-327:

❓ Verification inconclusive

New method to retrieve Safe’s HOPR allowance.

This addition checks the allowance between the Safe address and the channels contract. Confirm that channels is the expected contract for token allowance, especially if WXHOPR or other tokens are involved.


🏁 Script executed:

#!/bin/bash
# Verify where and how get_allowance is used across the codebase to confirm correctness:
rg "get_allowance\s*\("

Length of output: 1094


Verify the token contract used for HOPR allowance

The new method in chain/api/src/lib.rs (lines 322–327) retrieves the allowance by calling get_allowance(self.safe_address, self.contract_addresses.channels), implying that the Safe address is the owner and the channels contract is the expected spender. Please confirm that this is indeed the intended behavior—namely, that the channels contract is the correct destination for HOPR allowances. Note that similar allowance calls elsewhere in the codebase (e.g., in chain/indexer/src/block.rs) pass addresses in a different order, so double-check that this usage aligns with the intended functionality, especially in scenarios where WXHOPR or other tokens may be involved.

chain/indexer/src/block.rs (5)

1-2: New imports look fine.
The added imports from ethers::contract::EthEvent and ethers::types::{Filter, H256} provide event filtering capabilities and are used in the subsequent code.


8-8: Import for token event filters is appropriate.
The ApprovalFilter and TransferFilter are needed to properly detect token-related events.


11-11: No issues with these trait imports.
Bringing in FilterSet and HoprIndexerRpcOperations aligns with the new filtering logic.


128-128: Generate filters prior to streaming.
Calling generate_log_filters early ensures consistent filter configurations.


543-620: Safe balance & allowance updates post-sync.
The additional parameters (next_block_to_process, safe_address, channels_address) extend the indexing logic to update on-chain data after finishing sync. The approach is sound as long as reorgs are rare.

chain/rpc/src/lib.rs (5)

36-36: Exporting a new types module is fine.
Ensure that this module is properly documented if extended further.


167-176: Introduction of FilterSet for comprehensive filtering.
This struct neatly separates token vs non-token filters. Verify correct usage across all call sites to avoid unexpected filter overlaps.


345-346: New get_allowance method for HoprRpcOperations.
Allows retrieving HOPR token allowance, which is critical for checking spending permissions. Looks consistent with the rest of the trait.


412-417: Indexer-specific allowance & balance retrieval.
Adding these to HoprIndexerRpcOperations extends the indexer’s capabilities to manage token states. No immediate concerns.


425-427: Extended try_stream_logs signature.
Using FilterSet plus an is_synced flag is a logical technique for tailoring filters during nodal catch-up vs. real-time streaming.

chain/rpc/src/indexer.rs (7)

15-15: Concurrent stream extension import is appropriate.
The then_concurrent usage reduces retrieval time by processing multiple filters in parallel.


19-20: Primitive types import is unobtrusive.
Ensures easier usage of Address, Balance, etc.


23-23: Bringing in FilterSet and others from this crate.
Centralizing key types helps keep the indexing logic cohesive.


67-72: Renamed signature with multiple filters.
Transitioning to stream_logs(filters: Vec<Filter>, ...) properly aligns with chunk-based retrieval.


81-96: Concurrent retrieval for subrange filters.
Utilizing concurrency can significantly reduce the fetching time. Confirm that the provider’s rate limit or concurrency constraints are respected.


129-148: Revised try_stream_logs to use FilterSet and is_synced.
Conditionally excluding token contract logs until the indexer is synced helps reduce overhead. It’s a well-structured approach.


153-159: Empty filter guard.
Returning an error when FilterSet::all is empty avoids ambiguous scenarios in the stream. This is a good safeguard.

chain/indexer/src/handlers.rs (15)

5-5: No issues found.
This import is necessary for the Debug trait implementation.


19-21: Imports look good.
These imports cleanly introduce the new RPC types needed for enhanced on-chain interactions.


61-62: Field injection for RPC operations.
Adding rpc_operations to the struct is sensible for centralizing chain interaction logic. Make sure to avoid unintended concurrency issues when using it in async flows.


79-91: Constructor update for RPC injection.
The updated signature properly injects rpc_operations so the handlers can fetch on-chain data. This design should improve maintainability and testability.


194-199: Trace logging for balance decrease.
These trace logs are effective for debugging channel balance updates.


221-226: Trace logging for balance increase.
Similar to the prior addition, these logs will aid in diagnosing channel funding issues.


249-251: Trace logging for channel closure.
This mirrors other trace statements and keeps event visibility consistent.


549-566: Safe allowance update logic.
Fetching the allowance from the chain upon approval and logging a misconfiguration if irrelevant are both appropriate. Again, you might consider adjusting the log severity for non-critical mismatches.


835-835: Database trait constraints
Adding Debug + Send + Sync + 'static is consistent with usage in async contexts and debugging.


850-856: New trait methods for addresses
Exposing contract_addresses_map and safe_address is clear and maintains standard accessor patterns.


885-904: Improved error handling for log processing.
Wrapping the processing in a transaction and preserving errors ensures logs aren’t marked as processed prematurely, which aligns well with the PR objective of avoiding missed retries.


913-913: Test import usage
Bringing in the module with use super::*; is a standard approach for test scopes.


915-916: Additional test imports
These imports (anyhow, ethers, hex_literal, multiaddr, mock RPC) appear routine and pose no apparent issues.

Also applies to: 918-919, 921-922, 945-945


975-975: Mock RPC initialization in tests
Creating the mock RPC object helps isolate and validate chain interactions during testing.


993-993: Passing mock RPC into the handler
This ensures consistent, testable RPC usage in the test environment.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
chain/indexer/src/handlers.rs (1)

65-73: Debug implementation doesn't include rpc_operations field.

The Debug implementation doesn't display the rpc_operations field, which could be intentional but might hide useful information during debugging.

Consider including the rpc_operations field in the debug output:

-        f.debug_struct("ContractEventHandler")
-            .field("addresses", &self.addresses)
-            .field("safe_address", &self.safe_address)
-            .field("chain_key", &self.chain_key)
-            .finish_non_exhaustive()
+        f.debug_struct("ContractEventHandler")
+            .field("addresses", &self.addresses)
+            .field("safe_address", &self.safe_address)
+            .field("chain_key", &self.chain_key)
+            .field("rpc_operations", &"<RpcOperations>")
+            .finish_non_exhaustive()
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between ebfdfb2 and 85e7687.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (2)
  • chain/indexer/Cargo.toml (1 hunks)
  • chain/indexer/src/handlers.rs (16 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • chain/indexer/Cargo.toml
⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: tests-smoke-websocket
  • GitHub Check: Linter
🔇 Additional comments (18)
chain/indexer/src/handlers.rs (18)

5-5: Appropriate Debug implementation imports.

This addition properly supports the Debug trait implementation for the struct.


19-21: Good addition of RPC operation imports.

These imports support the new RPC interaction capabilities added to the handlers, aligning with the PR objective to reduce required logs by focusing on relevant events.


61-62: Appropriate new field for blockchain interaction.

Adding the rpc_operations field enables direct blockchain interactions, eliminating the need to process and store all token events locally.


79-92: Constructor updated correctly with new parameter.

The new constructor has been properly updated to accept and initialize the rpc_operations field.


194-198: Improved trace logging format.

The trace logging has been restructured for better readability and clarity.


221-225: Consistent logging structure.

This change maintains consistency with the improved trace logging format used elsewhere.


249-252: Improved logging with proper field formatting.

The trace logging has been updated to use a more consistent and readable format.


530-533: Enhanced error message clarity.

The error message has been improved to better explain the filter misconfiguration issue.


534-546: Implemented fetching balance from blockchain.

Instead of relying on the event data, the handler now fetches the actual balance from the blockchain, which is more accurate and resilient. This also helps reduce the required logs by only processing what's needed.


561-573: Improved allowance handling.

Similar to the balance handling, the code now fetches the current allowance from the blockchain instead of relying on event data. This improves accuracy and reduces dependency on processing all token events.


575-578: Improved error message for token approval events.

The error message now more clearly indicates the misconfiguration issue with approval events not involving the safe and channels contract.


890-919: Significant refactoring of log event collection.

The collect_log_event method has been refactored to process a single log event rather than a batch. This simplifies control flow and improves error handling by:

  1. Properly tracking transaction hash, log ID, and block ID
  2. Adding clear debug and error logging
  3. Ensuring that logs aren't marked as processed if an error occurs during processing

This implementation aligns with the PR objectives to enhance error handling and ensure logs aren't marked as processed on errors.


847-847: Added Debug trait bound to Db parameter.

The addition of the Debug trait bound is necessary to support the updated Debug implementation of ContractEventHandlers.


862-868: Added helpful accessor methods.

These methods provide convenient access to contract addresses map and safe address, improving the API's usability.


927-936: Test import updates.

The imports have been properly updated to align with the implementation changes.


957-958: Added RPC mock for testing.

The test now uses a MockRpcOperations, which allows proper testing of the RPC operation functionality.


987-992: Test initialization correctly updated.

The test initialization now includes creating a MockRpcOperations instance for the handler.


1005-1005: Updated struct initialization in tests.

Properly initializes the new rpc_operations field in the test handlers.

@tolbrino tolbrino modified the milestones: 2.2.3, 2.2.4 Mar 28, 2025
@github-actions github-actions bot added the stale label May 28, 2025
@Teebor-Choka
Copy link
Contributor

Retarget on master pls.

@tolbrino
Copy link
Contributor Author

tolbrino commented Jun 2, 2025

Replaced by #7213

@tolbrino tolbrino closed this Jun 2, 2025
@ausias-armesto ausias-armesto removed this from the 3.1.0 milestone Jul 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CT nodes might have insufficient FS access Inconsistent indexing between nodes
4 participants