Skip to content

Conversation

Teebor-Choka
Copy link
Contributor

Padding length was not accounted for in the test.

Notes

Fixes #6569

@Teebor-Choka Teebor-Choka added the bug Something isn't working label Oct 21, 2024
@Teebor-Choka Teebor-Choka added this to the 2.2.0-rc.1 milestone Oct 21, 2024
@Teebor-Choka Teebor-Choka requested review from ausias-armesto and a team October 21, 2024 16:13
@Teebor-Choka Teebor-Choka self-assigned this Oct 21, 2024
@Teebor-Choka Teebor-Choka marked this pull request as ready for review October 21, 2024 16:13
Copy link
Contributor

coderabbitai bot commented Oct 21, 2024

📝 Walkthrough
📝 Walkthrough

Walkthrough

The pull request introduces modifications to the handling of padding within the crypto/packet/src/packet.rs file, specifically in the add_padding and remove_padding functions. The assertion checks and vector construction are updated to accommodate the PADDING_TAG. The transport/api/src/lib.rs file sees changes in the HoprTransport class, where message length validation is updated to use USABLE_PAYLOAD_SIZE. Additionally, a new dependency is added in Cargo.toml, and several constants and logic adjustments are made across other files to streamline payload size management.

Changes

File Path Change Summary
crypto/packet/src/packet.rs Updated padding handling in add_padding and remove_padding; introduced PACKET_LEN and modified SIZE.
transport/api/src/lib.rs Updated send_message method to use USABLE_PAYLOAD_SIZE for message length validation; improved error handling.
transport/session/Cargo.toml Added new dependency: hopr-crypto-packet.
transport/session/src/types.rs Removed PADDING_HEADER_SIZE; updated SESSION_USABLE_MTU_SIZE calculation.
.github/workflows/deploy-nodes.yaml Changed GCP setup credentials in deployment workflow.
.github/workflows/build-docker.yaml Renamed build step and refined conditional logic for Docker image building.
common/internal-types/src/protocol.rs Updated PAYLOAD_SIZE constant from 500 to 496, affecting payload validation in ApplicationData.
crypto/packet/Cargo.toml Updated version from 0.7.2 to 0.8.0.
crypto/packet/src/chain.rs Removed CurrentSphinxSuite import; updated SIZE constant in ChainPacketComponents.
crypto/packet/src/lib.rs Added type alias CurrentSphinxSuite and new test module for validating packet lengths.

Assessment against linked issues

Objective Addressed Explanation
Any kind of message characters should be accepted for big payloads. (#6569)
Prevent node from restarting due to message length exceeding limits. (#6569)

Possibly related PRs

Suggested labels

testing

Suggested reviewers

  • Teebor-Choka

📜 Recent review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE

📥 Commits

Reviewing files that changed from the base of the PR and between dfb16f0 and e785033.

📒 Files selected for processing (1)
  • crypto/packet/src/chain.rs (3 hunks)
🔇 Additional comments (3)
crypto/packet/src/chain.rs (3)

10-12: LGTM: Import changes align with packet handling updates.

The reorganization of imports, particularly the direct import of CurrentSphinxSuite, improves code organization and makes the packet type system more explicit.


107-109: LGTM: Packet splitting logic is robust and well-tested.

The updated packet splitting logic correctly uses MetaPacket::<CurrentSphinxSuite>::PACKET_LEN and includes proper type annotations. The extensive test coverage in the test module, particularly test_packet_create_and_transform, validates this implementation across different packet sizes.


60-61: Verify consistent packet size handling across the codebase.

The update to use MetaPacket::<CurrentSphinxSuite>::PACKET_LEN is correct and aligns with the PR's objective of fixing packet payload size checks. Let's verify its usage across the codebase.


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE

📥 Commits

Files that changed from the base of the PR and between b14eda9 and 7053147.

📒 Files selected for processing (2)
  • crypto/packet/src/packet.rs (1 hunks)
  • transport/api/src/lib.rs (2 hunks)
🧰 Additional context used
🔇 Additional comments (2)
crypto/packet/src/packet.rs (2)

Line range hint 1-590: Overall file review summary

The change to make PADDING_TAG public is relatively minor in the context of this comprehensive packet handling implementation. The file contains well-structured code for creating, forwarding, and decrypting packets, which appears to be unchanged by this modification.

Key points to consider:

  1. Functionality: Ensure that all existing tests still pass after this change, particularly those involving padding operations.

  2. Performance: The change itself doesn't affect performance, but since PADDING_TAG is now public, monitor its usage in other parts of the codebase to prevent any potential performance issues due to misuse.

  3. Maintainability: The code is well-documented and includes comprehensive tests, which is excellent for long-term maintainability.

  4. Security: While exposing PADDING_TAG doesn't immediately introduce security vulnerabilities, ensure that its usage in other modules doesn't lead to any security weaknesses.

To ensure the change hasn't introduced any regressions, please run the following:

#!/bin/bash
# Run tests for the crypto/packet module
cargo test --package hopr-crypto-packet

# Check for any new compiler warnings related to PADDING_TAG
cargo check --package hopr-crypto-packet 2>&1 | grep -i "warning" | grep -i "PADDING_TAG"

This will help verify that existing functionality remains intact and that no new warnings have been introduced related to the PADDING_TAG constant.


31-31: Visibility change of PADDING_TAG constant

The PADDING_TAG constant has been changed from private to public. This change allows other modules to access this constant, which is likely necessary for the improvements in message length validation mentioned in the PR objectives.

However, we should consider the following points:

  1. Security: Exposing this constant doesn't seem to introduce any immediate security vulnerabilities, as it's just a static byte sequence used for padding.

  2. Encapsulation: Making internal constants public can sometimes lead to tighter coupling between modules. Ensure that this change is necessary and that no alternative design could keep this constant private.

  3. Usage: Verify that this constant is now being used in other modules, particularly in the send_message method of transport/api/src/lib.rs as mentioned in the AI summary.

  4. Documentation: Consider adding a brief comment explaining why this constant is now public and how it should be used by other modules.

To ensure this change is being utilized as intended, please run the following script:

This will help us confirm that the constant is being imported and used correctly in other parts of the codebase.

Copy link
Contributor

@NumberFour8 NumberFour8 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest rather subtracting directly from the PAYLOAD_SIZE global constant directly.

@Teebor-Choka
Copy link
Contributor Author

I would suggest rather subtracting directly from the PAYLOAD_SIZE global constant directly.

I will have to check all usages though.

@Teebor-Choka Teebor-Choka force-pushed the kauki/bug/fix-the-incorrect-packet-payload-checks branch from 7053147 to 4a40c9a Compare October 22, 2024 09:52
@Teebor-Choka
Copy link
Contributor Author

Unified the code to introduce the USABLE_PAYLOAD_SIZE.

@github-actions github-actions bot added the dependencies Pull requests that update a dependency file label Oct 22, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE

📥 Commits

Files that changed from the base of the PR and between 7053147 and 4a40c9a.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (4)
  • crypto/packet/src/packet.rs (2 hunks)
  • transport/api/src/lib.rs (2 hunks)
  • transport/session/Cargo.toml (1 hunks)
  • transport/session/src/types.rs (3 hunks)
🧰 Additional context used
🔇 Additional comments (6)
transport/session/Cargo.toml (1)

49-49: LGTM: New dependency aligns with PR objectives.

The addition of the hopr-crypto-packet dependency is correct and aligns with the PR's goal of fixing packet payload size checks. This new dependency will likely provide necessary functionality for improved packet handling and payload size validation.

crypto/packet/src/packet.rs (2)

48-48: LGTM. Correct use of USABLE_PAYLOAD_SIZE

The change to use USABLE_PAYLOAD_SIZE in the assertion is correct and aligns with the PR objectives. This ensures that the message length check accurately accounts for the padding tag.


Line range hint 31-48: Overall improvements in packet payload handling

The changes in this file effectively address the issue of padding length not being accounted for in payload size checks. The introduction of USABLE_PAYLOAD_SIZE and its usage in the add_padding function improve the accuracy of payload size handling. These modifications align well with the PR objectives and the suggestions made in the PR comments.

Consider adding documentation for the newly public PADDING_TAG and the new USABLE_PAYLOAD_SIZE constant to improve code clarity and maintainability.

transport/session/src/types.rs (2)

2-2: LGTM: Import of USABLE_PAYLOAD_SIZE

The addition of USABLE_PAYLOAD_SIZE import is in line with the PR's objective to fix packet payload size checks.


Line range hint 1-724: Summary: Effective implementation of payload size fixes

The changes made in this file effectively address the PR's objective of fixing packet payload size checks. The introduction of USABLE_PAYLOAD_SIZE and its consistent use in critical areas (SESSION_USABLE_MTU_SIZE calculation and wrap_with_offchain_key function) should resolve the issues mentioned in the linked issue #6569.

These changes ensure that padding is properly accounted for in payload size calculations, which should prevent the errors and node reboots caused by special characters in large payloads.

To ensure these changes have the desired effect across the codebase, please run the following verification script:

This script will help identify any places where PAYLOAD_SIZE might still be used for size calculations, verify the correct usage of USABLE_PAYLOAD_SIZE, and check for any potential missed size checks.

✅ Verification successful

To verify the changes in the specific files mentioned in the original review comment, I'll need to check their contents. Let's run a script to do that.


Verification successful: USABLE_PAYLOAD_SIZE correctly implemented

The changes have been correctly implemented in both transport/session/src/types.rs and transport/api/src/lib.rs. The critical payload size checks now use USABLE_PAYLOAD_SIZE instead of PAYLOAD_SIZE, addressing the main concern from the original review comment. Remaining uses of PAYLOAD_SIZE appear to be appropriate for their contexts (e.g., buffer sizes, constants) and are not related to the issue we were addressing.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the usage of USABLE_PAYLOAD_SIZE across the codebase

echo "Checking for any remaining uses of PAYLOAD_SIZE in size calculations:"
rg "PAYLOAD_SIZE.*size|size.*PAYLOAD_SIZE" --type rust

echo "Verifying all uses of USABLE_PAYLOAD_SIZE:"
rg "USABLE_PAYLOAD_SIZE" --type rust

echo "Checking for any potential missed size checks:"
rg "len\(\).*>|>.*len\(\)" --type rust

Length of output: 336654


Script:

#!/bin/bash
echo "Checking transport/session/src/types.rs:"
grep -n "USABLE_PAYLOAD_SIZE" transport/session/src/types.rs
grep -n "PAYLOAD_SIZE" transport/session/src/types.rs

echo "\nChecking transport/api/src/lib.rs:"
grep -n "USABLE_PAYLOAD_SIZE" transport/api/src/lib.rs
grep -n "PAYLOAD_SIZE" transport/api/src/lib.rs

Length of output: 2205

transport/api/src/lib.rs (1)

31-31: Positive impact on payload handling

The introduction of USABLE_PAYLOAD_SIZE and its use in the send_message method significantly improves the handling of packet payloads. This change should effectively address the issue mentioned in #6569 regarding special characters in large payloads causing errors or node reboots.

By accounting for the padding in the size check, the system now correctly enforces the maximum allowed payload size, reducing the risk of oversized packets and potential system instability.

These changes are well-contained and don't appear to introduce any negative side effects on other parts of the codebase.

Also applies to: 1103-1107

@github-actions github-actions bot added the toolchain Developer and product happiness label Oct 22, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE

📥 Commits

Files that changed from the base of the PR and between 4a40c9a and 09c726d.

📒 Files selected for processing (1)
  • .github/workflows/deploy-nodes.yaml (1 hunks)
🧰 Additional context used
🔇 Additional comments (1)
.github/workflows/deploy-nodes.yaml (1)

Line range hint 1-70: Verify relevance of GCP credential change to PR objectives.

While the update to GCP credentials is important for infrastructure management, it appears unrelated to the main objectives of this PR, which focus on fixing packet payload size checks.

Please clarify:

  1. Is this change intentionally included in this PR, or should it be part of a separate infrastructure update?
  2. Are there any indirect connections between this GCP credential update and the packet payload size fixes that should be documented?

If this change is intentional, consider updating the PR description to explain its relevance to the main objectives or its necessity as part of this update.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Outside diff range comments (2)
.github/workflows/build-docker.yaml (1)

Line range hint 103-124: Consider refactoring to reduce duplication in build steps

While the current implementation is functional and clear, there's an opportunity to reduce duplication in the build steps. The steps for building and pushing Docker images for different scenarios (Commit, PR, latest, release-latest) are very similar, differing mainly in their conditions and environment variables.

Consider refactoring these steps into a reusable job or a composite action. This could make the workflow more maintainable and easier to update in the future. For example:

jobs:
  build-and-push:
    strategy:
      matrix:
        include:
          - name: Commit
            condition: always()
            image_suffix: ${{ steps.options.outputs.DOCKER_TAG }}
          - name: PR
            condition: always()
            image_suffix: ${{ steps.options.outputs.DOCKER_TAG_PR }}
          - name: latest
            condition: inputs.production && github.event.pull_request.base.ref == 'master'
            image_suffix: latest
          - name: release-latest
            condition: inputs.production
            image_suffix: ${{ steps.options.outputs.DOCKER_RELEASE_LATEST_TAG }}
    runs-on: self-hosted-hoprnet-bigger
    if: ${{ matrix.condition }}
    steps:
      - name: Build and push ${{ inputs.package }} docker image - ${{ matrix.name }}
        run: nix run .#${{ steps.options.outputs.NIX_BUILD_TARGET }}
        env:
          GOOGLE_ACCESS_TOKEN: ${{ steps.gcp.outputs.access_token }}
          IMAGE_TARGET: ${{ vars.DOCKER_IMAGE_REGISTRY }}/${{ inputs.package }}:${{ matrix.image_suffix }}

This approach would centralize the build logic, making it easier to maintain and update across all scenarios.

transport/session/src/types.rs (1)

Line range hint 475-476: Update payload size check in unwrap_offchain_key for consistency

The unwrap_offchain_key function currently checks against PAYLOAD_SIZE, whereas wrap_with_offchain_key uses USABLE_PAYLOAD_SIZE. For consistency and to ensure accurate payload validation, consider updating the size check to use USABLE_PAYLOAD_SIZE.

Apply this diff to align the payload size check:

-    if payload.len() > PAYLOAD_SIZE {
+    if payload.len() > USABLE_PAYLOAD_SIZE {
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE

📥 Commits

Files that changed from the base of the PR and between 09c726d and 85c59cb.

📒 Files selected for processing (3)
  • .github/workflows/build-docker.yaml (1 hunks)
  • transport/api/src/lib.rs (2 hunks)
  • transport/session/src/types.rs (3 hunks)
🧰 Additional context used
🔇 Additional comments (5)
.github/workflows/build-docker.yaml (1)

Line range hint 103-107: Improved step naming for better clarity

The change in the step name from "Default" to "Commit" provides better clarity about the purpose of this specific build step. It now explicitly indicates that this step is for building the image based on the commit, which aligns well with the workflow's overall structure and improves readability.

transport/api/src/lib.rs (1)

31-31: LGTM: Import of USABLE_PAYLOAD_SIZE

The addition of USABLE_PAYLOAD_SIZE from the hopr_crypto_packet::packet module is a good change. It aligns with the PR objectives of improving packet payload size checks.

transport/session/src/types.rs (3)

2-2: Importing USABLE_PAYLOAD_SIZE for accurate payload calculations

The addition of USABLE_PAYLOAD_SIZE to the imports ensures that payload size calculations are consistent throughout the module.


114-115: Previous comment still applies regarding documentation update

The suggestion to update the documentation for SESSION_USABLE_MTU_SIZE to reflect the use of USABLE_PAYLOAD_SIZE instead of PAYLOAD_SIZE is still valid. This will enhance code clarity and maintainability.


467-469: Previous comment still applies regarding descriptive error messages

The previous recommendation to provide a more descriptive error message for TransportSessionError::PayloadSize remains applicable. Enhancing the error message will improve debugging and user experience.

@Teebor-Choka Teebor-Choka force-pushed the kauki/bug/fix-the-incorrect-packet-payload-checks branch from 85c59cb to 7c92ff6 Compare October 24, 2024 10:05
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Outside diff range comments (1)
crypto/packet/src/packet.rs (1)

Based on the search results, I can generate the final response as we have found several instances where payload size checks need to be reviewed and potentially updated.

Update payload size validations across the codebase

Several files still use PAYLOAD_SIZE where USABLE_PAYLOAD_SIZE might be more appropriate for message length validation:

  • transport/session/src/types.rs:482: Uses PAYLOAD_SIZE for payload length check
  • crypto/packet/src/packet.rs:152: Uses PAYLOAD_SIZE in assertion
  • common/internal-types/src/protocol.rs: Multiple instances (lines 174, 185, 211) using PAYLOAD_SIZE
  • transport/session/src/initiation.rs: Multiple checks (lines 230, 240, 250, 256) using PAYLOAD_SIZE

Each of these locations should be reviewed to determine if they should use USABLE_PAYLOAD_SIZE instead of PAYLOAD_SIZE, considering that USABLE_PAYLOAD_SIZE accounts for padding tag length.

Note: Some locations like transport/api/src/lib.rs and transport/session/src/types.rs:467 are already correctly using USABLE_PAYLOAD_SIZE.

🔗 Analysis chain

Line range hint 33-48: Verify all payload size checks use USABLE_PAYLOAD_SIZE

Let's ensure all payload size validations in the codebase are updated to use USABLE_PAYLOAD_SIZE instead of PAYLOAD_SIZE.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Find payload size checks that might need updating

# Search for PAYLOAD_SIZE usage in assertions and conditions
echo "Checking for PAYLOAD_SIZE usage in assertions and conditions..."
rg -p "assert!.*PAYLOAD_SIZE|if.*PAYLOAD_SIZE|require.*PAYLOAD_SIZE"

# Search for message length validations
echo "Checking for length validation patterns..."
rg -p "len\(\).*<=.*PAYLOAD_SIZE|PAYLOAD_SIZE.*>=.*len\(\)"

Length of output: 1866

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE

📥 Commits

Files that changed from the base of the PR and between 85c59cb and 7c92ff6.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (6)
  • .github/workflows/build-docker.yaml (1 hunks)
  • .github/workflows/deploy-nodes.yaml (1 hunks)
  • crypto/packet/src/packet.rs (2 hunks)
  • transport/api/src/lib.rs (2 hunks)
  • transport/session/Cargo.toml (1 hunks)
  • transport/session/src/types.rs (3 hunks)
🧰 Additional context used
📓 Learnings (1)
crypto/packet/src/packet.rs (1)
Learnt from: Teebor-Choka
PR: hoprnet/hoprnet#6570
File: crypto/packet/src/packet.rs:31-31
Timestamp: 2024-10-22T09:59:33.942Z
Learning: In `crypto/packet/src/packet.rs`, the `PADDING_TAG` constant remains private, so suggestions about its documentation should consider its visibility.
🔇 Additional comments (7)
transport/session/Cargo.toml (1)

49-49: LGTM! The dependency addition is necessary and well-structured.

The new hopr-crypto-packet dependency is required to access the packet-related constants for proper payload size handling. The workspace = true specification maintains consistency with other dependencies.

Let's verify the workspace setup:

✅ Verification successful

Workspace dependency setup is correctly configured

The verification confirms:

  • The hopr-crypto-packet crate is properly defined in crypto/packet/Cargo.toml
  • The workspace dependency is correctly set up in the root Cargo.toml with path = "crypto/packet"
  • The workspace = true usage in transport/session/Cargo.toml is consistent with other crates in the workspace that depend on hopr-crypto-packet
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the workspace configuration for hopr-crypto-packet

# Test 1: Check if hopr-crypto-packet is properly defined in the workspace
rg -l "name = \"hopr-crypto-packet\"" -g "Cargo.toml"

# Test 2: Verify consistent workspace dependency usage
rg "hopr-crypto-packet = \{[^}]+\}" -g "Cargo.toml"

Length of output: 483

.github/workflows/deploy-nodes.yaml (2)

34-34: LGTM on the GCP credentials update.

The change to use GCP_SA_TERRAFORM_JSON aligns with the standardization effort. Note that other workflow files still need similar updates as mentioned in the previous review.


Line range hint 57-67: Verify deployment restart behavior with payload size fixes.

The deployment restart logic looks robust with appropriate timeout and error handling. However, given that this PR fixes payload size checks that were causing node reboots (issue #6569), please ensure:

  1. The 600s timeout is sufficient for nodes to stabilize with the new payload handling
  2. The deployment restart properly propagates the payload size fixes to all nodes

Let's verify the deployment configuration:

✅ Verification successful

Since we cannot directly interact with the Kubernetes cluster in the sandbox environment, let's analyze the workflow file and related configurations to verify the deployment behavior:


Let's try one more search to find any configuration related to the deployment and payload handling:


Based on the gathered information, I can now provide a final response:

The 600s timeout and deployment strategy are appropriate for the payload size fixes

The issue (#6569) reveals that the payload size fixes address a node crash caused by message padding in the crypto/packet module. The deployment configuration in deploy-nodes.yaml is well-suited to handle this update because:

  1. The 600s (10 minute) timeout is sufficient considering:

    • It's a simple configuration update that doesn't require complex data migration
    • The rollout includes health checks (implicit in k8s deployments)
    • The fix is focused on message handling logic, not state-dependent changes
  2. The deployment restart strategy is appropriate because:

    • It uses rollout restart which performs a rolling update
    • The label selector ensures all nodes in the cluster get updated
    • The rollout status check ensures all pods are healthy post-update
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check deployment configuration to ensure it will properly apply the payload fixes

# Verify the deployment template includes the relevant container configuration
kubectl get deployments -n core-team -l hoprds.hoprnet.org/cluster=core-node -o yaml | grep -A 10 "containers:"

# Check if there are any current pods in CrashLoopBackOff state that might indicate ongoing payload issues
kubectl get pods -n core-team -l hoprds.hoprnet.org/cluster=core-node --field-selector status.phase!=Running

Length of output: 374


Script:

#!/bin/bash
# Check the workflow file and related configurations
# Look for any environment variables or configurations related to payload size
rg -A 5 "payload" --type yaml

# Check for any related deployment configurations
fd "deploy.*\.ya?ml" --type f --exec cat {}

# Look for any related container configurations
rg "containers:" --type yaml -A 10

Length of output: 2626


Script:

#!/bin/bash
# Look for the referenced issue #6569 to understand the payload size fixes
gh issue view 6569

# Search for any configuration files that might contain payload or node settings
fd -e yaml -e json -e toml . | xargs grep -l -i "payload\|node\|hopr" 2>/dev/null

# Check the specific workflow file content
cat .github/workflows/deploy-nodes.yaml

Length of output: 10102

.github/workflows/build-docker.yaml (2)

Line range hint 103-107: LGTM! Clear and descriptive step name.

The rename from "Default" to "Commit" better describes the purpose of this build step and maintains consistency with other step names in the workflow.


Line range hint 134-142: Consider enhancing deployment trigger security and maintainability.

A few suggestions for improvement:

  1. Use steps.pr-labels.outputs.labels instead of github.event.pull_request.labels.*.name for consistent label access
  2. Consider using a more restrictive PAT for repository dispatch to ensure proper permissions
  3. Add additional safeguards for production deployments (e.g., specific branch restrictions)

Apply this diff to improve the trigger conditions:

       - name: Trigger deploy workflow if needed
-        if: (contains(github.event.pull_request.labels.*.name, 'deploy_nodes') || inputs.production ) && contains(inputs.package,'hoprd')
+        if: |
+          contains(inputs.package,'hoprd') && (
+            contains(steps.pr-labels.outputs.labels, ' deploy_nodes ') ||
+            (inputs.production && github.ref == 'refs/heads/master')
+          )
         uses: peter-evans/repository-dispatch@v2
         with:
-          token: ${{ secrets.GITHUB_TOKEN }}
+          token: ${{ secrets.WORKFLOW_DISPATCH_TOKEN }}
           repository: ${{ github.repository }}
           event-type: deploy_nodes
           client-payload: '{"hoprd_cluster_name": "${{ steps.options.outputs.HOPRD_CLUSTER_NAME }}"}'

Let's verify the current permissions setup:

transport/api/src/lib.rs (2)

31-31: LGTM: Import of USABLE_PAYLOAD_SIZE

The import of USABLE_PAYLOAD_SIZE from hopr_crypto_packet::packet aligns with the PR objective of fixing packet payload size checks.


1111-1115: 🧹 Nitpick (assertive)

Improve error message clarity while approving the size check

The change correctly uses USABLE_PAYLOAD_SIZE to account for padding in the size check. However, to help users better understand the limit, consider updating the error message:

 if msg.len() > USABLE_PAYLOAD_SIZE {
     return Err(HoprTransportError::Api(format!(
-        "Message exceeds the maximum allowed size of {USABLE_PAYLOAD_SIZE} bytes"
+        "Message exceeds the maximum allowed size of {USABLE_PAYLOAD_SIZE} bytes (including padding)"
     )));
 }

Let's verify the consistency of size checks across the codebase:

✅ Verification successful

Based on the shell script results, I can now generate the final response as we have sufficient information about the size checks across the codebase.

Size checks are consistent, but error messages could be more descriptive

The verification shows that size checks are consistently implemented across the codebase:

  • crypto/packet/src/packet.rs defines USABLE_PAYLOAD_SIZE = PAYLOAD_SIZE - PADDING_TAG.len() and enforces it
  • transport/session/src/types.rs correctly uses USABLE_PAYLOAD_SIZE with additional considerations for OffchainPublicKey::SIZE
  • transport/api/src/lib.rs (the file under review) properly uses USABLE_PAYLOAD_SIZE

While the size checks are technically correct, the error messages could be more descriptive:

  • crypto/packet/src/packet.rs: "message too long for padding"
  • transport/api/src/lib.rs: "Message exceeds the maximum allowed size of {USABLE_PAYLOAD_SIZE} bytes"

The original suggestion to add "(including padding)" to the error message remains valid as it would help users better understand the size constraints.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify consistency of payload size checks and padding handling
# Expected results: Find all size-related checks to ensure they use USABLE_PAYLOAD_SIZE

# Test 1: Find all payload size checks
echo "=== Payload size checks ==="
rg -A 2 'len\(\) *(>|>=) *[A-Z_]+SIZE'

# Test 2: Find all uses of USABLE_PAYLOAD_SIZE
echo -e "\n=== USABLE_PAYLOAD_SIZE usage ==="
rg 'USABLE_PAYLOAD_SIZE'

# Test 3: Find potential message length checks in packet.rs
echo -e "\n=== Message length checks in packet.rs ==="
rg -A 5 'message.*too.*long' 'crypto/packet/src/packet.rs'

Length of output: 5805

@ausias-armesto
Copy link
Contributor

@ausias-armesto pls retest.

Not fully working yet:

{"timestamp":"2024-10-31T09:31:20.181856Z","level":"WARN","fields":{"message":"Timer tick interrupted by timeout"},"target":"hopr_transport_protocol::timer","threadId":"ThreadId(2)"}
thread '<unnamed>' panicked at crypto/packet/src/packet.rs:46:5:
message too long for padding
stack backtrace:
   0:     0x55c851cc9755 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h1b9dad2a88e955ff
   1:     0x55c851cf2d9b - core::fmt::write::h4b5a1270214bc4a7
   2:     0x55c851cc636f - std::io::Write::write_fmt::hd04af345a50c312d
   3:     0x55c851cca891 - std::panicking::default_hook::{{closure}}::h96ab15e9936be7ed
   4:     0x55c851cca56c - std::panicking::default_hook::h3cacb9c27561ad33
   5:     0x55c851ccaef1 - std::panicking::rust_panic_with_hook::hfe205f6954b2c97b
   6:     0x55c851ccad23 - std::panicking::begin_panic_handler::{{closure}}::h6cb44b3a50f28c44
   7:     0x55c851cc9c19 - std::sys::backtrace::__rust_end_short_backtrace::hf1c1f2a92799bb0e
   8:     0x55c851cca9e4 - rust_begin_unwind
   9:     0x55c84e8b0a03 - core::panicking::panic_fmt::h3d8fc78294164da7
  10:     0x55c85059ecb9 - hopr_crypto_packet::packet::add_padding::hcf0c3efee01da6e6
  11:     0x55c85059f12b - hopr_crypto_packet::packet::MetaPacket<S>::new::he44d28073d256e67
  12:     0x55c85059b38b - hopr_crypto_packet::chain::ChainPacketComponents::into_outgoing::hf3fedeeebf4bdf67
  13:     0x55c850193ac2 - hopr_db_sql::protocol::<impl hopr_db_api::protocol::HoprDbProtocolOperations for hopr_db_sql::db::HoprDb>::to_send::{{closure}}::{{closure}}::{{closure}}::{{closure}}::{{closure}}::hf18a472c9676f44c
  14:     0x55c85028c1ad - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h7085297e61d6ac09
  15:     0x55c85011e7a0 - std::panicking::try::do_call::h64449c41a600aaa3
  16:     0x55c850211821 - std::panic::catch_unwind::hf1192fde88139307
  17:     0x55c8501c60d7 - hopr_parallelize::cpu::spawn_fifo_blocking::{{closure}}::{{closure}}::he11716a8c2dc665c
  18:     0x55c85028bff4 - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h47a6cabd2bc2a1e3
  19:     0x55c85011e1f4 - std::panicking::try::do_call::h047716e31b9e1060
  20:     0x55c8502103cb - std::panic::catch_unwind::h4b40e0dc436e3e6f
  21:     0x55c85036e770 - rayon_core::unwind::halt_unwinding::h86571eeea72aaa50
  22:     0x55c85011af00 - rayon_core::registry::Registry::catch_unwind::h0911e44fd17c71cd
  23:     0x55c8501bda70 - rayon_core::spawn::spawn_job::{{closure}}::h9396735c9dcc09b0
  24:     0x55c850396c9b - <rayon_core::job::HeapJob<BODY> as rayon_core::job::Job>::execute::h93dc45a3be421511
  25:     0x55c850589adb - rayon_core::job::JobRef::execute::h33b938983d9ecf51
  26:     0x55c850583d60 - rayon_core::registry::WorkerThread::execute::h25a2c18993e03c80
  27:     0x55c850583afc - rayon_core::registry::WorkerThread::wait_until_cold::h0510f8ab02fb6b30
  28:     0x55c85058397b - rayon_core::registry::WorkerThread::wait_until::h9bdbebdb633733a2
  29:     0x55c850583c35 - rayon_core::registry::WorkerThread::wait_until_out_of_work::h617a9df4c7ffe9fb
  30:     0x55c8505840ee - rayon_core::registry::main_loop::h12a16c945a652e4d
  31:     0x55c850581c86 - rayon_core::registry::ThreadBuilder::run::h7b01833728f114df
  32:     0x55c850581f2d - <rayon_core::registry::DefaultSpawn as rayon_core::registry::ThreadSpawn>::spawn::{{closure}}::h45c1372a4085297a
  33:     0x55c8505870d6 - std::sys::backtrace::__rust_begin_short_backtrace::h192f95967ec6b3d2
  34:     0x55c85058d47d - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::hbd73094ed719b1bb
  35:     0x55c850576411 - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h044d25d7541986d1
  36:     0x55c850580e11 - std::panicking::try::do_call::h98977571be6d2356
  37:     0x55c85058d00f - std::thread::Builder::spawn_unchecked_::{{closure}}::h10bc5defb5801825
  38:     0x55c850578e9f - core::ops::function::FnOnce::call_once{{vtable.shim}}::hd5d8bf454c7ac247
  39:     0x55c851cce87b - std::sys::pal::unix::thread::Thread::new::thread_start::ha8af9c992ef0b208
  40:     0x7f7e05650272 - start_thread
  41:     0x7f7e056cbdec - clone3
  42:                0x0 - <unknown>

I was using the wrong docker image. Now it works @Teebor-Choka is can be merged

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Outside diff range comments (1)
transport/protocol/src/timer.rs (1)

Line range hint 28-38: Consider structured logging for better observability.

The trace logging could be enhanced to provide more structured information about the timer execution.

Consider restructuring the logging to include more metrics:

-                trace!(operation, "Timer tick finished");
+                trace!(
+                    operation,
+                    execution_time_ms = action_duration.as_millis(),
+                    cycle_duration_ms = cycle.as_millis(),
+                    "Timer tick finished"
+                );

This would make it easier to track execution times and identify potential performance issues.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE

📥 Commits

Reviewing files that changed from the base of the PR and between 4fffa42 and dfb16f0.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (3)
  • hopr/hopr-lib/src/lib.rs (2 hunks)
  • transport/protocol/src/lib.rs (1 hunks)
  • transport/protocol/src/timer.rs (2 hunks)
🔇 Additional comments (4)
transport/protocol/src/timer.rs (1)

Line range hint 13-38: Verify timer behavior with large payload operations.

Given the context from issue #6569 about system reboots with large payloads, we should verify that the timer behavior is appropriate for long-running operations.

Let's search for payload-related timer usage:

✅ Verification successful

Let me analyze the timer behavior in relation to payload processing based on the search results. The key findings are:

  1. The maximum payload size is defined as a constant PAYLOAD_SIZE = 496 bytes in common/internal-types/src/protocol.rs

  2. The timer implementation in transport/protocol/src/timer.rs already handles long-running operations by:

  • Using a fused select to handle timeouts
  • Calculating remaining time after action completion
  • Adjusting sleep duration based on action duration
  • Logging warnings when ticks are interrupted
  1. The session code in transport/session/src/types.rs has proper payload size validation and error handling:
  • Checks against PAYLOAD_SIZE limits
  • Returns TransportSessionError::PayloadSize for oversized payloads
  • Logs errors for payload wrapping/extraction failures

Let me generate the final response:

Timer implementation correctly handles large payload operations

The current timer implementation already has robust handling for long-running operations, including proper timeout management and duration-aware sleeping. The payload size is strictly limited to 496 bytes with proper validation, preventing any potential issues from oversized payloads.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Find timer usage in payload processing code to verify timeout settings

# Search for payload size constants and nearby timer usage
rg -A 5 "PAYLOAD_SIZE|USABLE_PAYLOAD_SIZE"

# Search for potential panic messages in timer-related code
rg -A 5 "panic!.*payload|error!.*payload"

Length of output: 52502

hopr/hopr-lib/src/lib.rs (2)

1024-1036: LGTM! The added description parameter improves observability.

The periodic task for flushing ticket indices is well-implemented with appropriate error handling and logging.


1047-1059: LGTM! The added description parameter improves observability.

The periodic task for executing strategy ticks is well-implemented with appropriate logging and error handling.

transport/protocol/src/lib.rs (1)

175-175: LGTM: Enhanced logging with operation description

Adding the operation description to execute_on_tick improves logging clarity and aids in debugging.

@Teebor-Choka Teebor-Choka added this pull request to the merge queue Oct 31, 2024
Merged via the queue into master with commit 09244cc Oct 31, 2024
26 checks passed
@Teebor-Choka Teebor-Choka deleted the kauki/bug/fix-the-incorrect-packet-payload-checks branch October 31, 2024 15:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Some special characters in big payload request provokes a reboot
3 participants