Remove the python-sdk auto-generation
#6772
Conversation
📝 WalkthroughWalkthroughThis pull request involves the removal and modification of SDK generation infrastructure for the HOPR project. The changes include deleting the GitHub Actions workflow for SDK generation, removing the Makefile target for Python SDK generation, updating the SDK generation script, and modifying the README documentation. The primary focus is on streamlining and simplifying the process of generating Python SDKs for the HOPRd API. Changes
Possibly related PRs
Suggested Labels
Suggested Reviewers
📜 Recent review detailsConfiguration used: .coderabbit.yaml 📒 Files selected for processing (1)
🧰 Additional context used📓 Learnings (1)scripts/generate-python-sdk.sh (1)🪛 Shellcheck (0.10.0)scripts/generate-python-sdk.sh[warning] 22-22: Use "${var:?}" to ensure this never expands to /* . (SC2115) ⏰ Context from checks skipped due to timeout of 90000ms (3)
🔇 Additional comments (8)
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
Done. |
There was a problem hiding this comment.
Actionable comments posted: 3
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
README.md(1 hunks)scripts/generate-python-sdk.sh(1 hunks)
🧰 Additional context used
🪛 Shellcheck (0.10.0)
scripts/generate-python-sdk.sh
[error] 1-1: Tips depend on target shell and yours is unknown. Add a shebang or a 'shell' directive.
(SC2148)
[warning] 1-1: Quote this to prevent word splitting.
(SC2046)
🪛 LanguageTool
README.md
[typographical] ~340-~340: The word “however” is an adverb that can’t be used like a conjunction, and therefore needs to be separated from the sentence.
Context: ...is available to connect to the HOPRd API, however it can be generated by using this [scri...
(HOWEVER_SENTENCE)
⏰ Context from checks skipped due to timeout of 90000ms (7)
- GitHub Check: hopli / docker
- GitHub Check: Docs / Rust docs
- GitHub Check: tests-unit-nightly
- GitHub Check: tests-smart-contracts
- GitHub Check: tests-unit
- GitHub Check: tests-smoke-websocket
- GitHub Check: Linter
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
hoprd/hoprd/example_cfg.yaml (1)
180-184: Document rationale for aggressive heartbeat settings.The configuration changes align with the updates in other files, but the example configuration would benefit from additional comments explaining:
- The reasoning behind the more aggressive probing strategy
- Potential resource implications of increased parallel probes
- Considerations for different network conditions
Consider adding these explanatory comments to the configuration:
heartbeat: # The maximum number of concurrent heartbeat probes + # Note: Higher values increase responsiveness but also increase resource usage max_parallel_probes: 50 # Interval in which the heartbeat is triggered in seconds interval: 60 # The time interval for which to consider peer heartbeat renewal in seconds + # Note: Lower threshold values make the network more responsive but may increase churn threshold: 40Also applies to: 227-227
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (5)
hoprd/hoprd/example_cfg.yaml(2 hunks)transport/network/Cargo.toml(1 hunks)transport/network/src/constants.rs(1 hunks)transport/protocol/Cargo.toml(1 hunks)transport/protocol/src/heartbeat/config.rs(1 hunks)
✅ Files skipped from review due to trivial changes (2)
- transport/protocol/Cargo.toml
- transport/network/Cargo.toml
🔇 Additional comments (3)
transport/protocol/src/heartbeat/config.rs (1)
13-13: Verify impact of reduced heartbeat timeout on network reliability.The reduction of the heartbeat timeout from 15 to 7 seconds is significant. While this makes the heartbeat mechanism more responsive, it could potentially affect:
- Nodes in high-latency networks
- Nodes behind NAT
- Nodes in regions with poor connectivity
Run this script to analyze historical network latencies and identify potential impact:
✅ Verification successful
Heartbeat timeout reduction to 7 seconds is appropriate for the system design
The reduced timeout aligns well with the system's other timing parameters:
- Fits within the 40-second heartbeat threshold for node availability checks
- Maintains good balance with the 10-second message acknowledgment timeout
- Provides faster failure detection while still being conservative enough for typical network conditions
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Check for latency-related issues in logs # Look for timeout patterns or latency issues that might be affected by the reduced timeout rg -i "timeout|latency|heartbeat.*fail"Length of output: 65994
Script:
#!/bin/bash # Search for heartbeat protocol implementation and related code echo "=== Heartbeat Protocol Implementation ===" ast-grep --pattern 'struct $name { $$$ timeout: $_ $$$ }' echo -e "\n=== Heartbeat Usage and Network Metrics ===" rg -B2 -A2 "heartbeat.*timeout|network.*latency" --type rust echo -e "\n=== Related Configuration Parameters ===" rg -B1 -A1 "Duration::from_secs|TIMEOUT" transport/Length of output: 12819
transport/network/src/constants.rs (2)
6-6: Consider network impact of reduced heartbeat threshold.Reducing the heartbeat threshold from 60 to 40 seconds makes the network more responsive but could increase the frequency of peer availability checks.
Consider implementing a gradual rollout or monitoring system to track the impact of this change on network load.
13-13: Verify network capacity for increased parallel pings.The significant increase in DEFAULT_MAX_PARALLEL_PINGS (14 → 50) could impact:
- Network bandwidth consumption
- Node resource utilization
- Connection pool limits
Run this script to check for connection-related issues:
✅ Verification successful
Change is safe due to enforced hard limit of 14 parallel pings
The increase in DEFAULT_MAX_PARALLEL_PINGS to 50 does not impact system stability because:
- The Pinger implementation enforces a hard limit of 14 parallel pings (MAX_PARALLEL_PINGS)
- The actual number of parallel pings is capped using: config.max_parallel_pings.min(MAX_PARALLEL_PINGS)
- The system implements proper backpressure and resource management through chunked stream processing
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Search for connection pool or resource exhaustion patterns # Look for evidence of connection-related issues that might be exacerbated rg -i "connection.*limit|too many.*connect|resource.*exhaust"Length of output: 89302
Script:
#!/bin/bash # Search for usage of DEFAULT_MAX_PARALLEL_PINGS and ping-related code # Exclude vendor directories to focus on project code rg -g '!vendor' "DEFAULT_MAX_PARALLEL_PINGS|parallel.*ping" -A 5 -B 5 # Also search for ping configuration and limits rg -g '!vendor' "ping.*config|max.*ping|ping.*limit" -A 3 -B 3Length of output: 18983
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
scripts/generate-python-sdk.sh (1)
1-37: Add usage documentation to README.md.Since this script replaces the auto-generation workflow, please add documentation to README.md explaining:
- The deprecation of auto-generated Python SDK
- Prerequisites (swagger-codegen3, hoprd-api-schema)
- Usage examples with environment variables
Would you like me to help draft the documentation for README.md?
🧰 Tools
🪛 Shellcheck (0.10.0)
[warning] 22-22: Use "${var:?}" to ensure this never expands to /* .
(SC2115)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
README.md(2 hunks)scripts/generate-python-sdk.sh(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- README.md
🧰 Additional context used
📓 Learnings (1)
scripts/generate-python-sdk.sh (1)
Learnt from: Teebor-Choka
PR: hoprnet/hoprnet#6772
File: scripts/generate-python-sdk.sh:1-10
Timestamp: 2025-01-15T16:14:10.492Z
Learning: In shell scripts, make paths configurable by using environment variables with default values using parameter expansion (e.g., `${VAR:-default}`). This allows users to override the default behavior without modifying the script.
🪛 Shellcheck (0.10.0)
scripts/generate-python-sdk.sh
[warning] 22-22: Use "${var:?}" to ensure this never expands to /* .
(SC2115)
⏰ Context from checks skipped due to timeout of 90000ms (8)
- GitHub Check: hoprd / docker
- GitHub Check: hopli / docker
- GitHub Check: tests-smoke-websocket
- GitHub Check: Docs / Rust docs
- GitHub Check: tests-unit
- GitHub Check: tests-unit-nightly
- GitHub Check: tests-smart-contracts
- GitHub Check: Linter
🔇 Additional comments (2)
scripts/generate-python-sdk.sh (2)
1-5: LGTM! Well-structured script setup.The script follows shell scripting best practices with proper error handling and configurable paths.
27-35: LGTM! Proper SDK generation with error handling.The SDK generation segment is well-implemented with proper error handling and path quoting.
python-sdk auto-generationpython-sdk auto-generation
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
scripts/generate-python-sdk.sh(1 hunks)
🧰 Additional context used
📓 Learnings (1)
scripts/generate-python-sdk.sh (1)
Learnt from: Teebor-Choka
PR: hoprnet/hoprnet#6772
File: scripts/generate-python-sdk.sh:1-10
Timestamp: 2025-01-15T16:14:10.492Z
Learning: In shell scripts, make paths configurable by using environment variables with default values using parameter expansion (e.g., `${VAR:-default}`). This allows users to override the default behavior without modifying the script.
🪛 Shellcheck (0.10.0)
scripts/generate-python-sdk.sh
[warning] 22-22: Use "${var:?}" to ensure this never expands to /* .
(SC2115)
⏰ Context from checks skipped due to timeout of 90000ms (8)
- GitHub Check: tests-smoke-websocket
- GitHub Check: tests-unit-nightly
- GitHub Check: tests-unit
- GitHub Check: tests-smart-contracts
- GitHub Check: hoprd / docker
- GitHub Check: hopli / docker
- GitHub Check: Docs / Rust docs
- GitHub Check: Linter
🔇 Additional comments (7)
scripts/generate-python-sdk.sh (7)
1-6: LGTM! Well-structured script setup.The script follows shell scripting best practices with proper error handling and configurable paths.
7-11: Add validation for hoprd-api-schema command.While swagger-codegen3 is validated, the script also needs to validate the hoprd-api-schema command that's used later.
20-22: Add safeguards for directory cleanup.The current directory cleanup is unsafe if TARGET_DIR is empty or unset.
🧰 Tools
🪛 Shellcheck (0.10.0)
[warning] 22-22: Use "${var:?}" to ensure this never expands to /* .
(SC2115)
24-25: Add error handling for OpenAPI spec generation.The OpenAPI spec generation needs proper error handling and should use the temporary file created earlier.
27-36: LGTM! Proper error handling for SDK generation.The SDK generation code follows best practices with proper error handling and output redirection.
38-38: LGTM! Clear success message.The success message clearly indicates where the SDK was generated.
13-15: 🛠️ Refactor suggestionImprove temporary file handling.
The trap should clean up all temporary files, including the OpenAPI spec file that's created later.
Apply this diff:
# Use mktemp for secure temporary file creation config_file=$(mktemp) -trap 'rm -f "${config_file}"' EXIT +spec_file=$(mktemp) +trap 'rm -f "${config_file}" "${spec_file}"' EXITLikely invalid or redundant comment.
The python autogenerated SDK isn't used by anyone within HOPR as it has been replaced by an async-compatible alternative.
In consequence, there's no need to generate the SDK anymore. Also, the
hopr-sdk-pythonis being deprecated