Backports #1558 and #1584 to 0.38.x #1592
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Blocked by #1589 |
|
For the experimental branch, would it not be easier to backport to 0.38 and then merge 0.38 into experimental? Probably depends on how we want to treat the experimental branch. |
492902a to
2dc96ed
Compare
…nsactions out (#1558) * maxpeers for mempool * mempool: fix max_peers bcast routine active flag * Use semaphore to limit concurrency * Rename MaxPeers to MaxOutboundPeers * Add max_outbound_peers to config toml template * Rename in error message * Renams the parameter to highlight its experimental nature. Extend the AddPeer method to return an error. Moves the semaphone to outside the broadcast routine * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * Fixing lint issue * renaming semaphore to something more meaningful * make default value 0, which is the same as the current behavior. 10 is the recommended value. * adding new flag to manifest.go * Adding changelog * Improve the description of the parameter in the generated config file. * Add metric to track the current number of active connections. * Change metric to gauge type and rename it. * e2e: Allow disabling the PEX reactor on all nodes in the testnet * Apply suggestions from code review Co-authored-by: Sergio Mena <sergio@informal.systems> * Update config/config.go comment * fix lint error * Improve config description * Rename metric (remove experimental prefix) * Add unit test * Improve unit test * Update mempool/reactor.go comment --------- Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: lasarojc <lasaro@informal.systems> Co-authored-by: hvanz <hernan.vanzetto@gmail.com> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems>
…ers (experimental) (#1584) * Ignore persistent peers from limiting of outbound connections * Update 1558-experimental-gossip-limiting.md Update changeling * Fix typo in mempool/metrics.go * Use two independent configs and semaphores for persistent and non-persistent peers * Forgot to rename in test * Update metric description * Rename semaphores * Add comment to unit test --------- Co-authored-by: hvanz <hernan.vanzetto@gmail.com>
8d8a2f9 to
19bd1ed
Compare
Done. The PR has now been rebased to v0.38.x instead of experimental. |
lasarojc
added a commit
that referenced
this pull request
Nov 14, 2023
* Experimental - Reduce # of connections effectively used to gossip transactions out (#1558) * maxpeers for mempool * mempool: fix max_peers bcast routine active flag * Use semaphore to limit concurrency * Rename MaxPeers to MaxOutboundPeers * Add max_outbound_peers to config toml template * Rename in error message * Renams the parameter to highlight its experimental nature. Extend the AddPeer method to return an error. Moves the semaphone to outside the broadcast routine * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * Fixing lint issue * renaming semaphore to something more meaningful * make default value 0, which is the same as the current behavior. 10 is the recommended value. * adding new flag to manifest.go * Adding changelog * Improve the description of the parameter in the generated config file. * Add metric to track the current number of active connections. * Change metric to gauge type and rename it. * e2e: Allow disabling the PEX reactor on all nodes in the testnet * Apply suggestions from code review Co-authored-by: Sergio Mena <sergio@informal.systems> * Update config/config.go comment * fix lint error * Improve config description * Rename metric (remove experimental prefix) * Add unit test * Improve unit test * Update mempool/reactor.go comment --------- Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: lasarojc <lasaro@informal.systems> Co-authored-by: hvanz <hernan.vanzetto@gmail.com> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems> * Updating test file, leaving it broken for now * mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (#1584) * Ignore persistent peers from limiting of outbound connections * Update 1558-experimental-gossip-limiting.md Update changeling * Fix typo in mempool/metrics.go * Use two independent configs and semaphores for persistent and non-persistent peers * Forgot to rename in test * Update metric description * Rename semaphores * Add comment to unit test --------- Co-authored-by: hvanz <hernan.vanzetto@gmail.com> * Reverting to old way of reporting errors * Reverting change that shouldn't have been included in cherry-pick * Reverting tests to use older functions * fix rebase merge --------- Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: hvanz <hernan.vanzetto@gmail.com> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems>
lasarojc
added a commit
that referenced
this pull request
Nov 15, 2023
* Experimental - Reduce # of connections effectively used to gossip transactions out (#1558) * maxpeers for mempool * mempool: fix max_peers bcast routine active flag * Use semaphore to limit concurrency * Rename MaxPeers to MaxOutboundPeers * Add max_outbound_peers to config toml template * Rename in error message * Renams the parameter to highlight its experimental nature. Extend the AddPeer method to return an error. Moves the semaphone to outside the broadcast routine * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * Fixing lint issue * renaming semaphore to something more meaningful * make default value 0, which is the same as the current behavior. 10 is the recommended value. * adding new flag to manifest.go * Adding changelog * Improve the description of the parameter in the generated config file. * Add metric to track the current number of active connections. * Change metric to gauge type and rename it. * e2e: Allow disabling the PEX reactor on all nodes in the testnet * Apply suggestions from code review * Update config/config.go comment * fix lint error * Improve config description * Rename metric (remove experimental prefix) * Add unit test * Improve unit test * Update mempool/reactor.go comment --------- * Updating test file, leaving it broken for now * mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (#1584) * Ignore persistent peers from limiting of outbound connections * Update 1558-experimental-gossip-limiting.md Update changeling * Fix typo in mempool/metrics.go * Use two independent configs and semaphores for persistent and non-persistent peers * Forgot to rename in test * Update metric description * Rename semaphores * Add comment to unit test --------- * Reverting to old way of reporting errors * Reverting change that shouldn't have been included in cherry-pick * Reverting tests to use older functions * fix rebase merge --------- Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: hvanz <hernan.vanzetto@gmail.com> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems>
hvanz
added a commit
that referenced
this pull request
Nov 16, 2023
Backports #1558 and #1584 to 0.38.x (#1592) * mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (#1584) * Experimental - Reduce # of connections effectively used to gossip transactions out (#1558) * maxpeers for mempool * mempool: fix max_peers bcast routine active flag * Use semaphore to limit concurrency * Rename MaxPeers to MaxOutboundPeers * Add max_outbound_peers to config toml template * Rename in error message * Renams the parameter to highlight its experimental nature. Extend the AddPeer method to return an error. Moves the semaphone to outside the broadcast routine * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * Fixing lint issue * renaming semaphore to something more meaningful * make default value 0, which is the same as the current behavior. 10 is the recommended value. * adding new flag to manifest.go * Adding changelog * Improve the description of the parameter in the generated config file. * Add metric to track the current number of active connections. * Change metric to gauge type and rename it. * e2e: Allow disabling the PEX reactor on all nodes in the testnet * Apply suggestions from code review * Update config/config.go comment * fix lint error * Improve config description * Rename metric (remove experimental prefix) * Add unit test * Improve unit test * Update mempool/reactor.go comment --------- * Updating test file, leaving it broken for now * mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (#1584) * Ignore persistent peers from limiting of outbound connections * Update 1558-experimental-gossip-limiting.md Update changeling * Fix typo in mempool/metrics.go * Use two independent configs and semaphores for persistent and non-persistent peers * Forgot to rename in test * Update metric description * Rename semaphores * Add comment to unit test --------- * Reverting to old way of reporting errors * Reverting change that shouldn't have been included in cherry-pick * Reverting tests to use older functions * fix rebase merge --------- Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: hvanz <hernan.vanzetto@gmail.com> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems>
3 tasks
hvanz
added a commit
that referenced
this pull request
Nov 16, 2023
* Backports #1558 and #1584 to 0.37.x (#1611) Backports #1558 and #1584 to 0.38.x (#1592) * mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (#1584) * Experimental - Reduce # of connections effectively used to gossip transactions out (#1558) * maxpeers for mempool * mempool: fix max_peers bcast routine active flag * Use semaphore to limit concurrency * Rename MaxPeers to MaxOutboundPeers * Add max_outbound_peers to config toml template * Rename in error message * Renams the parameter to highlight its experimental nature. Extend the AddPeer method to return an error. Moves the semaphone to outside the broadcast routine * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * Fixing lint issue * renaming semaphore to something more meaningful * make default value 0, which is the same as the current behavior. 10 is the recommended value. * adding new flag to manifest.go * Adding changelog * Improve the description of the parameter in the generated config file. * Add metric to track the current number of active connections. * Change metric to gauge type and rename it. * e2e: Allow disabling the PEX reactor on all nodes in the testnet * Apply suggestions from code review * Update config/config.go comment * fix lint error * Improve config description * Rename metric (remove experimental prefix) * Add unit test * Improve unit test * Update mempool/reactor.go comment --------- * Updating test file, leaving it broken for now * mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (#1584) * Ignore persistent peers from limiting of outbound connections * Update 1558-experimental-gossip-limiting.md Update changeling * Fix typo in mempool/metrics.go * Use two independent configs and semaphores for persistent and non-persistent peers * Forgot to rename in test * Update metric description * Rename semaphores * Add comment to unit test --------- * Reverting to old way of reporting errors * Reverting change that shouldn't have been included in cherry-pick * Reverting tests to use older functions * fix rebase merge --------- Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: hvanz <hernan.vanzetto@gmail.com> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems> * Comment that feature only applies to v0 mempool * Fix new metric * This commit makes the test be the same as in main, that is, it ignores the order of transactions in the receiving reactor. (#1629) * Update .changelog/unreleased/improvements/1558-experimental-gossip-limiting.md Co-authored-by: Thane Thomson <connect@thanethomson.com> --------- Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems> Co-authored-by: lasaro <lasaro@informal.systems> Co-authored-by: Thane Thomson <connect@thanethomson.com>
lasarojc
added a commit
that referenced
this pull request
Nov 24, 2023
* Experimental - Reduce # of connections effectively used to gossip transactions out (#1558) * maxpeers for mempool * mempool: fix max_peers bcast routine active flag * Use semaphore to limit concurrency * Rename MaxPeers to MaxOutboundPeers * Add max_outbound_peers to config toml template * Rename in error message * Renams the parameter to highlight its experimental nature. Extend the AddPeer method to return an error. Moves the semaphone to outside the broadcast routine * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * Fixing lint issue * renaming semaphore to something more meaningful * make default value 0, which is the same as the current behavior. 10 is the recommended value. * adding new flag to manifest.go * Adding changelog * Improve the description of the parameter in the generated config file. * Add metric to track the current number of active connections. * Change metric to gauge type and rename it. * e2e: Allow disabling the PEX reactor on all nodes in the testnet * Apply suggestions from code review Co-authored-by: Sergio Mena <sergio@informal.systems> * Update config/config.go comment * fix lint error * Improve config description * Rename metric (remove experimental prefix) * Add unit test * Improve unit test * Update mempool/reactor.go comment --------- Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: lasarojc <lasaro@informal.systems> Co-authored-by: hvanz <hernan.vanzetto@gmail.com> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems> * Updating test file, leaving it broken for now * mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (#1584) * Ignore persistent peers from limiting of outbound connections * Update 1558-experimental-gossip-limiting.md Update changeling * Fix typo in mempool/metrics.go * Use two independent configs and semaphores for persistent and non-persistent peers * Forgot to rename in test * Update metric description * Rename semaphores * Add comment to unit test --------- Co-authored-by: hvanz <hernan.vanzetto@gmail.com> * Reverting to old way of reporting errors * Reverting change that shouldn't have been included in cherry-pick * Reverting tests to use older functions * fix rebase merge --------- Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: hvanz <hernan.vanzetto@gmail.com> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems>
lasarojc
added a commit
that referenced
this pull request
Nov 24, 2023
* Do not block indefinitely on the semaphore (#1654) * Do not block indefinitely on the semaphore * Cancel the context, irrespective of the flow followed * Makes the code more readable * Improving comment * make linter happy * Updating comments to match * Commenting out `select` and leaving it as TODO for when Contexts are more widely used * Cleaned up comments (cherry picked from commit 2679498) # Conflicts: # config/config.go # config/toml.go # test/e2e/pkg/manifest.go * Backports #1558 and #1584 to 0.38.x (#1592) * Experimental - Reduce # of connections effectively used to gossip transactions out (#1558) * maxpeers for mempool * mempool: fix max_peers bcast routine active flag * Use semaphore to limit concurrency * Rename MaxPeers to MaxOutboundPeers * Add max_outbound_peers to config toml template * Rename in error message * Renams the parameter to highlight its experimental nature. Extend the AddPeer method to return an error. Moves the semaphone to outside the broadcast routine * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * Fixing lint issue * renaming semaphore to something more meaningful * make default value 0, which is the same as the current behavior. 10 is the recommended value. * adding new flag to manifest.go * Adding changelog * Improve the description of the parameter in the generated config file. * Add metric to track the current number of active connections. * Change metric to gauge type and rename it. * e2e: Allow disabling the PEX reactor on all nodes in the testnet * Apply suggestions from code review Co-authored-by: Sergio Mena <sergio@informal.systems> * Update config/config.go comment * fix lint error * Improve config description * Rename metric (remove experimental prefix) * Add unit test * Improve unit test * Update mempool/reactor.go comment --------- Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: lasarojc <lasaro@informal.systems> Co-authored-by: hvanz <hernan.vanzetto@gmail.com> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems> * Updating test file, leaving it broken for now * mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (#1584) * Ignore persistent peers from limiting of outbound connections * Update 1558-experimental-gossip-limiting.md Update changeling * Fix typo in mempool/metrics.go * Use two independent configs and semaphores for persistent and non-persistent peers * Forgot to rename in test * Update metric description * Rename semaphores * Add comment to unit test --------- Co-authored-by: hvanz <hernan.vanzetto@gmail.com> * Reverting to old way of reporting errors * Reverting change that shouldn't have been included in cherry-pick * Reverting tests to use older functions * fix rebase merge --------- Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: hvanz <hernan.vanzetto@gmail.com> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems> * Fixes conflict. --------- Co-authored-by: lasaro <lasaro@informal.systems> Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: hvanz <hernan.vanzetto@gmail.com> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems>
Galadrin
pushed a commit
to Galadrin/cometbft
that referenced
this pull request
Dec 2, 2023
…4.x (cometbft#1630) * Backports cometbft#1558 and cometbft#1584 to 0.37.x (cometbft#1611) Backports cometbft#1558 and cometbft#1584 to 0.38.x (cometbft#1592) * mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (cometbft#1584) * Experimental - Reduce # of connections effectively used to gossip transactions out (cometbft#1558) * maxpeers for mempool * mempool: fix max_peers bcast routine active flag * Use semaphore to limit concurrency * Rename MaxPeers to MaxOutboundPeers * Add max_outbound_peers to config toml template * Rename in error message * Renams the parameter to highlight its experimental nature. Extend the AddPeer method to return an error. Moves the semaphone to outside the broadcast routine * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * Fixing lint issue * renaming semaphore to something more meaningful * make default value 0, which is the same as the current behavior. 10 is the recommended value. * adding new flag to manifest.go * Adding changelog * Improve the description of the parameter in the generated config file. * Add metric to track the current number of active connections. * Change metric to gauge type and rename it. * e2e: Allow disabling the PEX reactor on all nodes in the testnet * Apply suggestions from code review * Update config/config.go comment * fix lint error * Improve config description * Rename metric (remove experimental prefix) * Add unit test * Improve unit test * Update mempool/reactor.go comment --------- * Updating test file, leaving it broken for now * mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (cometbft#1584) * Ignore persistent peers from limiting of outbound connections * Update 1558-experimental-gossip-limiting.md Update changeling * Fix typo in mempool/metrics.go * Use two independent configs and semaphores for persistent and non-persistent peers * Forgot to rename in test * Update metric description * Rename semaphores * Add comment to unit test --------- * Reverting to old way of reporting errors * Reverting change that shouldn't have been included in cherry-pick * Reverting tests to use older functions * fix rebase merge --------- Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: hvanz <hernan.vanzetto@gmail.com> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems> * Comment that feature only applies to v0 mempool * Fix new metric * This commit makes the test be the same as in main, that is, it ignores the order of transactions in the receiving reactor. (cometbft#1629) * Update .changelog/unreleased/improvements/1558-experimental-gossip-limiting.md Co-authored-by: Thane Thomson <connect@thanethomson.com> --------- Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems> Co-authored-by: lasaro <lasaro@informal.systems> Co-authored-by: Thane Thomson <connect@thanethomson.com>
Galadrin
pushed a commit
to Galadrin/cometbft
that referenced
this pull request
Dec 7, 2023
…4.x (cometbft#1630) * Backports cometbft#1558 and cometbft#1584 to 0.37.x (cometbft#1611) Backports cometbft#1558 and cometbft#1584 to 0.38.x (cometbft#1592) * mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (cometbft#1584) * Experimental - Reduce # of connections effectively used to gossip transactions out (cometbft#1558) * maxpeers for mempool * mempool: fix max_peers bcast routine active flag * Use semaphore to limit concurrency * Rename MaxPeers to MaxOutboundPeers * Add max_outbound_peers to config toml template * Rename in error message * Renams the parameter to highlight its experimental nature. Extend the AddPeer method to return an error. Moves the semaphone to outside the broadcast routine * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * Fixing lint issue * renaming semaphore to something more meaningful * make default value 0, which is the same as the current behavior. 10 is the recommended value. * adding new flag to manifest.go * Adding changelog * Improve the description of the parameter in the generated config file. * Add metric to track the current number of active connections. * Change metric to gauge type and rename it. * e2e: Allow disabling the PEX reactor on all nodes in the testnet * Apply suggestions from code review * Update config/config.go comment * fix lint error * Improve config description * Rename metric (remove experimental prefix) * Add unit test * Improve unit test * Update mempool/reactor.go comment --------- * Updating test file, leaving it broken for now * mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (cometbft#1584) * Ignore persistent peers from limiting of outbound connections * Update 1558-experimental-gossip-limiting.md Update changeling * Fix typo in mempool/metrics.go * Use two independent configs and semaphores for persistent and non-persistent peers * Forgot to rename in test * Update metric description * Rename semaphores * Add comment to unit test --------- * Reverting to old way of reporting errors * Reverting change that shouldn't have been included in cherry-pick * Reverting tests to use older functions * fix rebase merge --------- Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: hvanz <hernan.vanzetto@gmail.com> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems> * Comment that feature only applies to v0 mempool * Fix new metric * This commit makes the test be the same as in main, that is, it ignores the order of transactions in the receiving reactor. (cometbft#1629) * Update .changelog/unreleased/improvements/1558-experimental-gossip-limiting.md Co-authored-by: Thane Thomson <connect@thanethomson.com> --------- Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems> Co-authored-by: lasaro <lasaro@informal.systems> Co-authored-by: Thane Thomson <connect@thanethomson.com>
Galadrin
pushed a commit
to Galadrin/cometbft
that referenced
this pull request
Dec 7, 2023
…4.x (cometbft#1630) * Backports cometbft#1558 and cometbft#1584 to 0.37.x (cometbft#1611) Backports cometbft#1558 and cometbft#1584 to 0.38.x (cometbft#1592) * mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (cometbft#1584) * Experimental - Reduce # of connections effectively used to gossip transactions out (cometbft#1558) * maxpeers for mempool * mempool: fix max_peers bcast routine active flag * Use semaphore to limit concurrency * Rename MaxPeers to MaxOutboundPeers * Add max_outbound_peers to config toml template * Rename in error message * Renams the parameter to highlight its experimental nature. Extend the AddPeer method to return an error. Moves the semaphone to outside the broadcast routine * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * Fixing lint issue * renaming semaphore to something more meaningful * make default value 0, which is the same as the current behavior. 10 is the recommended value. * adding new flag to manifest.go * Adding changelog * Improve the description of the parameter in the generated config file. * Add metric to track the current number of active connections. * Change metric to gauge type and rename it. * e2e: Allow disabling the PEX reactor on all nodes in the testnet * Apply suggestions from code review * Update config/config.go comment * fix lint error * Improve config description * Rename metric (remove experimental prefix) * Add unit test * Improve unit test * Update mempool/reactor.go comment --------- * Updating test file, leaving it broken for now * mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (cometbft#1584) * Ignore persistent peers from limiting of outbound connections * Update 1558-experimental-gossip-limiting.md Update changeling * Fix typo in mempool/metrics.go * Use two independent configs and semaphores for persistent and non-persistent peers * Forgot to rename in test * Update metric description * Rename semaphores * Add comment to unit test --------- * Reverting to old way of reporting errors * Reverting change that shouldn't have been included in cherry-pick * Reverting tests to use older functions * fix rebase merge --------- Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: hvanz <hernan.vanzetto@gmail.com> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems> * Comment that feature only applies to v0 mempool * Fix new metric * This commit makes the test be the same as in main, that is, it ignores the order of transactions in the receiving reactor. (cometbft#1629) * Update .changelog/unreleased/improvements/1558-experimental-gossip-limiting.md Co-authored-by: Thane Thomson <connect@thanethomson.com> --------- Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems> Co-authored-by: lasaro <lasaro@informal.systems> Co-authored-by: Thane Thomson <connect@thanethomson.com>
Galadrin
pushed a commit
to Galadrin/cometbft
that referenced
this pull request
Dec 7, 2023
…4.x (cometbft#1630) * Backports cometbft#1558 and cometbft#1584 to 0.37.x (cometbft#1611) Backports cometbft#1558 and cometbft#1584 to 0.38.x (cometbft#1592) * mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (cometbft#1584) * Experimental - Reduce # of connections effectively used to gossip transactions out (cometbft#1558) * maxpeers for mempool * mempool: fix max_peers bcast routine active flag * Use semaphore to limit concurrency * Rename MaxPeers to MaxOutboundPeers * Add max_outbound_peers to config toml template * Rename in error message * Renams the parameter to highlight its experimental nature. Extend the AddPeer method to return an error. Moves the semaphone to outside the broadcast routine * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * Fixing lint issue * renaming semaphore to something more meaningful * make default value 0, which is the same as the current behavior. 10 is the recommended value. * adding new flag to manifest.go * Adding changelog * Improve the description of the parameter in the generated config file. * Add metric to track the current number of active connections. * Change metric to gauge type and rename it. * e2e: Allow disabling the PEX reactor on all nodes in the testnet * Apply suggestions from code review * Update config/config.go comment * fix lint error * Improve config description * Rename metric (remove experimental prefix) * Add unit test * Improve unit test * Update mempool/reactor.go comment --------- * Updating test file, leaving it broken for now * mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (cometbft#1584) * Ignore persistent peers from limiting of outbound connections * Update 1558-experimental-gossip-limiting.md Update changeling * Fix typo in mempool/metrics.go * Use two independent configs and semaphores for persistent and non-persistent peers * Forgot to rename in test * Update metric description * Rename semaphores * Add comment to unit test --------- * Reverting to old way of reporting errors * Reverting change that shouldn't have been included in cherry-pick * Reverting tests to use older functions * fix rebase merge --------- Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: hvanz <hernan.vanzetto@gmail.com> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems> * Comment that feature only applies to v0 mempool * Fix new metric * This commit makes the test be the same as in main, that is, it ignores the order of transactions in the receiving reactor. (cometbft#1629) * Update .changelog/unreleased/improvements/1558-experimental-gossip-limiting.md Co-authored-by: Thane Thomson <connect@thanethomson.com> --------- Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems> Co-authored-by: lasaro <lasaro@informal.systems> Co-authored-by: Thane Thomson <connect@thanethomson.com>
Galadrin
pushed a commit
to Galadrin/cometbft
that referenced
this pull request
Dec 7, 2023
…4.x (cometbft#1630) * Backports cometbft#1558 and cometbft#1584 to 0.37.x (cometbft#1611) Backports cometbft#1558 and cometbft#1584 to 0.38.x (cometbft#1592) * mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (cometbft#1584) * Experimental - Reduce # of connections effectively used to gossip transactions out (cometbft#1558) * maxpeers for mempool * mempool: fix max_peers bcast routine active flag * Use semaphore to limit concurrency * Rename MaxPeers to MaxOutboundPeers * Add max_outbound_peers to config toml template * Rename in error message * Renams the parameter to highlight its experimental nature. Extend the AddPeer method to return an error. Moves the semaphone to outside the broadcast routine * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * Fixing lint issue * renaming semaphore to something more meaningful * make default value 0, which is the same as the current behavior. 10 is the recommended value. * adding new flag to manifest.go * Adding changelog * Improve the description of the parameter in the generated config file. * Add metric to track the current number of active connections. * Change metric to gauge type and rename it. * e2e: Allow disabling the PEX reactor on all nodes in the testnet * Apply suggestions from code review * Update config/config.go comment * fix lint error * Improve config description * Rename metric (remove experimental prefix) * Add unit test * Improve unit test * Update mempool/reactor.go comment --------- * Updating test file, leaving it broken for now * mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (cometbft#1584) * Ignore persistent peers from limiting of outbound connections * Update 1558-experimental-gossip-limiting.md Update changeling * Fix typo in mempool/metrics.go * Use two independent configs and semaphores for persistent and non-persistent peers * Forgot to rename in test * Update metric description * Rename semaphores * Add comment to unit test --------- * Reverting to old way of reporting errors * Reverting change that shouldn't have been included in cherry-pick * Reverting tests to use older functions * fix rebase merge --------- Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: hvanz <hernan.vanzetto@gmail.com> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems> * Comment that feature only applies to v0 mempool * Fix new metric * This commit makes the test be the same as in main, that is, it ignores the order of transactions in the receiving reactor. (cometbft#1629) * Update .changelog/unreleased/improvements/1558-experimental-gossip-limiting.md Co-authored-by: Thane Thomson <connect@thanethomson.com> --------- Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems> Co-authored-by: lasaro <lasaro@informal.systems> Co-authored-by: Thane Thomson <connect@thanethomson.com>
3 tasks
nivasan1
pushed a commit
to skip-mev/cometbft
that referenced
this pull request
Jan 16, 2024
…ometbft#1611) * Experimental - Reduce # of connections effectively used to gossip transactions out (cometbft#1558) * maxpeers for mempool * mempool: fix max_peers bcast routine active flag * Use semaphore to limit concurrency * Rename MaxPeers to MaxOutboundPeers * Add max_outbound_peers to config toml template * Rename in error message * Renams the parameter to highlight its experimental nature. Extend the AddPeer method to return an error. Moves the semaphone to outside the broadcast routine * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * Fixing lint issue * renaming semaphore to something more meaningful * make default value 0, which is the same as the current behavior. 10 is the recommended value. * adding new flag to manifest.go * Adding changelog * Improve the description of the parameter in the generated config file. * Add metric to track the current number of active connections. * Change metric to gauge type and rename it. * e2e: Allow disabling the PEX reactor on all nodes in the testnet * Apply suggestions from code review * Update config/config.go comment * fix lint error * Improve config description * Rename metric (remove experimental prefix) * Add unit test * Improve unit test * Update mempool/reactor.go comment --------- * Updating test file, leaving it broken for now * mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (cometbft#1584) * Ignore persistent peers from limiting of outbound connections * Update 1558-experimental-gossip-limiting.md Update changeling * Fix typo in mempool/metrics.go * Use two independent configs and semaphores for persistent and non-persistent peers * Forgot to rename in test * Update metric description * Rename semaphores * Add comment to unit test --------- * Reverting to old way of reporting errors * Reverting change that shouldn't have been included in cherry-pick * Reverting tests to use older functions * fix rebase merge --------- Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: hvanz <hernan.vanzetto@gmail.com> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems>
nivasan1
pushed a commit
to skip-mev/cometbft
that referenced
this pull request
Jan 16, 2024
…ometbft#1689) * Do not block indefinitely on the semaphore (cometbft#1654) * Do not block indefinitely on the semaphore * Cancel the context, irrespective of the flow followed * Makes the code more readable * Improving comment * make linter happy * Updating comments to match * Commenting out `select` and leaving it as TODO for when Contexts are more widely used * Cleaned up comments (cherry picked from commit 2679498) # Conflicts: # config/config.go # config/toml.go # test/e2e/pkg/manifest.go * Backports cometbft#1558 and cometbft#1584 to 0.38.x (cometbft#1592) * Experimental - Reduce # of connections effectively used to gossip transactions out (cometbft#1558) * maxpeers for mempool * mempool: fix max_peers bcast routine active flag * Use semaphore to limit concurrency * Rename MaxPeers to MaxOutboundPeers * Add max_outbound_peers to config toml template * Rename in error message * Renams the parameter to highlight its experimental nature. Extend the AddPeer method to return an error. Moves the semaphone to outside the broadcast routine * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * reverting the addition of error to AddPeer. It fails if the context is done and handling this case will be done some other time, when an actual context is passed into acquire. * Fixing lint issue * renaming semaphore to something more meaningful * make default value 0, which is the same as the current behavior. 10 is the recommended value. * adding new flag to manifest.go * Adding changelog * Improve the description of the parameter in the generated config file. * Add metric to track the current number of active connections. * Change metric to gauge type and rename it. * e2e: Allow disabling the PEX reactor on all nodes in the testnet * Apply suggestions from code review Co-authored-by: Sergio Mena <sergio@informal.systems> * Update config/config.go comment * fix lint error * Improve config description * Rename metric (remove experimental prefix) * Add unit test * Improve unit test * Update mempool/reactor.go comment --------- Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: lasarojc <lasaro@informal.systems> Co-authored-by: hvanz <hernan.vanzetto@gmail.com> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems> * Updating test file, leaving it broken for now * mempool: Limit gossip connections to persistent and non-persistent peers (experimental) (cometbft#1584) * Ignore persistent peers from limiting of outbound connections * Update 1558-experimental-gossip-limiting.md Update changeling * Fix typo in mempool/metrics.go * Use two independent configs and semaphores for persistent and non-persistent peers * Forgot to rename in test * Update metric description * Rename semaphores * Add comment to unit test --------- Co-authored-by: hvanz <hernan.vanzetto@gmail.com> * Reverting to old way of reporting errors * Reverting change that shouldn't have been included in cherry-pick * Reverting tests to use older functions * fix rebase merge --------- Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: hvanz <hernan.vanzetto@gmail.com> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems> * Fixes conflict. --------- Co-authored-by: lasaro <lasaro@informal.systems> Co-authored-by: Adi Seredinschi <adizere@gmail.com> Co-authored-by: Ethan Buchman <ethan@coinculture.info> Co-authored-by: Daniel Cason <daniel.cason@informal.systems> Co-authored-by: hvanz <hernan.vanzetto@gmail.com> Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Sergio Mena <sergio@informal.systems>
This was referenced Feb 25, 2024
3 tasks
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See #1558 and #1584
PR checklist
.changelog(we use unclog to manage our changelog)docs/orspec/) and code comments