Skip to content

Conversation

asdacap
Copy link
Contributor

@asdacap asdacap commented Apr 6, 2025

  • To reduce timeout and high allocation, we limit the size of the bodies, receipts and snap request at protocol handler level.
  • This information however is not exposed to the sync code, so sync code still try to request in a large batch, potentially causing a large portion of the request to be retried later.
  • This is not a big deal for downloads that is paralelizable, bit it can be a problem for sync that requires order.

Types of changes

What types of changes does your code introduce?

  • Optimization

Testing

Requires testing

  • Yes
  • No

If yes, did you write tests?

  • Yes
  • No

Notes on testing

@asdacap asdacap merged commit c3ba425 into master Apr 7, 2025
80 checks passed
@asdacap asdacap deleted the perf/accurate-request-limit branch April 7, 2025 07:11
Comment on lines +73 to +79
public IEnumerator<BlockBody?> GetEnumerator()
{
foreach (var blockBody in _rawBodies)
{
yield return blockBody;
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just

Suggested change
public IEnumerator<BlockBody?> GetEnumerator()
{
foreach (var blockBody in _rawBodies)
{
yield return blockBody;
}
}
public IEnumerator<BlockBody?> GetEnumerator() => _rawBodies.GetEnumerator();

/// on the allocation strategy and context. May not be accurate as different peer may get allocated.
/// </summary>
Task<int?> EstimateRequestLimit(
RequestType bodies,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
RequestType bodies,
RequestType requestType,

Task<int?> EstimateRequestLimit(
RequestType bodies,
IPeerAllocationStrategy peerAllocationStrategy,
AllocationContexts blocks,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

allocationContext

asdacap added a commit that referenced this pull request Apr 15, 2025
asdacap added a commit that referenced this pull request Apr 30, 2025
* Parallel forward sync

* Singleton for the block downloader

* Some test

* Add test for parallel download

* Whitespace

* Map pruner

* Whitespace

* Fix build

* Move request sizer to node stats

* Expose request size

* For bodies

* Simplify

* Fast headers

* Receipt sync feed

* Also check for response count

* DLS and fix test

* Fix wrong signature

* Use tx count to estimate buffer size

* Nicer dsl

* Hard timeout

* Fix overlow error

* Fix overlow error

* More tuning

* Address comment from #8474

* Fix build

* Fix test

* Remove unnecessary code

* Trying to reduce change

* Minor change

* Make blocks request disposable

* Fix some test

* Fix test

* Slight cleanup

* Minor cleanup

* Whitespace

* Fix test

* Address copilot

* Fix test

* Update src/Nethermind/Nethermind.Synchronization/Blocks/BlockDownloader.cs

Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com>

* Update src/Nethermind/Nethermind.Synchronization/Blocks/BlockDownloader.cs

Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com>

* Update src/Nethermind/Nethermind.Synchronization/Blocks/BlockDownloader.cs

Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com>

* Update src/Nethermind/Nethermind.Synchronization/Blocks/BlockDownloader.cs

Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com>

* Update src/Nethermind/Nethermind.Synchronization/Blocks/BlockDownloader.cs

Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com>

* Update src/Nethermind/Nethermind.Synchronization/Blocks/BlockDownloader.cs

Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com>

* Update src/Nethermind/Nethermind.Synchronization/Blocks/BlockDownloader.cs

Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com>

* Update src/Nethermind/Nethermind.Synchronization/Blocks/BlocksSyncPeerSelectionStrategy.cs

Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com>

* Address comment

* Dont change tests

* Introduce allocate delay to prevent spinning

---------

Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants