Skip to content

scheduler: make hot v2 more suitable small hot region (#6827) #6865

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Aug 7, 2023
Merged

scheduler: make hot v2 more suitable small hot region (#6827) #6865

merged 4 commits into from
Aug 7, 2023

Conversation

ti-chi-bot
Copy link
Member

This is an automated cherry-pick of #6827

What problem does this PR solve?

Issue Number: Close #6645

What is changed and how does it work?

At the master, when the load difference between the high and low nodes is large, the hot peer needs to be larger than 20% of the diff or 2% of the low node.

In this pr, we introduce the 20th node to be compared with 2% of the low node and take the smaller of these values to avoid unscheduling when the load is all composed of small hotspots.

And we cache filterHotpeers to avoid redundancy calculations.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  1. test in some small hot regions
    master after evict leader
    image
    image

this pr after evict leader
image
image

  1. contrast to 7.2, there is no rollback of tpcc loads for this pr
    image

Release note

None.

@lhy1024 @ti-chi-bot
This is an automated cherry-pick of tikv#6827
80ed204 
close tikv#6645

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot ti-chi-bot mentioned this pull request Jul 31, 2023
Merged
@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Jul 31, 2023 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • bufferflies
  • nolouch

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added ok-to-test Indicates a PR is ready to be tested. release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. type/cherry-pick-for-release-7.1 This PR is cherry-picked to release-7.1 from a source PR. labels Jul 31, 2023
@ti-chi-bot ti-chi-bot bot requested review from lhy1024 and rleungx July 31, 2023 08:01
lhy1024 added 2 commits August 7, 2023 13:53
@lhy1024
fix conflict
2ece802 
Signed-off-by: lhy1024 <admin@liudos.us>
@lhy1024
fix test
16455db 
Signed-off-by: lhy1024 <admin@liudos.us>
@ti-chi-bot ti-chi-bot bot added the status/LGT1 Indicates that a PR has LGTM 1. label Aug 7, 2023
@ti-chi-bot ti-chi-bot added the cherry-pick-approved Cherry pick PR approved by release team. label Aug 7, 2023
@ti-chi-bot ti-chi-bot bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Aug 7, 2023
@nolouch
Copy link
Contributor

nolouch commented Aug 7, 2023

/merge

@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Aug 7, 2023

@nolouch: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Aug 7, 2023

This pull request has been accepted and is ready to merge.

Commit hash: 16455db

@ti-chi-bot ti-chi-bot bot added the status/can-merge Indicates a PR has been approved by a committer. label Aug 7, 2023
@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Aug 7, 2023

@ti-chi-bot: Your PR was out of date, I have automatically updated it for you.

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@codecov
Copy link

codecov bot commented Aug 7, 2023

Codecov Report

Patch coverage: 87.77% and project coverage change: +0.01% 🎉

Comparison is base (f8bf1d7) 74.96% compared to head (2781f05) 74.98%.
Report is 1 commits behind head on release-7.1.

Additional details and impacted files 
@@               Coverage Diff               @@
##           release-7.1    #6865      +/-   ##
===============================================
+ Coverage        74.96%   74.98%   +0.01%     
===============================================
  Files              406      406              
  Lines            40755    40812      +57     
===============================================
+ Hits             30551    30601      +50     
- Misses            7539     7542       +3     
- Partials          2665     2669       +4
Flag Coverage Δ
unittests 74.98% <87.77%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
client/errs/errno.go 100.00% <ø> (ø)
client/resource_group/controller/config.go 87.50% <ø> (ø)
client/resource_group/controller/limiter.go 72.06% <ø> (ø)
client/resource_group/controller/testutil.go 100.00% <ø> (ø)
pkg/schedule/placement/fit.go 94.94% <0.00%> (-1.08%) ⬇️
pkg/schedule/operator/create_operator.go 91.07% <72.72%> (-0.71%) ⬇️
pkg/schedule/checker/rule_checker.go 83.00% <76.92%> (-0.69%) ⬇️
client/resource_group/controller/controller.go 74.32% <100.00%> (+0.67%) ⬆️
pkg/schedule/operator/builder.go 86.68% <100.00%> (+0.08%) ⬆️
pkg/schedule/operator/operator.go 92.85% <100.00%> (ø)
... and 2 more

... and 30 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ti-chi-bot ti-chi-bot bot merged commit 09ebb3f into tikv:release-7.1 Aug 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nolouch nolouch nolouch approved these changes

@bufferflies bufferflies bufferflies approved these changes

@lhy1024 lhy1024 Awaiting requested review from lhy1024

@rleungx rleungx Awaiting requested review from rleungx

Assignees

@lhy1024 lhy1024

Labels
cherry-pick-approved Cherry pick PR approved by release team. ok-to-test Indicates a PR is ready to be tested. release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. type/cherry-pick-for-release-7.1 This PR is cherry-picked to release-7.1 from a source PR.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@ti-chi-bot @nolouch @bufferflies @lhy1024