Skip to content

scheduler: make hot v2 more suitable small hot region (#6827) #6864

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Aug 7, 2023
Merged

scheduler: make hot v2 more suitable small hot region (#6827) #6864

merged 4 commits into from
Aug 7, 2023

Conversation

ti-chi-bot
Copy link
Member

This is an automated cherry-pick of #6827

What problem does this PR solve?

Issue Number: Close #6645

What is changed and how does it work?

At the master, when the load difference between the high and low nodes is large, the hot peer needs to be larger than 20% of the diff or 2% of the low node.

In this pr, we introduce the 20th node to be compared with 2% of the low node and take the smaller of these values to avoid unscheduling when the load is all composed of small hotspots.

And we cache filterHotpeers to avoid redundancy calculations.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  1. test in some small hot regions
    master after evict leader
    image
    image

this pr after evict leader
image
image

  1. contrast to 7.2, there is no rollback of tpcc loads for this pr
    image

Release note

None.

@lhy1024 @ti-chi-bot
This is an automated cherry-pick of tikv#6827
9b7778b 
close tikv#6645

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot ti-chi-bot mentioned this pull request Jul 31, 2023
Merged
@ti-chi-bot ti-chi-bot added ok-to-test Indicates a PR is ready to be tested. release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. type/cherry-pick-for-release-6.5 This PR is cherry-picked to release-6.5 from a source PR. labels Jul 31, 2023
@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Jul 31, 2023 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • bufferflies
  • nolouch

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot bot requested review from disksing and lhy1024 July 31, 2023 08:00
@ti-chi-bot ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 31, 2023
@cfzjywxk
Copy link

cfzjywxk commented Aug 7, 2023

@lhy1024 @nolouch
Would it be merged on the release-6.5 branch?If so v6.5.4 requires this.

@nolouch
Copy link
Contributor

nolouch commented Aug 7, 2023

@cfzjywxk Yes, it's a bugfix, I think it's ok to merge to 6.5.

@lhy1024
Copy link
Contributor

lhy1024 commented Aug 7, 2023

@lhy1024 @nolouch
Would it be merged on the release-6.5 branch?If so v6.5.4 requires this.

Ok, I will fix this conflict

@ti-chi-bot ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Aug 7, 2023
@lhy1024
fix test
2d55756 
Signed-off-by: lhy1024 <admin@liudos.us>
@codecov
Copy link

codecov bot commented Aug 7, 2023

Codecov Report

Patch coverage: 100.00% and project coverage change: -0.05% ⚠️

Comparison is base (198860d) 75.63% compared to head (2d55756) 75.58%.

Additional details and impacted files 
@@               Coverage Diff               @@
##           release-6.5    #6864      +/-   ##
===============================================
- Coverage        75.63%   75.58%   -0.05%     
===============================================
  Files              328      328              
  Lines            33171    33189      +18     
===============================================
- Hits             25088    25087       -1     
- Misses            5938     5948      +10     
- Partials          2145     2154       +9
Flag Coverage Δ
unittests 75.58% <100.00%> (-0.05%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
server/schedulers/hot_region.go 85.38% <100.00%> (+0.69%) ⬆️
server/schedulers/hot_region_v2.go 89.83% <100.00%> (+1.52%) ⬆️

... and 28 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ti-chi-bot ti-chi-bot bot added the status/LGT1 Indicates that a PR has LGTM 1. label Aug 7, 2023
@ti-chi-bot ti-chi-bot bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Aug 7, 2023
@ti-chi-bot ti-chi-bot added the cherry-pick-approved Cherry pick PR approved by release team. label Aug 7, 2023
@lhy1024
Copy link
Contributor

lhy1024 commented Aug 7, 2023

/merge

@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Aug 7, 2023

@lhy1024: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Aug 7, 2023

This pull request has been accepted and is ready to merge.

Commit hash: 2d55756

@ti-chi-bot ti-chi-bot bot added the status/can-merge Indicates a PR has been approved by a committer. label Aug 7, 2023
@ti-chi-bot ti-chi-bot bot merged commit f2b53f4 into tikv:release-6.5 Aug 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nolouch nolouch nolouch approved these changes

@bufferflies bufferflies bufferflies approved these changes

@disksing disksing Awaiting requested review from disksing

@lhy1024 lhy1024 Awaiting requested review from lhy1024

Assignees

@lhy1024 lhy1024

Labels
cherry-pick-approved Cherry pick PR approved by release team. ok-to-test Indicates a PR is ready to be tested. release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. type/cherry-pick-for-release-6.5 This PR is cherry-picked to release-6.5 from a source PR.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@ti-chi-bot @cfzjywxk @nolouch @lhy1024 @bufferflies