statistics: more friendly output #3731

lhy1024 · 2021-06-02T17:03:35Z

Signed-off-by: lhy1024 admin@liudos.us

What problem does this PR solve?

After #3591, we use []float64 to represent hot peer loads, but output is no friendly.

After this pr:

What is changed and how it works?

Check List

Tests

Unit test

Release note

 None.

Signed-off-by: lhy1024 <admin@liudos.us>

codecov · 2021-06-02T17:30:40Z

Codecov Report

Merging #3731 (7788222) into master (ab8b5b3) will increase coverage by 0.01%.
The diff coverage is 88.57%.

@@            Coverage Diff             @@
##           master    #3731      +/-   ##
==========================================
+ Coverage   74.82%   74.84%   +0.01%     
==========================================
  Files         244      244              
  Lines       24374    24400      +26     
==========================================
+ Hits        18239    18261      +22     
- Misses       4511     4514       +3     
- Partials     1624     1625       +1

Flag	Coverage Δ
unittests	`74.84% <88.57%> (+0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
server/api/trend.go	`66.19% <50.00%> (ø)`
server/schedulers/utils.go	`93.75% <93.54%> (-0.23%)`	⬇️
server/tso/global_allocator.go	`70.55% <0.00%> (-5.00%)`	⬇️
server/id/id.go	`76.19% <0.00%> (-4.77%)`	⬇️
server/member/member.go	`65.59% <0.00%> (-3.23%)`	⬇️
server/election/leadership.go	`83.52% <0.00%> (-2.36%)`	⬇️
tools/pd-ctl/pdctl/command/operator.go	`65.99% <0.00%> (-1.22%)`	⬇️
client/base_client.go	`82.47% <0.00%> (-0.52%)`	⬇️
server/handler.go	`43.17% <0.00%> (-0.50%)`	⬇️
server/tso/allocator_manager.go	`77.46% <0.00%> (-0.34%)`	⬇️
... and 10 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ab8b5b3...7788222. Read the comment docs.

rleungx · 2021-06-03T02:44:57Z

server/statistics/hot_regions_stat.go

+	HotDegree      int       `json:"hot_degree"`
+	ByteRate       float64   `json:"flow_bytes"`
+	KeyRate        float64   `json:"flow_keys"`
+	AntiCount      int       `json:"anti_count"`


I don't think users can understand it.

But I think it's helpful for us to debug.

rleungx · 2021-06-03T02:46:38Z

server/api/trend.go

@@ -152,9 +152,9 @@ func (h *trendHandler) getStoreFlow(stats statistics.StoreHotPeersStat, regionSt
 		return
 	}
 	if stat, ok := stats[storeID]; ok {
-		storeFlow = stat.TotalLoads[regionStatKind]
+		storeFlow = stat.TotalBytesRate


The variable name doesn't match its meaning.

only revert it https://github.com/tikv/pd/pull/3591/files#diff-e7c11b02e9457f3fd65407917fc0c632537ea493dd5499aa18dfc8bc2c067d52L156

So we need to make it better.

rleungx · 2021-06-03T02:46:46Z

server/api/trend.go

 		for _, flow := range stat.Stats {
-			regionFlows = append(regionFlows, flow.GetLoad(regionStatKind))
+			regionFlows = append(regionFlows, flow.ByteRate)


server/statistics/hot_regions_stat.go

nolouch · 2021-06-03T02:58:54Z

Does this affect the TIDB_HOT_REGION table?

lhy1024 · 2021-06-03T04:03:31Z

Does this affect the TIDB_HOT_REGION table?

This pr will let TIDB_HOT_REGION table flow_byte column work well.

But I found region_count column, and I don't know what it means.

Signed-off-by: lhy1024 <admin@liudos.us>

ti-chi-bot · 2021-06-03T05:41:44Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

Yisaer
rleungx

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

lhy1024 · 2021-06-03T05:42:11Z

/merge

ti-chi-bot · 2021-06-03T05:42:12Z

@lhy1024: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2021-06-03T05:42:13Z

This pull request has been accepted and is ready to merge.

Commit hash: 70e266b

ti-chi-bot · 2021-06-03T05:46:44Z

@lhy1024: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

lhy1024 · 2021-06-03T05:53:53Z

/run-integration-common-test

* more friendly output Signed-off-by: lhy1024 <admin@liudos.us> * address comment Signed-off-by: lhy1024 <admin@liudos.us> * fix lint Signed-off-by: lhy1024 <admin@liudos.us> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>

more friendly output

ef9d8e8

Signed-off-by: lhy1024 <admin@liudos.us>

ti-chi-bot added the release-note-none Denotes a PR that doesn't merit a release note. label Jun 2, 2021

ti-chi-bot requested review from disksing and HunDunDM June 2, 2021 17:03

lhy1024 requested review from Yisaer and rleungx and removed request for disksing June 2, 2021 17:03

rleungx reviewed Jun 3, 2021

View reviewed changes

lhy1024 added 2 commits June 3, 2021 12:05

address comment

9fc7e77

Signed-off-by: lhy1024 <admin@liudos.us>

fix lint

70e266b

Signed-off-by: lhy1024 <admin@liudos.us>

Yisaer approved these changes Jun 3, 2021

View reviewed changes

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jun 3, 2021

rleungx approved these changes Jun 3, 2021

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jun 3, 2021

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jun 3, 2021

Merge branch 'master' into more-friendly-output

7788222

ti-chi-bot merged commit fb90e65 into tikv:master Jun 3, 2021

lhy1024 deleted the more-friendly-output branch June 3, 2021 06:33

nolouch mentioned this pull request Jul 13, 2021

Adjustable Priority Multidimensional Hot Scheduler Tasklist #3869

Closed

28 tasks

statistics: more friendly output #3731

statistics: more friendly output #3731

Uh oh!

Conversation

lhy1024 commented Jun 2, 2021

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

Uh oh!

codecov bot commented Jun 2, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nolouch commented Jun 3, 2021

Uh oh!

lhy1024 commented Jun 3, 2021

Uh oh!

ti-chi-bot commented Jun 3, 2021

Uh oh!

lhy1024 commented Jun 3, 2021

Uh oh!

ti-chi-bot commented Jun 3, 2021

Uh oh!

ti-chi-bot commented Jun 3, 2021

Uh oh!

ti-chi-bot commented Jun 3, 2021

Uh oh!

lhy1024 commented Jun 3, 2021

Uh oh!

Uh oh!

codecov bot commented Jun 2, 2021 •

edited

Loading