Skip to content

inIdexRangeScan for large table hit ERROR 9001 (HY000): PD server timeout due to "PD returned regions have gaps" #60627

@mayjiang0203

Description

@mayjiang0203

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

One 1 billion rows table with one index.
Run the following sql

mysql> show create table t;
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table     | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| t | CREATE TABLE `t` (
  `id` varchar(32) NOT NULL,
  `k` bigint NOT NULL,
  `c` varchar(32) DEFAULT NULL,
  `b` decimal(19,3) DEFAULT NULL,
  `r` datetime DEFAULT NULL,
  `comments` varchar(255) DEFAULT NULL,
  PRIMARY KEY (`id`) /*T![clustered_index] NONCLUSTERED */,
  KEY `idx_k` (`k`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin /*T! SHARD_ROW_ID_BITS=4 PRE_SPLIT_REGIONS=4 */ |
+-----------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.05 sec)

run the following sql

mysql> select count(customer_number) from t_account where customer_number <> 'name10000001650000001';
ERROR 9001 (HY000): PD server timeout:

2. What did you expect to see? (Required)

mysql> select count(k) from t where k <> 'name10000001650000001';
+------------------------+
| count(k) |
+------------------------+
|              999999999 |
+------------------------+
1 row in set (2 min 40.31 sec)

3. What did you see instead (Required)

mysql> select count(k) from t where k <> 'name10000001650000001';
ERROR 9001 (HY000): PD server timeout:

error logs

2025-04-17 12:01:49 (UTC+08:00)TiDB tc-tidb-1.tc-tidb-peer.e2e-oltp-sysbench-wr-stab-tps-7754415-1-906.svc:4000[backoff.go:179] ["pdRPC backoffer.maxSleep 10000ms is exceeded, errors:\nPD returned regions have gaps, range num: 2, limit: 128 at 2025-04-17T12:01:44.521850053+08:00\nPD returned regions have gaps, range num: 2, limit: 128 at 2025-04-17T12:01:46.075557903+08:00\nPD returned regions have gaps, range num: 2, limit: 128 at 2025-04-17T12:01:47.738450713+08:00\ntotal-backoff-times: 8, backoff-detail: pdRPC:8, maxBackoffTimeExceeded: true, maxExcludedTimeExceeded: false\nlongest sleep type: pdRPC, time: 11932ms"] [conn=1646264616] [session_alias=]

4. What is your TiDB version? (Required)

v8.5.1

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions