Enhancement Task

What did I do?

In dev, run:
./pd-tso-bench -v -duration 250000s -pd "http://serverless-cluster-pd-0.serverless-cluster-pd-peer.tidb-serverless.svc:2379" -client 1 -c 1 -interval 10s

Kill all PD/API pods at 11:59:18 PDT
~  kubectl delete pod serverless-cluster-pd-0 serverless-cluster-pd-1 serverless-cluster-pd-2 -n tidb-serverless --force --grace-period=0
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "serverless-cluster-pd-0" force deleted
pod "serverless-cluster-pd-1" force deleted
pod "serverless-cluster-pd-2" force deleted
~  date  ✔  10376  11:59:17
Mon Jun 26 11:59:18 PDT 2023

pod 0 started at 11:59:25 PDT and ready to serve at 12:00:03 PDT
starting pd-server ...
/pd-server --data-dir=/var/lib/pd --name=serverless-cluster-pd-0 --peer-urls=http://0.0.0.0:2380 --advertise-peer-urls=http://serverless-cluster-pd-0.serverless-cluster-pd-peer.tidb-serverless.svc:2380 --client-urls=http://0.0.0.0:2379 --advertise-client-urls=http://serverless-cluster-pd-0.serverless-cluster-pd-peer.tidb-serverless.svc:2379 --config=/etc/pd/pd.toml --join=http://serverless-cluster-pd-0.serverless-cluster-pd-peer.tidb-serverless.svc:2380,http://serverless-cluster-pd-1.serverless-cluster-pd-peer.tidb-serverless.svc:2380,http://serverless-cluster-pd-2.serverless-cluster-pd-peer.tidb-serverless.svc:2380
[2023/06/26 18:59:25.773 +00:00] [INFO] [versioninfo.go:89] ["Welcome to Placement Driver (API SERVICE)"]
...
[2023/06/26 19:00:03.123 +00:00] [INFO] [manager.go:74] ["Key visual service is started"]

PD client updateMember can't recover.
Check log here https://gist.githubusercontent.com/binshi-bing/d669ed80e48073f4923c51b29ce95642/raw/7b339f6c319333453e9e17dc136393f4a551a5ec/gistfile1.txt