[Bug]: metanode create metapartition raft panic. #2230
Description
Describe the bug
dTo]applied(96112591) is out of range [prevApplied(105291801), committed(105292441)])
/root/tmp/cfs/go/src/github.com/cubefs/cubefs/depends/tiglabs/raft/util/runtime.go:38
/root/tmp/cfs/go/src/github.com/cubefs/cubefs/depends/tiglabs/raft/util/runtime.go:28
/usr/local/go/src/runtime/panic.go:965
/root/tmp/cfs/go/src/github.com/cubefs/cubefs/depends/tiglabs/raft/raft_log.go:235
/root/tmp/cfs/go/src/github.com/cubefs/cubefs/depends/tiglabs/raft/raft_fsm.go:123
/root/tmp/cfs/go/src/github.com/cubefs/cubefs/depends/tiglabs/raft/raft.go:135
/root/tmp/cfs/go/src/github.com/cubefs/cubefs/depends/tiglabs/raft/server.go:162
/root/tmp/cfs/go/src/github.com/cubefs/cubefs/raftstore/raftstore.go:192
/root/tmp/cfs/go/src/github.com/cubefs/cubefs/metanode/partition.go:440
/root/tmp/cfs/go/src/github.com/cubefs/cubefs/metanode/partition.go:342
/root/tmp/cfs/go/src/github.com/cubefs/cubefs/metanode/partition.go:297
/root/tmp/cfs/go/src/github.com/cubefs/cubefs/metanode/manager.go:393
when metapartition recover from leader's snapshot, and then the meta node killed before store newest snapshot file to disk, mp create raft will be panic when start metanode again.
To Reproduce
- when store snapshot to disk, update code to sleep 1 minutes, to suppose that slow store snapshot to disk case.
- update retainlogs count to 20, so snapshot step can be happened easily.
- try to create 100 file with a new volume A in the root dir.
- stop the A's first mp metapartition replica B
- create another 100 files, sleep 5 minutes, create 100 files, sleep 5 miutes again
- then start replica B，the replica can't start and will panic.
Expected behavior
metanode service can start normally.