This repository was archived by the owner on Feb 18, 2025. It is now read-only.
This repository was archived by the owner on Feb 18, 2025. It is now read-only.
ConsulTxnStore: panic when get transaction returns error #1300
Description
After more real-world testing of the new transaction-based Consul KV store I encountered a panic when the get/read transaction fails with an error:
2021-02-04 05:06:27 DEBUG kv.SubmitMastersToKvStores, clusterName: , force: false: numPairs: 525
2021-02-04 05:06:27 DEBUG kv.SubmitMastersToKvStores: submitKvPairs: 0
2021-02-04 05:06:27 DEBUG consulTxnStore.DistributePairs(): distributing 525 pairs to 20 datacenters
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x921e22]
goroutine 92749 [running]:
github.com/openark/orchestrator/go/kv.(*consulTxnStore).updateDatacenterKVPairs(0xc00066e060, 0xc00744d280, 0xc00744c720, 0x7, 0xc00ac0e000, 0x20d, 0x400, 0x5, 0x0, 0x0, ...)
/tmp/brew2deb/packages/orchestrator/src/orchestrator-gh.git/.gopath/src/github.com/openark/orchestrator/go/kv/consul_txn.go:120 +0xa02
github.com/openark/orchestrator/go/kv.(*consulTxnStore).DistributePairs(0xc00066e060, 0xc008ace000, 0x20d, 0x300, 0x0, 0x0)
/tmp/brew2deb/packages/orchestrator/src/orchestrator-gh.git/.gopath/src/github.com/openark/orchestrator/go/kv/consul_txn.go:229 +0x498
github.com/openark/orchestrator/go/kv.DistributePairs(0xc008ace000, 0x20d, 0x300, 0xc00a499f48, 0x1)
/tmp/brew2deb/packages/orchestrator/src/orchestrator-gh.git/.gopath/src/github.com/openark/orchestrator/go/kv/kv.go:109 +0x85
github.com/openark/orchestrator/go/logic.SubmitMastersToKvStores(0x0, 0x0, 0xc000584500, 0xc007d5b76d, 0x449a96, 0xcde850, 0xc007d5b7d0, 0x9ddba5, 0xc0001ea6b0)
/tmp/brew2deb/packages/orchestrator/src/orchestrator-gh.git/.gopath/src/github.com/openark/orchestrator/go/logic/orchestrator.go:462 +0x59f
created by github.com/openark/orchestrator/go/logic.ContinuousDiscovery.func5
/tmp/brew2deb/packages/orchestrator/src/orchestrator-gh.git/.gopath/src/github.com/openark/orchestrator/go/logic/orchestrator.go:582 +0x316
This happens because go/kv/consul_txn.go:120 is not checking if getTxnResp is nil, which happens if the get transaction fails
This PR updates the logic to consider this failure and adds better unit testing of the ConsulTxnStore. Following this change if there is a failure to read/get the existing keys the ConsulTxnStore will still attempt send updates to the KVs without panic'ing
A PR to fix this coming soon!