Shutdown deadlock in SyncWithValidationInterfaceQueue #12229
Description
I'm hoping to take a look at this at some point in the next day or two, but logging here in case I forget. Ping @TheBlueMatt.
Trigger conditions: I was catching up after ~1week offline, and interrupted halfway through.
The backtrace is straightforward:
(gdb) thread apply all bt
Thread 3 (Thread 0x7fbb37fff700 (LWP 31792)):
#0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
#1 0x00007fbb75a9aa16 in std::__atomic_futex_unsigned_base::_M_futex_wait_until(unsigned int*, unsigned int, bool, std::chrono::duration<long, std::ratio<1l, 1l> >, std::chrono::duration<long, std::ratio<1l, 1000000000l> >) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#2 0x000055d498e501e0 in SyncWithValidationInterfaceQueue() ()
#3 0x000055d498e34965 in CChainState::ActivateBestChain(CValidationState&, CChainParams const&, std::shared_ptr<CBlock const>) ()
#4 0x000055d498e36847 in ProcessNewBlock(CChainParams const&, std::shared_ptr<CBlock const>, bool, bool*) ()
#5 0x000055d498d32464 in ProcessMessage(CNode*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, CDataStream&, long, CChainParams const&, CConnman*, std::atomic<bool> const&) [clone .constprop.1364] ()
#6 0x000055d498d3dd80 in PeerLogicValidation::ProcessMessages(CNode*, std::atomic<bool>&) ()
#7 0x000055d498cee7cd in CConnman::ThreadMessageHandler() ()
#8 0x000055d498cce47f in void TraceThread<std::function<void ()> >(char const*, std::function<void ()>) ()
#9 0x000055d498d065cb in std::thread::_Impl<std::_Bind_simple<void (*(char const*, std::function<void ()>))(char const*, std::function<void ()>)> >::_M_run() ()
#10 0x00007fbb75a9cc80 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#11 0x00007fbb75d6d6ba in start_thread (arg=0x7fbb37fff700) at pthread_create.c:333
#12 0x00007fbb7520241d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
Thread 2 (Thread 0x7fbb61e98700 (LWP 31778)):
#0 pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1 0x000055d499034a1b in leveldb::(anonymous namespace)::PosixEnv::BGThreadWrapper(void*) ()
#2 0x00007fbb75d6d6ba in start_thread (arg=0x7fbb61e98700) at pthread_create.c:333
#3 0x00007fbb7520241d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
Thread 1 (Thread 0x7fbb76384740 (LWP 31765)):
#0 0x00007fbb75d6e98d in pthread_join (threadid=140442075133696, thread_return=0x0) at pthread_join.c:90
#1 0x00007fbb75a9cb97 in std::thread::join() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#2 0x000055d498d037ab in CConnman::Stop() ()
#3 0x000055d498cc9630 in Shutdown() ()
#4 0x000055d498c9e2a3 in AppInit(int, char**) ()
#5 0x000055d498c9164f in main ()