Update to 1.38.0 #40
Merged
Update to 1.38.0 #40
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* build aarch64 version of protoc * remove csharp artifact x86 build logic for unix systems * build grpc_csharp_ext artifact for linux aarch64 * refactor platform detection * add generated dllimports for arm64 * fix native library loading on arm64 * include arm64 artifacts in Grpc.Tools * add Grpc.Tools codegen support for linux aarch64 * grpc.tools cleanup
* removed unused DOCKER_BASE_IMAGE functionality * remove legacy docker images for arm build * build python armv7 wheel via crosscompilation * add pyconfig.h hack * improve the dockerfile * yapf format code
grpc#25735) Note that this doesn't turn on the tests. The tests needs to be turned on in the kokoro shell scripts.
The exception is being interpreted as a `limit` (int) to cap the number of stack trace entries to print. https://docs.python.org/3/library/traceback.html#traceback.print_exc
This reverts commit 27de24a.
…ebug symbols for grpc_csharp_ext (grpc#25729) * generate separate .so with/without debug symbols for grpc_csharp_ext * add linux .dbginfo.so libraries to Grpc.Core.NativeDebug package * fix aarch64 grpc_csharp_ext stripping * lots of linux artifact build jobs are currently timing out
* tools/buildgen/extract_metadata_from_bazel_xml.py changes from grpc#25272 * regenerate projects * get rid of useless _deduplicate_append * extract_deps worked for both collapsed_sources and collapsed_deps only due to mismatch of bazel label vs target name mismatch * special handling for absl has no effect * improve comments * remove changes that are not needed yet
This reverts commit 2ee7017.
* Fix const-ness of callback client read RPC requests * Fix golden file
* Revert "Revert "CSDS Implementation (grpc#25038)" (grpc#25745)" This reverts commit 98fd4e1. * Add xDS special Bazel build rules * Add 2 todos to remove the added rules
* Implement C++ Admin Interface API * Address reviewer's requests * Remove static asserts for raw pointers * Make sanity tests happy * Windows: pacify conflict between ifndef and macros * Disable admin services test on iOS * Make iOS happy by: * Letting grpcpp_admin conditionally depend on grpcpp_csds * Fix an unexpected side-effect of dependency update
* Add Python PSM security example * Fix lint * Add in maintenance port configuration * Align CLI flags with Java * Pylint
* Updating the config update timeout to 600s * Remove unnecessary lines and comments
* Implement FilterChainMatch algorithm (grpc#25757) (grpc#25815) * Implement FilterChainMatch logic * Add tests for transport protocol too * Tests for duplicate NACKing * Introduce ConnectionManager as an interface for config fetchers * Do not parameterize IncrementIfNonZero * Some formatting * Reviewer comments * Add filter chain match information for duplicate match error * Reviewer comments * Some cleanup * Reviewer comments * Reviewer comments * Reviewer comments * Clang-tidy * Bump to 1.37.0-pre1 (grpc#25839) * Bump to v1.37.0-pre1 * Regenerate projects * Use Realtime instead of Monotonic time for CSDS (grpc#25864) * Add ruby 3.0 support for mac binary packages (grpc#25869) * Bump to 1.37.0 for Final Release (grpc#25891) * Bump version to 1.37.0 * Regenerate projects * PHP: Fix windows build (grpc#25904) * [Backport] xdsinterop: extend the ports to use (grpc#25916) This is to add more ports for forwarding-rule. It's in theory not necessary, because forwarding-rule doesn't need to use the same port as the services. This is a limitation of the test framework, and can be fixed in the future. * try/catch exceptions of both php7 and php8 (grpc#25918) (grpc#25927) * Fix use-after-unref bug in fault_injection_filter (grpc#25903) (grpc#25935) Co-authored-by: Yash Tibrewal <yashkt@google.com> * Fix fault injection filter to still run the original trailing metadata closure when no error (grpc#25933) * also build python3.6 aarch64 manylinux2014 wheel (grpc#25944) * Enable channelz for xds_interop_client and xds_interop_server (grpc#25939) (grpc#25968) * Enable channelz for xds_interop_client and xds_interop_server * Regenerate projects * Fix grpc#25897 to avoid crashes when certificates are not yet updated (grpc#25899) (grpc#25965) * xds: move path_matching and header_matching to all; add fault_injection to ruby (grpc#26030) * Backport several additions / fixes to PHP / Ruby xDS Interop tests (grpc#26037) * PHP: allow xDS interop client to start RPCs asynchronously (grpc#25696) * PHP: allow xDS interop client to start RPCs asynchronously * Address review comments * Remove adhoc test config * PHP: enable fault_injection xds interop test case (grpc#25943) * PHP: enable fault_injection xds interop test case * sanity check yaph_code.sh * some mysterious SIGTERM is being observed * Remove adhoc test cfg * Ruby: Fix xds fault_injection test (grpc#26006) * Fix header_matching for PHP and Ruby (grpc#26017) * [Backport][1.37.x] Add kokoro job for xds psm security tests (grpc#26035) * Add kokoro job for xds psm security tests (grpc#26033) * add kokoro job for xds psm security * Fix scripts * Cleanup * Add copyright * Use existing repo for test driver (grpc#26041) * Use existing repo for test driver * Reviewer comments * Don't check local certs for xds k8s kokoro job (grpc#26046) * xds k8s kokoro - Configure auth for docker (grpc#26050) * xds k8s kokoro - Configure auth for docker * Add -q * xds k8s kokoro job - Add git commit as tag for images (grpc#26051) * [Backport][1.37.x] test_csds (grpc#26047) * Add CSDS xDS interop test (grpc#26007) * Add CSDS xDS interop test * Add CSDS test to the test suite * Fix a typo * Address comments * Improve the logging of each attempt * Improve Python readability * Fix a typo in CSDS test (grpc#26021) * Update the Python dependency for xDS interop test (grpc#26024) `xds-protos` includes all the generated Python files for Envoy protos, which is required by Envoy. * Upgrade setuptools and ProtoBuf Python in prep_xds.sh (grpc#26029) * Upgrade * Trick Kokoro to run xDS test * Restore grpc_python_bazel_test.cfg * Update protobuf * Add extra layer of protection * [C++] Add admin and reflection to xds interop binaries (grpc#25964) * Add admin and reflection to xds interop binaries * Prepare for grpc#25978 * Remove _xds rules * Import json_format * [Backport][1.37.x] Backport xds-k8s driver changes (grpc#26067) * xds-k8s: Update Private CA GKE workload certificates config (grpc#25875) * xds-k8s: Update GKE workload certificates: fix annotation (grpc#25882) * xds-k8s: Use latest TD bootstrap supporting new secrets dir (grpc#25925) * Naming fix (secrets manager -> secret manager) (grpc#25990) Co-authored-by: Seth Vargo <seth@sethvargo.com> * temporarily change ::createInsecure() back to return NULL (grpc#26054) * buildscripts: xds-k8s pin pip to 21.0.1 (grpc#26088) pip 21.1 released on Apr 24 introduced a regression for python 3.6.1. The regression was identified on Apr 24, the fix merged on Apr 25. The fix is expected to be delivered in the 21.1.1 patch. There's no clear date, when 21.1.1 will be released. Until then, pin is temporarily pinned to the previous release, 21.0.1. * Bump to 1.37.1 (grpc#26040) * Bump to 1.37.1 * Regenerate projects * [Backport][1.37.x] xds-k8s buildscripts sync (grpc#26103) * Python PSM Security Interop Client+Server (grpc#25991) * Add channelz to security client * Make client changes. Some config needs to be reverted * Add missing security field to channelz Socket * Add test * Fix test * Remove whitespaces for windows * Remove local_certificate check from PSM security tests * Byte pack Python channelz IPs * Move address packing to Core * WIP * Unbork security tests * Python interop server * Turn up server logging * Clean up PR * Clean up some more * Yapf * Fix up docker images * Swap fillllllles! * Add more robust boolean arg parsing * Use bool_arg parsing in server as well * Add copyright Co-authored-by: Yash Tibrewal <yashkt@google.com> * ADD Python xDS security interop test CI scripts (grpc#26073) * xds-k8s kokoro buildscripts: exclude from tests suites (grpc#26098) * xds-k8s buildscript should use the latest version of the driver (grpc#26100) Co-authored-by: Richard Belleville <rbellevi@google.com> Co-authored-by: Yash Tibrewal <yashkt@google.com> Co-authored-by: Richard Belleville <rbellevi@google.com> Co-authored-by: Lidi Zheng <lidiz@google.com> Co-authored-by: apolcyn <apolcyn@google.com> Co-authored-by: Stanley Cheung <stanleycheung@google.com> Co-authored-by: Menghan Li <menghanl@google.com> Co-authored-by: Hannah Shi <hannahshisfb@gmail.com> Co-authored-by: Jan Tattermusch <jtattermusch@users.noreply.github.com> Co-authored-by: Doug Fawley <dfawley@google.com> Co-authored-by: Sergii Tkachenko <sergiitk@google.com> Co-authored-by: Seth Vargo <seth@sethvargo.com>
The 30min timeout has caused the multiarch artifact build to fail twice in the past week. See http://sponge2/55f7f6f8-d3c0-4b6e-91c8-c0c316f4717b
…c#26090)" (grpc#26197) This reverts commit 9bc421c.
* add single-job python distribtests for linux * add kokoro job config grpc_distribtests_python.cfg * address review comments
This continues the work of ironing out EventEngine API details.
* Add steps to run xDS in a non-exclusive project * Update tools/run_tests/xds_k8s_test_driver/README.md Co-authored-by: Sergii Tkachenko <hi@sergii.org> * Update tools/run_tests/xds_k8s_test_driver/README.md Co-authored-by: Sergii Tkachenko <hi@sergii.org> * Add .cfg and clean-up sections * Use a newer commit hash Co-authored-by: Sergii Tkachenko <hi@sergii.org>
* Bump version to v1.38.0-pre1 * Regenerate projects
Backport of grpc#26220 to v1.38.x. This is the only backport, since the EventEngine code was introduced after v1.37 was cut. This file is needed for the Objective-C cocoapod build.
…c#26282) (grpc#26288) Co-authored-by: Mark D. Roth <roth@google.com>
* Enable Traffic Director time tracer (grpc#26111) * Just update the timeout (grpc#26223) * Increase xds job timeouts (grpc#26187) * Revert grpc_xds_k8s job timeouts back to 120 mins (grpc#26219) * Revert grpc_xds_k8s_python timeout to 120mins (grpc#26221) Seems to have been inadvertently increased in grpc#26187 Co-authored-by: Eric Gribkoff <ericgribkoff@google.com> Co-authored-by: Yash Tibrewal <yashkt@google.com>
* Bump version to v1.38.0 * Regenerate projects
…rpc#26283) (grpc#26313) * Provide custom sponge configs * Correct the script * Allow test scripts to produce artifacts through Docker * Overcome the direcotry clone in our docker script * Expose TESTGRID_EXCLUDE and GIT_ORIGIN_URL through docker run * Make sanity tests happy
BusyJay
pushed a commit
that referenced
this pull request
Sep 28, 2021
…0.2. (grpc#25865) * Add Python mTLS greeter example (#40) * Revert "Add Python mTLS greeter example (#40)" This reverts commit 383c247. * Postpone EVP_cleanup until after last server_ssl_test run completes. * Fix readahead_hs_server_ssl * Clang fixes and client side initialization fix. * Comment out EVP_cleanup on client side. * remove TLS 1.3 ciphers' * change to using server0 credentials * log what TLS method is used' * check compatibility of private key and cert * Try allowing server to use all ciphers. * Add logging for test server. * Fix private key check logging. * add include for tracing * define tsi_tracing_enabled flag * rename tsi_tracing_enabled flag * try printing bytes to send to peer * Add automatic curve selection * Remove logging from SSL transport security * Add back TLS 1.3 ciphersuites. Co-authored-by: Ryan Kim <Ryanfsdf@users.noreply.github.com>
BusyJay
pushed a commit
that referenced
this pull request
Aug 1, 2023
…2491) This fixes a very strange TSAN flake seen in an internal test (b/268292646), which seems to have been introduced in grpc#32326. I've included an example of the TSAN failure below, for future reference. I don't fully understand what is causing this failure. It looks like the async callback from a previous picker update caused the pick to be re-queued but then got stuck draining queued callbacks in its `ExecCtx` instance, while the async callback from the next picker update had already failed the call. But the part I don't understand is why the call combiner cancellation callback wound up scheduled on the `ExecCtx` instance from the first thread -- that seems likely to be caused by either the `WorkSerializer` or maybe `ExecCtx` work-stealing, but the exact details are likely to be hard to nail down. Switching from `EventEngine::Run()` back to `ExecCtx::Run()` seems to fix the flake, so that's what I'm doing in this PR. I find this work-around deeply dissatisfying, especially since I do not fully understand the root cause of the problem. However, given that we are working toward eliminating the `FilterBasedLoadBalancedCall` code entirely as part of the promise conversion, this problem is probably not worth further investigation. ``` WARNING: ThreadSanitizer: data race (pid=7406) Read of size 8 at 0x7b8800037558 by thread T47: #0 grpc_core::ClientChannel::FilterBasedLoadBalancedCall::~FilterBasedLoadBalancedCall() third_party/grpc/src/core/ext/filters/client_channel/client_channel.cc:2780:5 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Uclient_Uchannel.so+0x530ed) (BuildId: a8369742d3c435f7f1a244a535f4907c) #1 Delete third_party/grpc/src/core/lib/gprpp/ref_counted.h:248:31 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Uclient_Uchannel.so+0x64a73) (BuildId: a8369742d3c435f7f1a244a535f4907c) #2 Unref third_party/grpc/src/core/lib/gprpp/orphanable.h:102:7 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Uclient_Uchannel.so+0x64a73) #3 ~RefCountedPtr third_party/grpc/src/core/lib/gprpp/ref_counted_ptr.h:103:36 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Uclient_Uchannel.so+0x64a73) #4 ~LbQueuedCallCanceller third_party/grpc/src/core/ext/filters/client_channel/client_channel.cc:3132:51 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Uclient_Uchannel.so+0x64a73) #5 grpc_core::ClientChannel::FilterBasedLoadBalancedCall::LbQueuedCallCanceller::CancelLocked(void*, absl::Status) third_party/grpc/src/core/ext/filters/client_channel/client_channel.cc:3168:5 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Uclient_Uchannel.so+0x64a73) #6 exec_ctx_run third_party/grpc/src/core/lib/iomgr/exec_ctx.cc:45:3 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibexec_Uctx.so+0x4199) (BuildId: cafb035b95f6c781e93706a71dce95f3) #7 grpc_core::ExecCtx::Flush() third_party/grpc/src/core/lib/iomgr/exec_ctx.cc:72:9 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibexec_Uctx.so+0x4199) #8 ~ExecCtx third_party/grpc/src/core/lib/iomgr/exec_ctx.h:117:5 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Uclient_Uchannel.so+0x6c095) (BuildId: a8369742d3c435f7f1a244a535f4907c) #9 operator() third_party/grpc/src/core/ext/filters/client_channel/client_channel.cc:3201:3 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Uclient_Uchannel.so+0x6c095) #10 __invoke<(lambda at third_party/grpc/src/core/ext/filters/client_channel/client_channel.cc:3196:46) &> third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__functional/invoke.h:394:23 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Uclient_Uchannel.so+0x6c095) #11 invoke<(lambda at third_party/grpc/src/core/ext/filters/client_channel/client_channel.cc:3196:46) &> third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__functional/invoke.h:539:12 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Uclient_Uchannel.so+0x6c095) #12 InvokeR<void, (lambda at third_party/grpc/src/core/ext/filters/client_channel/client_channel.cc:3196:46) &, void> third_party/absl/functional/internal/any_invocable.h:131:3 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Uclient_Uchannel.so+0x6c095) #13 void absl::internal_any_invocable::LocalInvoker<false, void, grpc_core::ClientChannel::FilterBasedLoadBalancedCall::RetryPickLocked()::$_0&>(absl::internal_any_invocable::TypeErasedState*) third_party/absl/functional/internal/any_invocable.h:301:10 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Uclient_Uchannel.so+0x6c095) #14 operator() third_party/absl/functional/internal/any_invocable.h:855:1 (libST-a5842b3303c3_net_Sgrpc_Sinternal_Ssrc_Score_Sext_Sevent_Uengine_Slibevent_Uengine_Uem_Uimpl.so+0x16a74) (BuildId: 7802a789e61dddac5a33230a2f294046) #15 util::functional::internal::FunctorCallback<Closure, false, absl::AnyInvocable<void ()>, void ()>::Run() util/functional/to_callback_internal.h:87:27 (libST-a5842b3303c3_net_Sgrpc_Sinternal_Ssrc_Score_Sext_Sevent_Uengine_Slibevent_Uengine_Uem_Uimpl.so+0x16a74) #16 void eventmanager::EventManager::RunTask<false>(eventmanager::EventManager::WorkerThread*, eventmanager::TaskInfo*) net/eventmanager/em1/eventmanager.cc:2464:20 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x2ae1b) (BuildId: b3d97518d96e160722658c4aa0795013) #17 void eventmanager::EventManager::RunWorkerLoop<false, false>(eventmanager::EventManager::WorkerThread*) net/eventmanager/em1/eventmanager.cc (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x2a45d) (BuildId: b3d97518d96e160722658c4aa0795013) #18 eventmanager::EventManager::WorkerThread::WorkerMain() net/eventmanager/em1/eventmanager.cc:430:19 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x1d841) (BuildId: b3d97518d96e160722658c4aa0795013) #19 operator() net/eventmanager/em1/eventmanager.cc:396:64 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x28351) (BuildId: b3d97518d96e160722658c4aa0795013) #20 __invoke<(lambda at net/eventmanager/em1/eventmanager.cc:396:55)> third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__functional/invoke.h:394:23 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x28351) #21 invoke<(lambda at net/eventmanager/em1/eventmanager.cc:396:55)> third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__functional/invoke.h:539:12 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x28351) #22 InvokeR<void, (lambda at net/eventmanager/em1/eventmanager.cc:396:55), void> third_party/absl/functional/internal/any_invocable.h:131:3 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x28351) #23 void absl::internal_any_invocable::LocalInvoker<false, void, eventmanager::EventManager::WorkerThread::StartThread(int, int, thread::SchedPolicy, std::__tsan::basic_string_view<char, std::__tsan::char_traits<char>>)::'lambda'()&&>(absl::internal_any_invocable::TypeErasedState*) third_party/absl/functional/internal/any_invocable.h:301:10 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x28351) #24 operator() third_party/absl/functional/internal/any_invocable.h:863:1 (libST-a5842b3303c3_security_Sganpati_Ssharded_Uclient_Slibacl_Ucache.so+0x24c1e) (BuildId: a86eb59aee0be7b5882a1b532922e3ca) #25 ClosureThread::Run() thread/thread.h:459:25 (libST-a5842b3303c3_security_Sganpati_Ssharded_Uclient_Slibacl_Ucache.so+0x24c1e) #26 Thread::ThreadBody(void*) thread/thread.cc:1284:16 (libST-a5842b3303c3_thread_Slibthread.so+0x251e9) (BuildId: a1762ef5c5732b3ad8a3b3d798292e27) Previous write of size 8 at 0x7b8800037558 by thread T40: #0 grpc_core::ClientChannel::FilterBasedLoadBalancedCall::PendingBatchesFail(absl::Status, bool (*)(grpc_core::CallCombinerClosureList const&)) third_party/grpc/src/core/ext/filters/client_channel/client_channel.cc:2863:13 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Uclient_Uchannel.so+0x537c7) (BuildId: a8369742d3c435f7f1a244a535f4907c) #1 grpc_core::ClientChannel::FilterBasedLoadBalancedCall::TryPick(bool) third_party/grpc/src/core/ext/filters/client_channel/client_channel.cc:3179:7 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Uclient_Uchannel.so+0x5537f) (BuildId: a8369742d3c435f7f1a244a535f4907c) #2 operator() third_party/grpc/src/core/ext/filters/client_channel/client_channel.cc:3200:5 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Uclient_Uchannel.so+0x6c077) (BuildId: a8369742d3c435f7f1a244a535f4907c) #3 __invoke<(lambda at third_party/grpc/src/core/ext/filters/client_channel/client_channel.cc:3196:46) &> third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__functional/invoke.h:394:23 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Uclient_Uchannel.so+0x6c077) #4 invoke<(lambda at third_party/grpc/src/core/ext/filters/client_channel/client_channel.cc:3196:46) &> third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__functional/invoke.h:539:12 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Uclient_Uchannel.so+0x6c077) #5 InvokeR<void, (lambda at third_party/grpc/src/core/ext/filters/client_channel/client_channel.cc:3196:46) &, void> third_party/absl/functional/internal/any_invocable.h:131:3 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Uclient_Uchannel.so+0x6c077) #6 void absl::internal_any_invocable::LocalInvoker<false, void, grpc_core::ClientChannel::FilterBasedLoadBalancedCall::RetryPickLocked()::$_0&>(absl::internal_any_invocable::TypeErasedState*) third_party/absl/functional/internal/any_invocable.h:301:10 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Uclient_Uchannel.so+0x6c077) #7 operator() third_party/absl/functional/internal/any_invocable.h:855:1 (libST-a5842b3303c3_net_Sgrpc_Sinternal_Ssrc_Score_Sext_Sevent_Uengine_Slibevent_Uengine_Uem_Uimpl.so+0x16a74) (BuildId: 7802a789e61dddac5a33230a2f294046) #8 util::functional::internal::FunctorCallback<Closure, false, absl::AnyInvocable<void ()>, void ()>::Run() util/functional/to_callback_internal.h:87:27 (libST-a5842b3303c3_net_Sgrpc_Sinternal_Ssrc_Score_Sext_Sevent_Uengine_Slibevent_Uengine_Uem_Uimpl.so+0x16a74) #9 void eventmanager::EventManager::RunTask<false>(eventmanager::EventManager::WorkerThread*, eventmanager::TaskInfo*) net/eventmanager/em1/eventmanager.cc:2464:20 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x2ae1b) (BuildId: b3d97518d96e160722658c4aa0795013) #10 void eventmanager::EventManager::RunWorkerLoop<false, false>(eventmanager::EventManager::WorkerThread*) net/eventmanager/em1/eventmanager.cc (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x2a45d) (BuildId: b3d97518d96e160722658c4aa0795013) #11 eventmanager::EventManager::WorkerThread::WorkerMain() net/eventmanager/em1/eventmanager.cc:430:19 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x1d841) (BuildId: b3d97518d96e160722658c4aa0795013) #12 operator() net/eventmanager/em1/eventmanager.cc:396:64 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x28351) (BuildId: b3d97518d96e160722658c4aa0795013) #13 __invoke<(lambda at net/eventmanager/em1/eventmanager.cc:396:55)> third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__functional/invoke.h:394:23 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x28351) #14 invoke<(lambda at net/eventmanager/em1/eventmanager.cc:396:55)> third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__functional/invoke.h:539:12 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x28351) #15 InvokeR<void, (lambda at net/eventmanager/em1/eventmanager.cc:396:55), void> third_party/absl/functional/internal/any_invocable.h:131:3 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x28351) #16 void absl::internal_any_invocable::LocalInvoker<false, void, eventmanager::EventManager::WorkerThread::StartThread(int, int, thread::SchedPolicy, std::__tsan::basic_string_view<char, std::__tsan::char_traits<char>>)::'lambda'()&&>(absl::internal_any_invocable::TypeErasedState*) third_party/absl/functional/internal/any_invocable.h:301:10 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x28351) #17 operator() third_party/absl/functional/internal/any_invocable.h:863:1 (libST-a5842b3303c3_security_Sganpati_Ssharded_Uclient_Slibacl_Ucache.so+0x24c1e) (BuildId: a86eb59aee0be7b5882a1b532922e3ca) #18 ClosureThread::Run() thread/thread.h:459:25 (libST-a5842b3303c3_security_Sganpati_Ssharded_Uclient_Slibacl_Ucache.so+0x24c1e) #19 Thread::ThreadBody(void*) thread/thread.cc:1284:16 (libST-a5842b3303c3_thread_Slibthread.so+0x251e9) (BuildId: a1762ef5c5732b3ad8a3b3d798292e27) Thread T47 'EventManager_Default' (tid=7454, running) created by main thread at: #0 pthread_create third_party/llvm/llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1048:3 (c0da98875dfb74bd2cfcbd76306f382f2d05e1326f778b3a4cefdef1a42a75e7_020000b94930+0x1887b9) (BuildId: e468d94e410a7e13e19b26c4f7277674) #1 Thread::CreatePthread(pthread_attr_t&) thread/thread.cc:485:13 (libST-a5842b3303c3_thread_Slibthread.so+0x24b1d) (BuildId: a1762ef5c5732b3ad8a3b3d798292e27) #2 Thread::Start() thread/thread.cc:667:3 (libST-a5842b3303c3_thread_Slibthread.so+0x2574b) (BuildId: a1762ef5c5732b3ad8a3b3d798292e27) #3 StartThread net/eventmanager/em1/eventmanager.cc:397:14 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x1bbed) (BuildId: b3d97518d96e160722658c4aa0795013) #4 eventmanager::EventManager::EventManager(eventmanager::EventManager::Options const&) net/eventmanager/em1/eventmanager.cc:1274:23 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x1bbed) #5 InitializeDefaultEventManager() net/eventmanager/em1/eventmanager.cc:3042:39 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x25c06) (BuildId: b3d97518d96e160722658c4aa0795013) #6 __invoke<void (*&)()> third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__functional/invoke.h:394:23 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaab9) (BuildId: fe43c5a995ecc22efa243d05ef607fb9) #7 invoke<void (*&)()> third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__functional/invoke.h:539:12 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaab9) #8 void absl::base_internal::CallOnceImpl<void (*&)()>(std::__tsan::atomic<unsigned int>*, absl::base_internal::SchedulingMode, void (*&)()) third_party/absl/base/call_once.h:184:5 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaab9) #9 GoogleOnceInternalInit base/once.cc:23:5 (libST-a5842b3303c3_base_Slibonce.so+0xeee) (BuildId: 6c436371e89d7962c2b544d27465d6a0) #10 GoogleOnceInternalInitSchedCoopAndKernel(std::__tsan::atomic<unsigned int>*, void (*)()) base/once.cc:15:3 (libST-a5842b3303c3_base_Slibonce.so+0xeee) #11 GoogleOnceInit base/once.h:72:5 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x25ad9) (BuildId: b3d97518d96e160722658c4aa0795013) #12 eventmanager::EventManager::DefaultEventManager() net/eventmanager/em1/eventmanager.cc:3059:3 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x25ad9) #13 Default net/eventmanager/em1/eventmanager.h:780:43 (libST-a5842b3303c3_net_Seventmanager_Slibeventmanager_Udefault.so+0x14c5) (BuildId: 2e1dde05e8bce5319856279d896c4405) #14 eventmanager::Default() net/eventmanager/eventmanager_default.cc:22:10 (libST-a5842b3303c3_net_Seventmanager_Slibeventmanager_Udefault.so+0x14c5) #15 grpc::(anonymous namespace)::InitGlobalEventManager() net/grpc/internal/src/core/ext/event_engine/grpc_event_manager.cc:38:23 (libST-a5842b3303c3_net_Sgrpc_Sinternal_Ssrc_Score_Sext_Sevent_Uengine_Slibgrpc_Uevent_Umanager.so+0x1c28) (BuildId: 8bd3c1826fd11ffa128e7839a85c05a0) #16 __invoke<void (*&)()> third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__functional/invoke.h:394:23 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaab9) (BuildId: fe43c5a995ecc22efa243d05ef607fb9) #17 invoke<void (*&)()> third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__functional/invoke.h:539:12 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaab9) #18 void absl::base_internal::CallOnceImpl<void (*&)()>(std::__tsan::atomic<unsigned int>*, absl::base_internal::SchedulingMode, void (*&)()) third_party/absl/base/call_once.h:184:5 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaab9) #19 call_once<void (*&)()> third_party/absl/base/call_once.h:216:5 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaa0b) (BuildId: fe43c5a995ecc22efa243d05ef607fb9) #20 gpr_once_init third_party/grpc/src/core/lib/gpr/sync_abseil.cc:107:3 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaa0b) #21 grpc::GetGrpcEventManager() net/grpc/internal/src/core/ext/event_engine/grpc_event_manager.cc:51:3 (libST-a5842b3303c3_net_Sgrpc_Sinternal_Ssrc_Score_Sext_Sevent_Uengine_Slibgrpc_Uevent_Umanager.so+0x18c1) (BuildId: 8bd3c1826fd11ffa128e7839a85c05a0) #22 check_engine_available(bool) net/grpc/internal/src/core/ext/poller/event_manager_poller/ev_event_manager_linux.cc:1271:21 (libST-a5842b3303c3_net_Sgrpc_Sinternal_Ssrc_Score_Sext_Spoller_Sevent_Umanager_Upoller_Slibev_Uevent_Umanager_Ulinux.so+0x4fcb) (BuildId: 620adb83259aed32440614774678b3a1) #23 try_engine third_party/grpc/src/core/lib/iomgr/ev_posix.cc:141:9 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Ubase.so+0xad888) (BuildId: 3cb6b043bd10222927301e33a3760646) #24 operator() third_party/grpc/src/core/lib/iomgr/ev_posix.cc:184:7 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Ubase.so+0xad888) #25 grpc_event_engine_init()::$_0::__invoke() third_party/grpc/src/core/lib/iomgr/ev_posix.cc:175:35 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Ubase.so+0xad888) #26 __invoke<void (*&)()> third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__functional/invoke.h:394:23 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaab9) (BuildId: fe43c5a995ecc22efa243d05ef607fb9) #27 invoke<void (*&)()> third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__functional/invoke.h:539:12 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaab9) #28 void absl::base_internal::CallOnceImpl<void (*&)()>(std::__tsan::atomic<unsigned int>*, absl::base_internal::SchedulingMode, void (*&)()) third_party/absl/base/call_once.h:184:5 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaab9) #29 call_once<void (*&)()> third_party/absl/base/call_once.h:216:5 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaa0b) (BuildId: fe43c5a995ecc22efa243d05ef607fb9) #30 gpr_once_init third_party/grpc/src/core/lib/gpr/sync_abseil.cc:107:3 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaa0b) #31 grpc_event_engine_init() third_party/grpc/src/core/lib/iomgr/ev_posix.cc:175:3 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Ubase.so+0xacab1) (BuildId: 3cb6b043bd10222927301e33a3760646) #32 iomgr_platform_init() third_party/grpc/src/core/lib/iomgr/iomgr_posix.cc:43:3 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Ubase.so+0xaf1d0) (BuildId: 3cb6b043bd10222927301e33a3760646) #33 grpc_iomgr_platform_init() third_party/grpc/src/core/lib/iomgr/iomgr_internal.cc:35:35 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibexec_Uctx.so+0x688b) (BuildId: cafb035b95f6c781e93706a71dce95f3) #34 grpc_iomgr_init() third_party/grpc/src/core/lib/iomgr/iomgr.cc:66:3 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Ubase.so+0xae777) (BuildId: 3cb6b043bd10222927301e33a3760646) #35 grpc_init third_party/grpc/src/core/lib/surface/init.cc:144:5 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc.so+0x4518) (BuildId: 778b43e2e8f8c132a7acf9b3da117fff) #36 k3::(anonymous namespace)::GrpcInitHackEnvironment::SetUp() storage/k3/grpc/grpc_server_test.cc:144:27 (c0da98875dfb74bd2cfcbd76306f382f2d05e1326f778b3a4cefdef1a42a75e7_020000b94930+0x212d21) (BuildId: e468d94e410a7e13e19b26c4f7277674) #37 SetUpEnvironment third_party/googletest/googletest/src/gtest.cc:5763:55 (libST-a5842b3303c3_third_Uparty_Sgoogletest_Slibgtest.so+0x6f58b) (BuildId: ec51e8d56b1390e9939f663d4700c7da) #38 for_each<std::__tsan::__wrap_iter<testing::Environment *const *>, void (*)(testing::Environment *)> third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__algorithm/for_each.h:26:5 (libST-a5842b3303c3_third_Uparty_Sgoogletest_Slibgtest.so+0x6f58b) #39 ForEach<std::__tsan::vector<testing::Environment *, std::__tsan::allocator<testing::Environment *> >, void (*)(testing::Environment *)> third_party/googletest/googletest/src/gtest-internal-inl.h:288:3 (libST-a5842b3303c3_third_Uparty_Sgoogletest_Slibgtest.so+0x6f58b) #40 testing::internal::UnitTestImpl::RunAllTests() third_party/googletest/googletest/src/gtest.cc:5876:9 (libST-a5842b3303c3_third_Uparty_Sgoogletest_Slibgtest.so+0x6f58b) #41 HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> third_party/googletest/googletest/src/gtest.cc (libST-a5842b3303c3_third_Uparty_Sgoogletest_Slibgtest.so+0x6ee41) (BuildId: ec51e8d56b1390e9939f663d4700c7da) #42 testing::UnitTest::Run() third_party/googletest/googletest/src/gtest.cc:5464:10 (libST-a5842b3303c3_third_Uparty_Sgoogletest_Slibgtest.so+0x6ee41) #43 RUN_ALL_TESTS third_party/googletest/googletest/include/gtest/gtest.h:2329:73 (c0da98875dfb74bd2cfcbd76306f382f2d05e1326f778b3a4cefdef1a42a75e7_020000b94930+0x2aa83a) (BuildId: e468d94e410a7e13e19b26c4f7277674) #44 main testing/base/internal/gunit_main.cc:86:10 (c0da98875dfb74bd2cfcbd76306f382f2d05e1326f778b3a4cefdef1a42a75e7_020000b94930+0x2aa83a) Thread T40 'EventManager_Default' (tid=7447, running) created by main thread at: #0 pthread_create third_party/llvm/llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1048:3 (c0da98875dfb74bd2cfcbd76306f382f2d05e1326f778b3a4cefdef1a42a75e7_020000b94930+0x1887b9) (BuildId: e468d94e410a7e13e19b26c4f7277674) #1 Thread::CreatePthread(pthread_attr_t&) thread/thread.cc:485:13 (libST-a5842b3303c3_thread_Slibthread.so+0x24b1d) (BuildId: a1762ef5c5732b3ad8a3b3d798292e27) #2 Thread::Start() thread/thread.cc:667:3 (libST-a5842b3303c3_thread_Slibthread.so+0x2574b) (BuildId: a1762ef5c5732b3ad8a3b3d798292e27) #3 StartThread net/eventmanager/em1/eventmanager.cc:397:14 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x1bbed) (BuildId: b3d97518d96e160722658c4aa0795013) #4 eventmanager::EventManager::EventManager(eventmanager::EventManager::Options const&) net/eventmanager/em1/eventmanager.cc:1274:23 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x1bbed) #5 InitializeDefaultEventManager() net/eventmanager/em1/eventmanager.cc:3042:39 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x25c06) (BuildId: b3d97518d96e160722658c4aa0795013) #6 __invoke<void (*&)()> third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__functional/invoke.h:394:23 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaab9) (BuildId: fe43c5a995ecc22efa243d05ef607fb9) #7 invoke<void (*&)()> third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__functional/invoke.h:539:12 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaab9) #8 void absl::base_internal::CallOnceImpl<void (*&)()>(std::__tsan::atomic<unsigned int>*, absl::base_internal::SchedulingMode, void (*&)()) third_party/absl/base/call_once.h:184:5 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaab9) #9 GoogleOnceInternalInit base/once.cc:23:5 (libST-a5842b3303c3_base_Slibonce.so+0xeee) (BuildId: 6c436371e89d7962c2b544d27465d6a0) #10 GoogleOnceInternalInitSchedCoopAndKernel(std::__tsan::atomic<unsigned int>*, void (*)()) base/once.cc:15:3 (libST-a5842b3303c3_base_Slibonce.so+0xeee) #11 GoogleOnceInit base/once.h:72:5 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x25ad9) (BuildId: b3d97518d96e160722658c4aa0795013) #12 eventmanager::EventManager::DefaultEventManager() net/eventmanager/em1/eventmanager.cc:3059:3 (libST-a5842b3303c3_net_Seventmanager_Sem1_Slibem1.so+0x25ad9) #13 Default net/eventmanager/em1/eventmanager.h:780:43 (libST-a5842b3303c3_net_Seventmanager_Slibeventmanager_Udefault.so+0x14c5) (BuildId: 2e1dde05e8bce5319856279d896c4405) #14 eventmanager::Default() net/eventmanager/eventmanager_default.cc:22:10 (libST-a5842b3303c3_net_Seventmanager_Slibeventmanager_Udefault.so+0x14c5) #15 grpc::(anonymous namespace)::InitGlobalEventManager() net/grpc/internal/src/core/ext/event_engine/grpc_event_manager.cc:38:23 (libST-a5842b3303c3_net_Sgrpc_Sinternal_Ssrc_Score_Sext_Sevent_Uengine_Slibgrpc_Uevent_Umanager.so+0x1c28) (BuildId: 8bd3c1826fd11ffa128e7839a85c05a0) #16 __invoke<void (*&)()> third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__functional/invoke.h:394:23 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaab9) (BuildId: fe43c5a995ecc22efa243d05ef607fb9) #17 invoke<void (*&)()> third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__functional/invoke.h:539:12 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaab9) #18 void absl::base_internal::CallOnceImpl<void (*&)()>(std::__tsan::atomic<unsigned int>*, absl::base_internal::SchedulingMode, void (*&)()) third_party/absl/base/call_once.h:184:5 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaab9) #19 call_once<void (*&)()> third_party/absl/base/call_once.h:216:5 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaa0b) (BuildId: fe43c5a995ecc22efa243d05ef607fb9) #20 gpr_once_init third_party/grpc/src/core/lib/gpr/sync_abseil.cc:107:3 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaa0b) #21 grpc::GetGrpcEventManager() net/grpc/internal/src/core/ext/event_engine/grpc_event_manager.cc:51:3 (libST-a5842b3303c3_net_Sgrpc_Sinternal_Ssrc_Score_Sext_Sevent_Uengine_Slibgrpc_Uevent_Umanager.so+0x18c1) (BuildId: 8bd3c1826fd11ffa128e7839a85c05a0) #22 check_engine_available(bool) net/grpc/internal/src/core/ext/poller/event_manager_poller/ev_event_manager_linux.cc:1271:21 (libST-a5842b3303c3_net_Sgrpc_Sinternal_Ssrc_Score_Sext_Spoller_Sevent_Umanager_Upoller_Slibev_Uevent_Umanager_Ulinux.so+0x4fcb) (BuildId: 620adb83259aed32440614774678b3a1) #23 try_engine third_party/grpc/src/core/lib/iomgr/ev_posix.cc:141:9 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Ubase.so+0xad888) (BuildId: 3cb6b043bd10222927301e33a3760646) #24 operator() third_party/grpc/src/core/lib/iomgr/ev_posix.cc:184:7 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Ubase.so+0xad888) #25 grpc_event_engine_init()::$_0::__invoke() third_party/grpc/src/core/lib/iomgr/ev_posix.cc:175:35 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Ubase.so+0xad888) #26 __invoke<void (*&)()> third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__functional/invoke.h:394:23 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaab9) (BuildId: fe43c5a995ecc22efa243d05ef607fb9) #27 invoke<void (*&)()> third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__functional/invoke.h:539:12 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaab9) #28 void absl::base_internal::CallOnceImpl<void (*&)()>(std::__tsan::atomic<unsigned int>*, absl::base_internal::SchedulingMode, void (*&)()) third_party/absl/base/call_once.h:184:5 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaab9) #29 call_once<void (*&)()> third_party/absl/base/call_once.h:216:5 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaa0b) (BuildId: fe43c5a995ecc22efa243d05ef607fb9) #30 gpr_once_init third_party/grpc/src/core/lib/gpr/sync_abseil.cc:107:3 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgpr.so+0xaa0b) #31 grpc_event_engine_init() third_party/grpc/src/core/lib/iomgr/ev_posix.cc:175:3 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Ubase.so+0xacab1) (BuildId: 3cb6b043bd10222927301e33a3760646) #32 iomgr_platform_init() third_party/grpc/src/core/lib/iomgr/iomgr_posix.cc:43:3 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Ubase.so+0xaf1d0) (BuildId: 3cb6b043bd10222927301e33a3760646) #33 grpc_iomgr_platform_init() third_party/grpc/src/core/lib/iomgr/iomgr_internal.cc:35:35 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibexec_Uctx.so+0x688b) (BuildId: cafb035b95f6c781e93706a71dce95f3) #34 grpc_iomgr_init() third_party/grpc/src/core/lib/iomgr/iomgr.cc:66:3 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc_Ubase.so+0xae777) (BuildId: 3cb6b043bd10222927301e33a3760646) #35 grpc_init third_party/grpc/src/core/lib/surface/init.cc:144:5 (libST-a5842b3303c3_third_Uparty_Sgrpc_Sgoogle_Slibgrpc.so+0x4518) (BuildId: 778b43e2e8f8c132a7acf9b3da117fff) #36 k3::(anonymous namespace)::GrpcInitHackEnvironment::SetUp() storage/k3/grpc/grpc_server_test.cc:144:27 (c0da98875dfb74bd2cfcbd76306f382f2d05e1326f778b3a4cefdef1a42a75e7_020000b94930+0x212d21) (BuildId: e468d94e410a7e13e19b26c4f7277674) #37 SetUpEnvironment third_party/googletest/googletest/src/gtest.cc:5763:55 (libST-a5842b3303c3_third_Uparty_Sgoogletest_Slibgtest.so+0x6f58b) (BuildId: ec51e8d56b1390e9939f663d4700c7da) #38 for_each<std::__tsan::__wrap_iter<testing::Environment *const *>, void (*)(testing::Environment *)> third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__algorithm/for_each.h:26:5 (libST-a5842b3303c3_third_Uparty_Sgoogletest_Slibgtest.so+0x6f58b) #39 ForEach<std::__tsan::vector<testing::Environment *, std::__tsan::allocator<testing::Environment *> >, void (*)(testing::Environment *)> third_party/googletest/googletest/src/gtest-internal-inl.h:288:3 (libST-a5842b3303c3_third_Uparty_Sgoogletest_Slibgtest.so+0x6f58b) #40 testing::internal::UnitTestImpl::RunAllTests() third_party/googletest/googletest/src/gtest.cc:5876:9 (libST-a5842b3303c3_third_Uparty_Sgoogletest_Slibgtest.so+0x6f58b) #41 HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> third_party/googletest/googletest/src/gtest.cc (libST-a5842b3303c3_third_Uparty_Sgoogletest_Slibgtest.so+0x6ee41) (BuildId: ec51e8d56b1390e9939f663d4700c7da) #42 testing::UnitTest::Run() third_party/googletest/googletest/src/gtest.cc:5464:10 (libST-a5842b3303c3_third_Uparty_Sgoogletest_Slibgtest.so+0x6ee41) #43 RUN_ALL_TESTS third_party/googletest/googletest/include/gtest/gtest.h:2329:73 (c0da98875dfb74bd2cfcbd76306f382f2d05e1326f778b3a4cefdef1a42a75e7_020000b94930+0x2aa83a) (BuildId: e468d94e410a7e13e19b26c4f7277674) #44 main testing/base/internal/gunit_main.cc:86:10 (c0da98875dfb74bd2cfcbd76306f382f2d05e1326f778b3a4cefdef1a42a75e7_020000b94930+0x2aa83a) SUMMARY: ThreadSanitizer: data race third_party/grpc/src/core/ext/filters/client_channel/client_channel.cc:2780:5 in grpc_core::ClientChannel::FilterBasedLoadBalancedCall::~FilterBasedLoadBalancedCall() ```
BusyJay
pushed a commit
that referenced
this pull request
Aug 1, 2023
Long story here: CallbackAlternativeCQ operates a thread pool which processes a completion queue and then directly invokes the completion function in the same thread. This thread pool is initialized on first Ref() and unallocated on last Unref(). When running an in-process synchronous server (as we do for tests, using this https://github.com/google/tensorstore/blob/master/tensorstore/internal/grpc/grpc_mock.h), called by an async() interface caller, if the async() callback happens to drop the last reference to the grpc Channel, then the channel shutdown will attempt to run in one of the the CallbackAlternativeCQ threads. This will cause a deadlock/race condition, as `CallbackAlternativeCQ` is not designed to shutdown itself. When this deadlock happens, `pthread_join(pthread_id_)` will return `EDEADLK` and the thread will keep running. However `EDEADLK` is silently ignored by Join() so `CallbackAlternativeCQ::Unref` will continue to delete the underlying grpc_completion_queue, leading to a `SIGSEGV` later in the process. https://github.com/grpc/grpc/blob/97ba9871324cb68b93f22fd1860934392cd476ee/src/cpp/common/completion_queue_cc.cc#L115 This adds an assert that pthread_join succeeded, which is useful as it avoids a later SIGSEBV. Alternatively, the thread implementation could gpr_log the errorcode before asserting. Example backtrace of crash: frame #0: 0x0000000194f1e868 libsystem_kernel.dylib`__pthread_kill + 8 frame #1: 0x0000000194f55cec libsystem_pthread.dylib`pthread_kill + 288 frame #2: 0x0000000194e8e2c8 libsystem_c.dylib`abort + 180 frame #3: 0x0000000194e8d620 libsystem_c.dylib`__assert_rtn + 272 frame #4: 0x0000000100a64f50 grpc_kvstore_test`grpc_core::(anonymous namespace)::ThreadInternalsPosix::Join() + 188 frame #5: 0x00000001009c5dd0 grpc_kvstore_test`grpc_core::Thread::Join() + 56 frame #6: 0x0000000100154474 grpc_kvstore_test`grpc::(anonymous namespace)::CallbackAlternativeCQ::Unref() + 216 frame #7: 0x0000000100154390 grpc_kvstore_test`grpc::CompletionQueue::ReleaseCallbackAlternativeCQ(grpc::CompletionQueue*) + 120 frame #8: 0x000000010014130c grpc_kvstore_test`grpc::Channel::~Channel() + 220 frame #9: 0x00000001001413c8 grpc_kvstore_test`grpc::Channel::~Channel() + 28 frame #10: 0x000000010014d678 grpc_kvstore_test`std::__1::default_delete<grpc::Channel>::operator()(grpc::Channel*) const + 44 frame #11: 0x000000010014d358 grpc_kvstore_test`std::__1::__shared_ptr_pointer<grpc::Channel*, std::__1::shared_ptr<grpc::Channel>::__shared_ptr_default_delete<grpc::Channel, grpc::Channel>, std::__1::allocator<grpc::Channel> >::__on_zero_shared() + 72 frame #12: 0x000000010002ab5c grpc_kvstore_test`std::__1::__shared_count::__release_shared() + 60 frame #13: 0x000000010002ab00 grpc_kvstore_test`std::__1::__shared_weak_count::__release_shared() + 28 frame #14: 0x000000010002aad0 grpc_kvstore_test`std::__1::shared_ptr<grpc::ServerCredentials>::~shared_ptr() + 56 frame #15: 0x00000001000053ec grpc_kvstore_test`std::__1::shared_ptr<tensorstore_grpc::kvstore::grpc_gen::KvStoreService::Stub>::~shared_ptr() + 28 frame #16: 0x000000010014653c grpc_kvstore_test`grpc::ClientContext::~ClientContext() + 356 frame #17: 0x0000000100146570 grpc_kvstore_test`grpc::ClientContext::~ClientContext() + 28 frame #18: 0x00000001000ab000 grpc_kvstore_test`tensorstore::(anonymous namespace)::ReadTask::~ReadTask() + 68 frame #19: 0x00000001000aae90 grpc_kvstore_test`tensorstore::(anonymous namespace)::ReadTask::~ReadTask() + 28 frame #20: 0x00000001000aae18 grpc_kvstore_test`tensorstore::internal::intrusive_ptr_decrement(tensorstore::internal::AtomicReferenceCount<tensorstore::(anonymous namespace)::ReadTask> const*) + 68 frame #21: 0x00000001000aadc8 grpc_kvstore_test`void tensorstore::internal::DefaultIntrusivePtrTraits::decrement<tensorstore::(anonymous namespace)::ReadTask*>(tensorstore::(anonymous namespace)::ReadTask*) + 24 frame #22: 0x00000001000aad9c grpc_kvstore_test`tensorstore::internal::IntrusivePtr<tensorstore::(anonymous namespace)::ReadTask, tensorstore::internal::DefaultIntrusivePtrTraits>::~IntrusivePtr() + 52 frame #23: 0x00000001000a5994 grpc_kvstore_test`tensorstore::internal::IntrusivePtr<tensorstore::(anonymous namespace)::ReadTask, tensorstore::internal::DefaultIntrusivePtrTraits>::~IntrusivePtr() + 28 frame #24: 0x00000001000aac24 grpc_kvstore_test`tensorstore::(anonymous namespace)::ReadTask::Start(tensorstore_grpc::kvstore::grpc_gen::KvStoreService::StubInterface*, absl::Time)::'lambda'(grpc::Status)::~() + 40 frame #25: 0x00000001000a6280 grpc_kvstore_test`tensorstore::(anonymous namespace)::ReadTask::Start(tensorstore_grpc::kvstore::grpc_gen::KvStoreService::StubInterface*, absl::Time)::'lambda'(grpc::Status)::~() + 28 frame #26: 0x00000001000a84ac grpc_kvstore_test`std::__1::__compressed_pair_elem<tensorstore::(anonymous namespace)::ReadTask::Start(tensorstore_grpc::kvstore::grpc_gen::KvStoreService::StubInterface*, absl::Time)::'lambda'(grpc::Status), 0, false>::~__compressed_pair_elem() + 28 frame #27: 0x00000001000a86c0 grpc_kvstore_test`std::__1::__compressed_pair<tensorstore::(anonymous namespace)::ReadTask::Start(tensorstore_grpc::kvstore::grpc_gen::KvStoreService::StubInterface*, absl::Time)::'lambda'(grpc::Status), std::__1::allocator<tensorstore::(anonymous namespace)::ReadTask::Start(tensorstore_grpc::kvstore::grpc_gen::KvStoreService::StubInterface*, absl::Time)::'lambda'(grpc::Status)> >::~__compressed_pair() + 28 frame #28: 0x00000001000a8694 grpc_kvstore_test`std::__1::__compressed_pair<tensorstore::(anonymous namespace)::ReadTask::Start(tensorstore_grpc::kvstore::grpc_gen::KvStoreService::StubInterface*, absl::Time)::'lambda'(grpc::Status), std::__1::allocator<tensorstore::(anonymous namespace)::ReadTask::Start(tensorstore_grpc::kvstore::grpc_gen::KvStoreService::StubInterface*, absl::Time)::'lambda'(grpc::Status)> >::~__compressed_pair() + 28 frame #29: 0x00000001000a990c grpc_kvstore_test`std::__1::__function::__alloc_func<tensorstore::(anonymous namespace)::ReadTask::Start(tensorstore_grpc::kvstore::grpc_gen::KvStoreService::StubInterface*, absl::Time)::'lambda'(grpc::Status), std::__1::allocator<tensorstore::(anonymous namespace)::ReadTask::Start(tensorstore_grpc::kvstore::grpc_gen::KvStoreService::StubInterface*, absl::Time)::'lambda'(grpc::Status)>, void (grpc::Status)>::destroy() + 24 frame #30: 0x00000001000a7ea0 grpc_kvstore_test`std::__1::__function::__func<tensorstore::(anonymous namespace)::ReadTask::Start(tensorstore_grpc::kvstore::grpc_gen::KvStoreService::StubInterface*, absl::Time)::'lambda'(grpc::Status), std::__1::allocator<tensorstore::(anonymous namespace)::ReadTask::Start(tensorstore_grpc::kvstore::grpc_gen::KvStoreService::StubInterface*, absl::Time)::'lambda'(grpc::Status)>, void (grpc::Status)>::destroy() + 28 frame #31: 0x00000001000aabbc grpc_kvstore_test`std::__1::__function::__value_func<void (grpc::Status)>::~__value_func() + 68 frame #32: 0x00000001000aab68 grpc_kvstore_test`std::__1::__function::__value_func<void (grpc::Status)>::~__value_func() + 28 frame #33: 0x00000001000aab3c grpc_kvstore_test`std::__1::function<void (grpc::Status)>::~function() + 28 frame #34: 0x00000001000a6254 grpc_kvstore_test`std::__1::function<void (grpc::Status)>::~function() + 28 frame #35: 0x0000000100108ae0 grpc_kvstore_test`grpc::internal::CallbackWithStatusTag::Run(bool) + 368 frame #36: 0x0000000100108964 grpc_kvstore_test`grpc::internal::CallbackWithStatusTag::StaticRun(grpc_completion_queue_functor*, int) + 44 frame #37: 0x0000000100154cb0 grpc_kvstore_test`grpc::(anonymous namespace)::CallbackAlternativeCQ::ThreadLoop(void*) + 356 frame #38: 0x0000000100a650b8 grpc_kvstore_test`grpc_core::(anonymous namespace)::ThreadInternalsPosix::ThreadInternalsPosix(char const*, void (*)(void*), void*, bool*, grpc_core::Thread::Options const&)::'lambda'(void*)::operator()(void*) const + 240 frame #39: 0x0000000100a64fbc grpc_kvstore_test`grpc_core::(anonymous namespace)::ThreadInternalsPosix::ThreadInternalsPosix(char const*, void (*)(void*), void*, bool*, grpc_core::Thread::Options const&)::'lambda'(void*)::__invoke(void*) + 28 frame #40: 0x0000000194f5606c libsystem_pthread.dylib`_pthread_start + 148
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.