"Cannot expire connection, it is owned by a different thread" in Rails 7.2 tests #52973
Description
Steps to reproduce
After upgrading 7.1.4 -> 7.2.1, we've started seeing some sporadic failures in our test suite. I've narrowed it down to happening when a Capybara system test is followed by a unit test that sets use_transactional_tests = false, followed by another Capybara system test. Capybara then fails to load pages, with errors along the lines of:
ActiveRecord::ActiveRecordError:
Cannot expire connection, it is owned by a different thread: #<Thread:0x000000010051b0e8 sleep>. Current thread: #<Thread:0x0000000110dc03c8 /Users/jon/Developer/web/vendor/bundle/nix-bundle/ruby/3.2.0/gems/puma-6.4.2/lib/puma/thread_pool.rb:113 run>.
# ./vendor/bundle/ruby/3.2.0/gems/activerecord-7.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:316:in `expire'
# ./vendor/bundle/ruby/3.2.0/gems/activerecord-7.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:555:in `block (3 levels) in checkin'
# ./vendor/bundle/ruby/3.2.0/gems/activesupport-7.2.1/lib/active_support/callbacks.rb:110:in `run_callbacks'
# ./vendor/bundle/ruby/3.2.0/gems/activesupport-7.2.1/lib/active_support/callbacks.rb:913:in `_run_checkin_callbacks'
# ./vendor/bundle/ruby/3.2.0/gems/activerecord-7.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:554:in `block (2 levels) in checkin'
# /nix/store/cvwv2rkas498bv81lrd2wxdr50kfkqhv-ruby-3.2.4/lib/ruby/3.2.0/monitor.rb:202:in `synchronize'
# /nix/store/cvwv2rkas498bv81lrd2wxdr50kfkqhv-ruby-3.2.4/lib/ruby/3.2.0/monitor.rb:202:in `mon_synchronize'
# ./vendor/bundle/ruby/3.2.0/gems/activerecord-7.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:551:in `block in checkin'
# ./vendor/bundle/ruby/3.2.0/gems/activesupport-7.2.1/lib/active_support/concurrency/null_lock.rb:9:in `synchronize'
# ./vendor/bundle/ruby/3.2.0/gems/activerecord-7.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:550:in `checkin'
# ./vendor/bundle/ruby/3.2.0/gems/activerecord-7.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:367:in `release_connection'
# ./vendor/bundle/ruby/3.2.0/gems/activerecord-7.2.1/lib/active_record/query_cache.rb:48:in `block in complete'
# ./vendor/bundle/ruby/3.2.0/gems/activerecord-7.2.1/lib/active_record/connection_adapters/abstract/connection_handler.rb:108:in `block (2 levels) in each_connection_pool'
# ./vendor/bundle/ruby/3.2.0/gems/activerecord-7.2.1/lib/active_record/connection_adapters/pool_manager.rb:31:in `each_value'
# ./vendor/bundle/ruby/3.2.0/gems/activerecord-7.2.1/lib/active_record/connection_adapters/pool_manager.rb:31:in `block in each_pool_config'
# ./vendor/bundle/ruby/3.2.0/gems/activerecord-7.2.1/lib/active_record/connection_adapters/pool_manager.rb:30:in `each_value'
# ./vendor/bundle/ruby/3.2.0/gems/activerecord-7.2.1/lib/active_record/connection_adapters/pool_manager.rb:30:in `each_pool_config'
# ./vendor/bundle/ruby/3.2.0/gems/activerecord-7.2.1/lib/active_record/connection_adapters/abstract/connection_handler.rb:107:in `block in each_connection_pool'
# ./vendor/bundle/ruby/3.2.0/gems/concurrent-ruby-1.3.4/lib/concurrent-ruby/concurrent/map.rb:265:in `block in each_value'
# ./vendor/bundle/ruby/3.2.0/gems/concurrent-ruby-1.3.4/lib/concurrent-ruby/concurrent/collection/map/non_concurrent_map_backend.rb:101:in `block in each_pair'
# ./vendor/bundle/ruby/3.2.0/gems/concurrent-ruby-1.3.4/lib/concurrent-ruby/concurrent/collection/map/non_concurrent_map_backend.rb:100:in `each_pair'
# ./vendor/bundle/ruby/3.2.0/gems/concurrent-ruby-1.3.4/lib/concurrent-ruby/concurrent/collection/map/non_concurrent_map_backend.rb:100:in `each_pair'
# ./vendor/bundle/ruby/3.2.0/gems/concurrent-ruby-1.3.4/lib/concurrent-ruby/concurrent/map.rb:276:in `each_pair'
# ./vendor/bundle/ruby/3.2.0/gems/concurrent-ruby-1.3.4/lib/concurrent-ruby/concurrent/map.rb:265:in `each_value'
# ./vendor/bundle/ruby/3.2.0/gems/activerecord-7.2.1/lib/active_record/connection_adapters/abstract/connection_handler.rb:106:in `each_connection_pool'
# ./vendor/bundle/ruby/3.2.0/gems/activerecord-7.2.1/lib/active_record/query_cache.rb:47:in `complete'
# ./vendor/bundle/ruby/3.2.0/gems/activesupport-7.2.1/lib/active_support/execution_wrapper.rb:36:in `before'
# ./vendor/bundle/ruby/3.2.0/gems/activesupport-7.2.1/lib/active_support/callbacks.rb:385:in `block in make_lambda'
# ./vendor/bundle/ruby/3.2.0/gems/activesupport-7.2.1/lib/active_support/callbacks.rb:179:in `block in call'
# ./vendor/bundle/ruby/3.2.0/gems/activesupport-7.2.1/lib/active_support/callbacks.rb:668:in `block (2 levels) in default_terminator'
# ./vendor/bundle/ruby/3.2.0/gems/activesupport-7.2.1/lib/active_support/callbacks.rb:667:in `catch'
# ./vendor/bundle/ruby/3.2.0/gems/activesupport-7.2.1/lib/active_support/callbacks.rb:667:in `block in default_terminator'
# ./vendor/bundle/ruby/3.2.0/gems/activesupport-7.2.1/lib/active_support/callbacks.rb:180:in `call'
# ./vendor/bundle/ruby/3.2.0/gems/activesupport-7.2.1/lib/active_support/callbacks.rb:559:in `block in invoke_before'
# ./vendor/bundle/ruby/3.2.0/gems/activesupport-7.2.1/lib/active_support/callbacks.rb:559:in `each'
# ./vendor/bundle/ruby/3.2.0/gems/activesupport-7.2.1/lib/active_support/callbacks.rb:559:in `invoke_before'
# ./vendor/bundle/ruby/3.2.0/gems/activesupport-7.2.1/lib/active_support/callbacks.rb:109:in `run_callbacks'
# ./vendor/bundle/ruby/3.2.0/gems/activesupport-7.2.1/lib/active_support/execution_wrapper.rb:142:in `complete'
# ./vendor/bundle/ruby/3.2.0/gems/activesupport-7.2.1/lib/active_support/execution_wrapper.rb:136:in `complete!'
# ./vendor/bundle/ruby/3.2.0/gems/actionpack-7.2.1/lib/action_dispatch/middleware/executor.rb:28:in `ensure in call'
# ./vendor/bundle/ruby/3.2.0/gems/actionpack-7.2.1/lib/action_dispatch/middleware/executor.rb:28:in `call'
# ./vendor/bundle/ruby/3.2.0/gems/actionpack-7.2.1/lib/action_dispatch/middleware/static.rb:27:in `call'
# ./vendor/bundle/ruby/3.2.0/gems/rack-3.1.7/lib/rack/sendfile.rb:114:in `call'
# ./vendor/bundle/ruby/3.2.0/gems/actionpack-7.2.1/lib/action_dispatch/middleware/host_authorization.rb:143:in `call'
# ./vendor/bundle/ruby/3.2.0/gems/rack-cors-2.0.2/lib/rack/cors.rb:102:in `call'
# ./vendor/bundle/ruby/3.2.0/gems/rack-mini-profiler-3.3.1/lib/mini_profiler.rb:191:in `call'
# ./vendor/bundle/ruby/3.2.0/gems/railties-7.2.1/lib/rails/engine.rb:535:in `call'
# ./vendor/bundle/ruby/3.2.0/gems/rack-3.1.7/lib/rack/urlmap.rb:76:in `block in call'
# ./vendor/bundle/ruby/3.2.0/gems/rack-3.1.7/lib/rack/urlmap.rb:60:in `each'
# ./vendor/bundle/ruby/3.2.0/gems/rack-3.1.7/lib/rack/urlmap.rb:60:in `call'
# ./vendor/bundle/ruby/3.2.0/gems/rack-3.1.7/lib/rack/builder.rb:277:in `call'
# ./vendor/bundle/ruby/3.2.0/gems/capybara-3.40.0/lib/capybara/server/middleware.rb:60:in `call'
# ./vendor/bundle/ruby/3.2.0/gems/puma-6.4.2/lib/puma/configuration.rb:272:in `call'
# ./vendor/bundle/ruby/3.2.0/gems/puma-6.4.2/lib/puma/request.rb:100:in `block in handle_request'
# ./vendor/bundle/ruby/3.2.0/gems/puma-6.4.2/lib/puma/thread_pool.rb:378:in `with_force_shutdown'
# ./vendor/bundle/ruby/3.2.0/gems/puma-6.4.2/lib/puma/request.rb:99:in `handle_request'
# ./vendor/bundle/ruby/3.2.0/gems/puma-6.4.2/lib/puma/server.rb:464:in `process_client'
# ./vendor/bundle/ruby/3.2.0/gems/puma-6.4.2/lib/puma/server.rb:245:in `block in run'
# ./vendor/bundle/ruby/3.2.0/gems/puma-6.4.2/lib/puma/thread_pool.rb:155:in `block in spawn_thread'
The puma thread seems to be trying to expire a connection owned by the main thread.
I've also occasionally seen the error reported being "Cannot expire connection, it is not currently leased".
I've bisected the Rails issue down to starting at #51725 - prior to that commit, I don't see any of these errors.
The transactionless test could be as simple as:
RSpec.describe User, type: :model do
self.use_transactional_tests = false
it "can handle simultaneous transactions" do
Array.new(5) {
Thread.new { User.first }
}.map(&:join)
end
endI'm still working on a reproduction script, but having tried to narrow it down for several hours without any luck I thought I'd post this in the hope that someone might immediately recognise the problem.
@casperisfine I don't suppose this rings any bells for you? Sorry, I appreciate it's a vague question - I'll carry on trying to come up with a demo app if not.
System configuration
Rails version: 7.1.4
Ruby version: 3.2.4