Fix socket leak on monitor wakeup `NoMethodError` in `Reactor#select_loop` #3696

joshuay03 · 2025-08-18T19:36:44Z

Description

This fixes the linked issue by narrowing the scope of the NoMethodError rescue logic in the reactor's select loop. Previously, the rescue logic targeted the entire loop iteration including wakeup processing. Now it only protects the @selector.select call itself, which was the original intention.

When a lowlevel_error_handler returns an invalid response (like a string instead of an array), the reactor thread would die during wakeup processing but leave the selector open, causing file descriptor leaks. By moving the rescue block to only wrap @selector.select, exceptions during wakeup processing no longer terminate the reactor thread, preventing file descriptor exhaustion under high load.

The NoMethodError rescue logic was originally added in commit fd259b7 to handle rare selector exceptions. It seems like it shouldn't be required, and that there's some underlying issue that should be fixed. I'd be happy to look into this in a follow-up so we can remove it altogether, but this patch should do for now.

I will come back to add a test for this when I get a chance. I thought I'd open it early for feedback.

Manual testing:

master

# Initial
Tue Aug 19 05:06:17 AEST 2025
Total FDs:       63

puma [master] ./slow_body_http_requests.sh valid
Sending 500 slow body HTTP requests (valid)...

# Increases
Tue Aug 19 05:06:38 AEST 2025
Total FDs:      563

# Drops
Tue Aug 19 05:06:51 AEST 2025
Total FDs:       63

puma [master] ./slow_body_http_requests.sh invalid
Sending 500 slow body HTTP requests (invalid)...

# Increases
Tue Aug 19 05:08:23 AEST 2025
Total FDs:      564

# Does not drop
Tue Aug 19 05:08:59 AEST 2025
Total FDs:      564

puma [master] ./slow_body_http_requests.sh valid
Sending 500 slow body HTTP requests (valid)...

# Increases
Tue Aug 19 05:09:40 AEST 2025
Total FDs:     1064

# Does not drop
Tue Aug 19 05:10:24 AEST 2025
Total FDs:     1064

puma [master] ./slow_body_http_requests.sh valid
Sending 500 slow body HTTP requests (valid)...

# Increases
Tue Aug 19 05:10:42 AEST 2025
Total FDs:     1564

# Does not drop
Tue Aug 19 05:11:09 AEST 2025
Total FDs:     1564

puma [master] be bin/puma -t 3 -w 2 -C puma.rb app.ru
[20946] Puma starting in cluster mode...
[20946] * Puma version: 7.0.0.pre1 ("Romantic Warrior")
[20946] * Ruby version: ruby 3.4.5 (2025-07-16 revision 20cda200d3) +YJIT +PRISM [arm64-darwin25]
[20946] *  Min threads: 3
[20946] *  Max threads: 3
[20946] *  Environment: development
[20946] *   Master PID: 20946
[20946] *      Workers: 2
[20946] *     Restarts: (✔) hot (✔) phased (✖) refork
[20946] * Listening on http://0.0.0.0:9292
[20946] Use Ctrl-C to stop
[20946] - Worker 1 (PID: 20960) booted in 0.01s, phase: 0
[20946] - Worker 0 (PID: 20959) booted in 0.01s, phase: 0
"Running lowlevel_error_handler with exception: #<Puma::HttpParserError: Invalid HTTP format, parsing fails. Are you trying to open an SSL connection to a non-SSL Puma?>, status_code: 400"
"Running lowlevel_error_handler with exception: #<Puma::HttpParserError: Invalid HTTP format, parsing fails. Are you trying to open an SSL connection to a non-SSL Puma?>, status_code: 400"
Error in reactor loop escaped: undefined method 'call' for an instance of String (NoMethodError)
/Users/joshuayoung/Projects/puma/lib/puma/request.rb:266:in 'Puma::Request#prepare_response'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:585:in 'Puma::Server#response_to_error'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:551:in 'Puma::Server#client_error'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:315:in 'Puma::Server#reactor_wakeup'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:257:in 'block in Puma::Server#run'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:129:in 'Puma::Reactor#wakeup!'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:84:in 'block in Puma::Reactor#select_loop'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:84:in 'NIO::Selector#select'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:84:in 'Puma::Reactor#select_loop'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:47:in 'block in Puma::Reactor#run'
Error in reactor loop escaped: undefined method 'call' for an instance of String (NoMethodError)
/Users/joshuayoung/Projects/puma/lib/puma/request.rb:266:in 'Puma::Request#prepare_response'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:585:in 'Puma::Server#response_to_error'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:551:in 'Puma::Server#client_error'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:315:in 'Puma::Server#reactor_wakeup'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:257:in 'block in Puma::Server#run'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:129:in 'Puma::Reactor#wakeup!'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:84:in 'block in Puma::Reactor#select_loop'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:84:in 'NIO::Selector#select'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:84:in 'Puma::Reactor#select_loop'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:47:in 'block in Puma::Reactor#run'
"Running lowlevel_error_handler with exception: #<Puma::HttpParserError: Invalid HTTP format, parsing fails. Are you trying to open an SSL connection to a non-SSL Puma?>, status_code: 400"
#<Thread:0x000000012199db90@puma reactor /Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:45 run> terminated with exception (report_on_exception is true):
/Users/joshuayoung/Projects/puma/lib/puma/request.rb:266:in 'Puma::Request#prepare_response': undefined method 'call' for an instance of String (NoMethodError)

        response_hijack.call socket
                       ^^^^^
	from /Users/joshuayoung/Projects/puma/lib/puma/server.rb:585:in 'Puma::Server#response_to_error'
	from /Users/joshuayoung/Projects/puma/lib/puma/server.rb:551:in 'Puma::Server#client_error'
	from /Users/joshuayoung/Projects/puma/lib/puma/server.rb:315:in 'Puma::Server#reactor_wakeup'
	from /Users/joshuayoung/Projects/puma/lib/puma/server.rb:257:in 'block in Puma::Server#run'
	from /Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:112:in 'Array#each'
	from /Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:112:in 'Puma::Reactor#select_loop'
	from /Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:47:in 'block in Puma::Reactor#run'
/Users/joshuayoung/Projects/puma/lib/puma/client.rb:310:in 'Puma::HttpParser#execute': Invalid HTTP format, parsing fails. Are you trying to open an SSL connection to a non-SSL Puma? (Puma::HttpParserError)
	from /Users/joshuayoung/Projects/puma/lib/puma/client.rb:310:in 'Puma::Client#parser_execute'
	from /Users/joshuayoung/Projects/puma/lib/puma/client.rb:277:in 'Puma::Client#try_to_finish'
	from /Users/joshuayoung/Projects/puma/lib/puma/server.rb:306:in 'Puma::Server#reactor_wakeup'
	from /Users/joshuayoung/Projects/puma/lib/puma/server.rb:257:in 'block in Puma::Server#run'
	from /Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:112:in 'Array#each'
	from /Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:112:in 'Puma::Reactor#select_loop'
	from /Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:47:in 'block in Puma::Reactor#run'

fix-reactor-wakeup-no-method-error-fd-leak

# Initial
Tue Aug 19 05:12:12 AEST 2025
Total FDs:       63

puma [fix-reactor-wakeup-no-method-error-fd-leak] ./slow_body_http_requests.sh valid
Sending 500 slow body HTTP requests (valid)...

# Increases
Tue Aug 19 05:12:27 AEST 2025
Total FDs:      563

# Drops
Tue Aug 19 05:12:57 AEST 2025
Total FDs:       63

puma [fix-reactor-wakeup-no-method-error-fd-leak] ./slow_body_http_requests.sh invalid
Sending 500 slow body HTTP requests (invalid)...

# Increases
Tue Aug 19 05:13:18 AEST 2025
Total FDs:      563

# Drops
Tue Aug 19 05:13:29 AEST 2025
Total FDs:       63

puma [fix-reactor-wakeup-no-method-error-fd-leak] ./slow_body_http_requests.sh valid
Sending 500 slow body HTTP requests (valid)...

# Increases
Tue Aug 19 05:14:24 AEST 2025
Total FDs:      563

# Drops
Tue Aug 19 05:14:44 AEST 2025
Total FDs:       63

puma [fix-reactor-wakeup-no-method-error-fd-leak] ./slow_body_http_requests.sh valid
Sending 500 slow body HTTP requests (valid)...

# Increases
Tue Aug 19 05:15:06 AEST 2025
Total FDs:      563

# Drops
Tue Aug 19 05:15:18 AEST 2025
Total FDs:       63

puma [fix-reactor-wakeup-no-method-error-fd-leak] be bin/puma -t 3 -w 2 -C puma.rb app.ru
[36806] Puma starting in cluster mode...
[36806] * Puma version: 7.0.0.pre1 ("Romantic Warrior")
[36806] * Ruby version: ruby 3.4.5 (2025-07-16 revision 20cda200d3) +YJIT +PRISM [arm64-darwin25]
[36806] *  Min threads: 3
[36806] *  Max threads: 3
[36806] *  Environment: development
[36806] *   Master PID: 36806
[36806] *      Workers: 2
[36806] *     Restarts: (✔) hot (✔) phased (✖) refork
[36806] * Listening on http://0.0.0.0:9292
[36806] Use Ctrl-C to stop
[36806] - Worker 1 (PID: 36811) booted in 0.01s, phase: 0
[36806] - Worker 0 (PID: 36810) booted in 0.01s, phase: 0
"Running lowlevel_error_handler with exception: #<Puma::HttpParserError: Invalid HTTP format, parsing fails. Are you trying to open an SSL connection to a non-SSL Puma?>, status_code: 400"
Error in reactor loop escaped: undefined method 'call' for an instance of String (NoMethodError)
/Users/joshuayoung/Projects/puma/lib/puma/request.rb:266:in 'Puma::Request#prepare_response'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:585:in 'Puma::Server#response_to_error'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:551:in 'Puma::Server#client_error'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:315:in 'Puma::Server#reactor_wakeup'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:257:in 'block in Puma::Server#run'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:133:in 'Puma::Reactor#wakeup!'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:87:in 'block in Puma::Reactor#select_loop'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:85:in 'NIO::Selector#select'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:85:in 'Puma::Reactor#select_loop'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:47:in 'block in Puma::Reactor#run'
"Running lowlevel_error_handler with exception: #<Puma::HttpParserError: Invalid HTTP format, parsing fails. Are you trying to open an SSL connection to a non-SSL Puma?>, status_code: 400"
Error in reactor loop escaped: undefined method 'call' for an instance of String (NoMethodError)
/Users/joshuayoung/Projects/puma/lib/puma/request.rb:266:in 'Puma::Request#prepare_response'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:585:in 'Puma::Server#response_to_error'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:551:in 'Puma::Server#client_error'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:315:in 'Puma::Server#reactor_wakeup'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:257:in 'block in Puma::Server#run'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:133:in 'Puma::Reactor#wakeup!'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:87:in 'block in Puma::Reactor#select_loop'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:85:in 'NIO::Selector#select'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:85:in 'Puma::Reactor#select_loop'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:47:in 'block in Puma::Reactor#run'
"Running lowlevel_error_handler with exception: #<Puma::HttpParserError: Invalid HTTP format, parsing fails. Are you trying to open an SSL connection to a non-SSL Puma?>, status_code: 400"
Error in reactor loop escaped: undefined method 'call' for an instance of String (NoMethodError)
/Users/joshuayoung/Projects/puma/lib/puma/request.rb:266:in 'Puma::Request#prepare_response'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:585:in 'Puma::Server#response_to_error'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:551:in 'Puma::Server#client_error'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:315:in 'Puma::Server#reactor_wakeup'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:257:in 'block in Puma::Server#run'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:133:in 'Puma::Reactor#wakeup!'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:87:in 'block in Puma::Reactor#select_loop'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:85:in 'NIO::Selector#select'
/Users/joshuayoung/Projects/puma/lib/puma/reactor"Running lowlevel_error_handler with exception: #<Puma::HttpParserError: Invalid HTTP format, parsing fails. Are you trying to open an SSL connection to a non-SSL Puma?>, status_code: 400"
Error in reactor loop escaped: undefined method 'call' for an instance of String (NoMethodError)
/Users/joshuayoung/Projects/puma/lib/puma/request.rb:266:in 'Puma::Request#prepare_response'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:585:in 'Puma::Server#response_to_error'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:551:in 'Puma::Server#client_error'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:315:in 'Puma::Server#reactor_wakeup'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:257:in 'block in Puma::Server#run'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:133:in 'Puma::Reactor#wakeup!'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:87:in 'block in Puma::Reactor#select_loop'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:85:in 'NIO::Selector#select'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:85:in 'Puma::Reactor#select_loop'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:47:in 'block in Puma::Reactor#run'
"Running lowlevel_error_handler with exception: #<Puma::HttpParserError: Invalid HTTP format, parsing fails. Are you trying to open an SSL connection to a non-SSL Puma?>, status_code: 400"
.rb:85:in 'Puma::Reactor#select_loop'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:47:in 'block in Puma::Reactor#run'
Error in reactor loop escaped: undefined method 'call' for an instance of String (NoMethodError)
/Users/joshuayoung/Projects/puma/lib/puma/request.rb:266:in 'Puma::Request#prepare_response'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:585:in 'Puma::Server#response_to_error'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:551:in 'Puma::Server#client_error'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:315:in 'Puma::Server#reactor_wakeup'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:257:in 'block in Puma::Server#run'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:133:in 'Puma::Reactor#wakeup!'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:87:in 'block in Puma::Reactor#select_loop'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:85:in 'NIO::Selector#select'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:85:in 'Puma::Reactor#select_loop'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:47:in 'block in Puma::Reactor#run'
"Running lowlevel_error_handler with exception: #<Puma::HttpParserError: Invalid HTTP format, parsing fails. Are you trying to open an SSL connection to a non-SSL Puma?>, status_code: 400"
Error in reactor loop escaped: undefined method 'call' for an instance of String (NoMethodError)
/Users/joshuayoung/Projects/puma/lib/puma/request.rb:266:in 'Puma::Request#prepare_response'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:585:in 'Puma::Server#response_to_error'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:551:in 'Puma::Server#client_error'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:315:in 'Puma::Server#reactor_wakeup'
/Users/joshuayoung/Projects/puma/lib/puma/server.rb:257:in 'block in Puma::Server#run'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:133:in 'Puma::Reactor#wakeup!'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:87:in 'block in Puma::Reactor#select_loop'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:85:in 'NIO::Selector#select'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:85:in 'Puma::Reactor#select_loop'
/Users/joshuayoung/Projects/puma/lib/puma/reactor.rb:47:in 'block in Puma::Reactor#run'
...

Your checklist for this pull request

I have reviewed the guidelines for contributing to this repository.
I have added (or updated) appropriate tests if this PR fixes a bug or adds a feature.
My pull request is 100 lines added/removed or less so that it can be easily reviewed.
If this PR doesn't need tests (docs change), I added [ci skip] to the title of the PR.
If this closes any issues, I have added "Closes #issue" to the PR description or my commit messages.
I have updated the documentation accordingly.
All new and existing tests passed, including Rubocop.

…loop`

MSP-Greg · 2025-08-19T22:32:47Z

When a lowlevel_error_handler returns an invalid response

So, something (an error being raised, could be client or app) has to trigger a lowlevel_error_handler, then the handler has to return an invalid response, and then Puma fails.

But, Puma should never fail. Sometimes I hate that...

Thanks for the PR, I'll wait to see if others have comments.

dentarg · 2025-08-31T15:22:16Z

The NoMethodError rescue logic was originally added in commit fd259b7 to handle rare selector exceptions. It seems like it shouldn't be required, and that there's some underlying issue that should be fixed. I'd be happy to look into this in a follow-up so we can remove it altogether, but this patch should do for now.

Do we have an issue for this?

joshuay03 · 2025-09-01T03:00:15Z

The NoMethodError rescue logic was originally added in commit fd259b7 to handle rare selector exceptions. It seems like it shouldn't be required, and that there's some underlying issue that should be fixed. I'd be happy to look into this in a follow-up so we can remove it altogether, but this patch should do for now.

Do we have an issue for this?

It does now: #3718

Fix socket leak on monitor wakeup NoMethodError in `Reactor#select_…

07bee10

…loop`

joshuay03 mentioned this pull request Aug 18, 2025

Socket leak on monitor wakeup NoMethodError in Reactor#select_loop #3695

Closed

github-actions bot added the waiting-for-review Waiting on review from anyone label Aug 18, 2025

joshuay03 moved this to In Progress / Pending Review in Open Source Aug 18, 2025

joshuay03 added this to Open Source Aug 18, 2025

joshuay03 moved this from In Progress / Pending Review to Done in Open Source Aug 18, 2025

joshuay03 moved this from Done to In Progress / Pending Review in Open Source Aug 18, 2025

MSP-Greg approved these changes Aug 19, 2025

View reviewed changes

github-actions bot added waiting-for-merge and removed waiting-for-review Waiting on review from anyone labels Aug 19, 2025

MSP-Greg merged commit c816afc into puma:master Aug 22, 2025
89 of 90 checks passed

joshuay03 moved this from In Progress / Pending Review to Done in Open Source Aug 22, 2025

dentarg added bug and removed waiting-for-merge labels Aug 31, 2025

joshuay03 mentioned this pull request Sep 1, 2025

Remove NoMethodError rescue logic for NIO::Selector#select in reactor #3718

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix socket leak on monitor wakeup `NoMethodError` in `Reactor#select_loop` #3696

Fix socket leak on monitor wakeup `NoMethodError` in `Reactor#select_loop` #3696

Uh oh!

joshuay03 commented Aug 18, 2025 •

edited

Loading

Uh oh!

MSP-Greg commented Aug 19, 2025

Uh oh!

Uh oh!

dentarg commented Aug 31, 2025

Uh oh!

joshuay03 commented Sep 1, 2025

Uh oh!

Uh oh!

Fix socket leak on monitor wakeup NoMethodError in Reactor#select_loop #3696

Fix socket leak on monitor wakeup NoMethodError in Reactor#select_loop #3696

Uh oh!

Conversation

joshuay03 commented Aug 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Your checklist for this pull request

Uh oh!

MSP-Greg commented Aug 19, 2025

Uh oh!

Uh oh!

dentarg commented Aug 31, 2025

Uh oh!

joshuay03 commented Sep 1, 2025

Uh oh!

Uh oh!

Fix socket leak on monitor wakeup `NoMethodError` in `Reactor#select_loop` #3696

Fix socket leak on monitor wakeup `NoMethodError` in `Reactor#select_loop` #3696

joshuay03 commented Aug 18, 2025 •

edited

Loading