Skip to content

os.read/write waits until file descriptor is ready. #975

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Jul 16, 2024
Merged

os.read/write waits until file descriptor is ready. #975

merged 2 commits into from
Jul 16, 2024

Conversation

ethan-vanderheijden
Copy link
Contributor

Fixes #973
Fixes #634

Currently, calling os.read(fd, size) will attempt to read the file descriptor immediately. If there is no data to be read, the thread will block until data is available, causing the program to hang. With this pull request, os.read will wait until the file descriptor is ready to read before attempting a read. It will not wait if the file descriptor is a regular file (according to select/poll, regular files are always ready to read).

My fear is that os.write has the same bug:

eventlet/eventlet/green/os.py

Lines 57 to 68 in 06ec828

def write(fd, st):
"""write(fd, string) -> byteswritten
Write a string to a file descriptor.
"""
while True:
try:
return __original_write__(fd, st)
except OSError as e:
if get_errno(e) not in [errno.EAGAIN, errno.EPIPE]:
raise
hubs.trampoline(fd, write=True)

It's not common for a write call to block, but it can happen, and if it does happen, the entire program will hang. Do you want me to investigate and fix this as well?

@4383
Copy link
Member

4383 commented Jul 15, 2024

Thanks for this patch.

It's not common for a write call to block, but it can happen, and if it does happen, the entire program will hang. Do you want me to investigate and fix this as well?

Yes please

@codecov Codecov
Copy link

codecov bot commented Jul 15, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 69.23077% with 4 lines in your changes missing coverage. Please review.

Project coverage is 56%. Comparing base (06ec828) to head (0f2b4e1).

Files Patch % Lines
eventlet/green/os.py 69% 4 Missing ⚠️
Additional details and impacted files 
@@          Coverage Diff          @@
##           master   #975   +/-   ##
=====================================
  Coverage      56%    56%           
=====================================
  Files          89     89           
  Lines        9783   9791    +8     
  Branches     1824   1826    +2     
=====================================
+ Hits         5488   5498   +10     
+ Misses       3925   3921    -4     
- Partials      370    372    +2
Flag Coverage Δ
ipv6 23% <7%> (-1%) ⬇️
py310asyncio 52% <69%> (+<1%) ⬆️
py310epolls 53% <69%> (-1%) ⬇️
py310poll 53% <69%> (-1%) ⬇️
py310selects 53% <69%> (-1%) ⬇️
py311asyncio 52% <69%> (+<1%) ⬆️
py311epolls 53% <69%> (-1%) ⬇️
py312asyncio 51% <69%> (+<1%) ⬆️
py312epolls 51% <69%> (-1%) ⬇️
py37asyncio 50% <61%> (+<1%) ⬆️
py37epolls 51% <61%> (+<1%) ⬆️
py38asyncio 51% <61%> (+<1%) ⬆️
py38epolls 53% <61%> (+<1%) ⬆️
py38openssl 51% <61%> (-1%) ⬇️
py38poll 53% <61%> (-1%) ⬇️
py38selects 52% <61%> (+<1%) ⬆️
py39asyncio 51% <61%> (+<1%) ⬆️
py39dnspython1 51% <61%> (+<1%) ⬆️
py39epolls 53% <61%> (-1%) ⬇️
py39poll 53% <61%> (+<1%) ⬆️
py39selects 52% <61%> (+<1%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ethan-vanderheijden ethan-vanderheijden changed the title os.read() waits until file descriptor is ready before reading. os.read/write waits until file descriptor is ready. Jul 15, 2024
@ethan-vanderheijden
Copy link
Contributor Author

Ok, I made the same change with os.write(). Now, it waits until the file descriptor is ready before attempting a write.

According to the manual (for select, poll, epoll):

However, even if a file descriptor indicates as writable, a large write may still block.

If someone calls os.write() with a single large chunk of data, the program may still hang. The only solution is to mark the file descriptor as non-blocking prior to the write, but that is something the user has to do, not us.

@4383
Copy link
Member

4383 commented Jul 16, 2024

I just re-enqueued all jobs as some jobs failed during the previous run. No logs were available so I've no clue. I want to be sure than nothing is broken.

@4383
Copy link
Member

4383 commented Jul 16, 2024

Ok, it was just a flaky CI, lets move on

@4383 4383 merged commit c4f4e69 into eventlet:master Jul 16, 2024
30 checks passed
@4383
Copy link
Member

4383 commented Jul 16, 2024

@ethan-vanderheijden thanks for your investigations and for your work. Good job!

@ethan-vanderheijden ethan-vanderheijden mentioned this pull request Jul 16, 2024
Closed
@ethan-vanderheijden
Copy link
Contributor Author

@4383 Thanks for the quick review!

Sorry to bug you, but when is the next release of Eventlet scheduled?

@4383
Copy link
Member

4383 commented Jul 19, 2024

@ethan-vanderheijden I'll generate a new version early next week. I need to do an inventory of the latest changes first.

@4383 4383 mentioned this pull request Jul 22, 2024
Closed
@4383
Copy link
Member

4383 commented Jul 22, 2024

@ethan-vanderheijden I'm preparing the next release, you can follow its progress there => #979

@4383
Copy link
Member

4383 commented Sep 12, 2024

@ethan-vanderheijden eventlet 0.37.0 is out! https://pypi.org/project/eventlet/0.37.0/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@4383 4383 4383 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

os.read() blocks even after monkey patching. eventlet with Flask/SocketIO in debug mode with watchdog installed hangs on startup
2 participants
@ethan-vanderheijden @4383