Skip to content

readpartial must not return more data than requested #43

@headius

Description

@headius

The contract of readpartial is that it will return up to the amount requested and no more. However the implementation of readpartial in WEBrick's HTTPRequest does not honor this contract:

# for IO.copy_stream. Note: we may return a larger string than +size+
# here; but IO.copy_stream does not care.
def readpartial(size, buf = ''.b) # :nodoc
res = @body_tmp.shift or raise EOFError, 'end of file reached'
buf.replace(res)
res.clear
@body_rd.resume # get more chunks
buf
end

The assumption described here may hold in CRuby, but in JRuby's implementation of copy_stream we may use an intermediate buffer to hold data on its way, and if readpartial returns more data than requested we will overflow that buffer. I will be fixing JRuby to raise a more Ruby-friendly error, but it's still going to be an error if we readpartial N bytes and get N+1.

https://github.com/jruby/jruby/blob/a612f900243926e45a3d3465bc8ae5d9e5258cce/core/src/main/java/org/jruby/RubyIO.java#L4433-L4454

This is the root cause of a failure in test_big_bodies from test_httpproxy.rb as described in jruby/jruby#6246. The copy_stream call deep inside the proxy test fails due to this buffer overflow, which causes the request's piped content to be prematurely closed and the proxy test to error out while trying to write request data.

th = Thread.new { nr.times { wr.write(rand_str) }; wr.close }

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions