Skip to content

Drop disk buffering and support for rewindable inputs. #1148

@thesmart

Description

@thesmart

I'm curious about Rack's requirement for rewindable input, and because of this requirement, the Rack::Multipart class will buffer a read-once stream to disk presumably so rewind is available. Modern cloud-based NAS throughput imposes limitation on how well this feature scales. Also, RAM is wildly cheap these days. Considering how the entire Rack ecosystem could depend on rewindable streams, I was wondering...

Is it a good time for future versions of Rack 2.x to drop disk buffering as a default behavior, and move it to an optional Rack middleware specifically for handling "large" bodies / file uploads?

I'd like to propose an alternative strategy for handling "large" bodies.

(0) Remove any use of Tempfile in Rack::Multipart or other default body parsers.

(1) Provide a configurable post body buffer limit max_body_buffer_length. If content_length.present? && content_length <= max_body_buffer_length then read it ASAP (ideally non-blocking) into rack.input as StringIO or other. Else, if content_length.nil? || content_length > max_body_buffer_length, then do not read the stream. This would mean that no chunked encodings are handled by default, and no buffering to disk for streams larger than the configured limit.

(2) When request.params or request.body is accessed, if body was not parsed due to length or chunked encoding, report a warning message that can be disabled.

(3) Provide automatic "large body" parsing as a middleware module plugin that can be optionally installed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions