-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Description
I'm curious about Rack's requirement for rewindable input, and because of this requirement, the Rack::Multipart
class will buffer a read-once stream to disk presumably so rewind
is available. Modern cloud-based NAS throughput imposes limitation on how well this feature scales. Also, RAM is wildly cheap these days. Considering how the entire Rack ecosystem could depend on rewindable streams, I was wondering...
Is it a good time for future versions of Rack 2.x to drop disk buffering as a default behavior, and move it to an optional Rack middleware specifically for handling "large" bodies / file uploads?
I'd like to propose an alternative strategy for handling "large" bodies.
(0) Remove any use of Tempfile
in Rack::Multipart
or other default body parsers.
(1) Provide a configurable post body buffer limit max_body_buffer_length
. If content_length.present? && content_length <= max_body_buffer_length
then read it ASAP (ideally non-blocking) into rack.input
as StringIO
or other. Else, if content_length.nil? || content_length > max_body_buffer_length
, then do not read the stream. This would mean that no chunked encodings are handled by default, and no buffering to disk for streams larger than the configured limit.
(2) When request.params
or request.body
is accessed, if body was not parsed due to length or chunked encoding, report a warning message that can be disabled.
(3) Provide automatic "large body" parsing as a middleware module plugin that can be optionally installed.