Can't convert Array into String. (TypeError) #585
Description
This looks like a bug in the HTTP parser used (https://github.com/tmm1/http_parser.rb), but I thought I would first report it here. (I haven't yet taken the time to reproduce it with only http_parser.rb.)
The bug happens when you try to follow really long redirect URLs. In my case the URL have the whole page content embedded, that's why it's so long (see https://github.com/alcor/itty-bitty/). https://burd.se/httprb2 is an example that redirect to such an URL.
Repro:
$ git clone git@github.com:httprb/http.git
$ cd http
$ docker run --rm -it -v $(pwd):/app ruby:2.6 bash
root@8b3b9a934e9b:/# cd /app
root@8b3b9a934e9b:/app# gem install bundler
root@8b3b9a934e9b:/app# bundle
root@8b3b9a934e9b:/app# gem build http
root@8b3b9a934e9b:/app# gem install http-5.0.0.pre.gem
root@8b3b9a934e9b:/app# ruby -rhttp -e 'puts HTTP::VERSION; HTTP.follow.get("https://burd.se/httprb2")'
5.0.0.pre
Traceback (most recent call last):
7: from -e:1:in `<main>'
6: from /usr/local/bundle/gems/http-5.0.0.pre/lib/http/chainable.rb:20:in `get'
5: from /usr/local/bundle/gems/http-5.0.0.pre/lib/http/client.rb:34:in `request'
4: from /usr/local/bundle/gems/http-5.0.0.pre/lib/http/redirector.rb:61:in `perform'
3: from /usr/local/bundle/gems/http-5.0.0.pre/lib/http/redirector.rb:98:in `redirect_to'
2: from /usr/local/bundle/gems/http-5.0.0.pre/lib/http/request.rb:106:in `redirect'
1: from /usr/local/lib/ruby/2.6.0/forwardable.rb:230:in `join'
/usr/local/bundle/gems/addressable-2.7.0/lib/addressable/uri.rb:1884:in `join': Can't convert Array into String. (TypeError)I suspect it is the parser, because with this patch
diff --git a/lib/http/response/parser.rb b/lib/http/response/parser.rb
index 19989d5..caa8412 100644
--- a/lib/http/response/parser.rb
+++ b/lib/http/response/parser.rb
@@ -53,6 +53,7 @@ module HTTP
end
def on_header_value(_response, value)
+ p value
@headers.add(@field, value) if @field
endThis is the output
root@8b3b9a934e9b:/app# ruby -rhttp -e 'puts HTTP::VERSION; HTTP.follow.get("https://burd.se/httprb2")'
5.0.0.pre
"Mon, 13 Jan 2020 16:38:18 GMT"
"text/html;charset=utf-8"
"chunked"
"close"
"__cfduid=d0408cdfd6fe21b1ab4b6adde29adb2921578933498; expires=Wed, 12-Feb-20 16:38:18 GMT; path=/; domain=.burd.se; HttpOnly; SameSite=Lax"
"https://itty.bitty.site/#/?XQAAAAKhFgAAAAAAAAAeGQknlxhIQakTrqORUwLuwNEJhII5BpfZZe5tdT5QIwTSvOsK2m3Bfw5s9+y2qRxSY1BcMjsN6DS6cL01kzF+/htSy2Bl4kCeWPe2MR3DiTiQ+y3SwAXN7Cxy+UVIJS6RNKq2PuLesCU9Q+7WTJVZ0Av/eNbp5ES4W/mXBdTuS/ePyhh5VlUKyMKkTqakZqYXgeS9YtMYinYS5oQhdkYruc4Bv2D3r621D2i5JheDlF7kTQ3uZSTtoWW+y8/ox2KYVZP8j5YOLwJTNQSLYdZ8im1nJynRN0UhzX0ztHlae7rDee5gTzByVPXv+c6Xd7UHSsdhMdgY2WSG67yQ1Zs1jPGX3daqF5VGETN1aUwCJbKa1ke7yxhlMdatQ9p8t3V4ISbW4DPyStoGlk87MfFyqpZOqL/nzUWOeHqHeb77VP/GQn83m9phRjBBkImblZSdIKAMJE4+K/Df4YWH0zlor7WAnO1dwS8hXYkY5FiCSCPKZ888EQgOeQhf3tBPe9NK7HX54cl2BN4AzBFjSRPnGFEa3cbD6cm2zmmpDoMfF+BD0H9Xv9JTegg8kG8T+erwf8I7/lvs9AKpFa2dyT/Pn2RNM2ZUHgLnhmbAKpms/lWmk5HZ355T3HSue4WSvnd8VFGxvRg1fnzh4+uqT3GeYHpak0XRE4FXtdvnNml3WUVER/GeAfeiXOw2UGXDuneHfhaLPam9xuVaL1YX54fwVQ/iWBjPzc9ZBFO5AKm2BbqCmBykyBYQL9mrO6AHG2TqxbAULXb3YOFfITtHTokI7lmr7Vi6iA6uj+3LWdcHEieXgdR42tAOetFjcxnkg2O8ebYgs0kO+aklrZP8/V6Q1UE858rNDz9ejavwc20k+bh5UwL7Ls4VSOk3Yqvrk3wG1aMsOA6V9vxk6nyPLtR8oBXkbhE5ElHiCYd1qd6f94JiRUWcqgfCSNDYes8pWifn9tj9YE2ZygFJ2e2l8mmMcnSQ0m1oXEGYnzY9dnBGEmL9cHYzppASVxner"
"AYHjVVPcUX8l6Zh8txIrDwWJU8RKjBwYRYq0yYW9X3bP5gumSWRPT7rConR1ulDEHhrHiF2WbF366uigOO/dHffQYtxCOA0ep5wy4nhuHYnPLuCsc/aeaAktAWZXOiQ8W/obYr9Ng52dfmILf6X8o56O8h5vvhD21NOzRsTVCiA/l7qMutRhB87GYP5kQ/DlMy27u8xeQ8pip4rPwinxaWDx5BkSIMTcFHHa5B0Xn9PUlUBXeugCuE2Onqf34SMgp3vxMEsIZBeU942SBysVYrmvPFS0T1pwZU8qiwIX4H8T7kxGzyp9CcomeFnWJb24slRzSglUV9LkyAQ4NibfOBwFQCcp58YSTMtYVytkQGV2a3OxivDXxPdewmrM7mx4X9Ezvz5xO2u0a/h2nMBRGk1xeOwLH14CTOTN1MBbYr+zQysRBxB+xtHIIxhBz/UdoZeVnIwGltv79NbukWn1+J+56Aafch3PQ6HVuoYG2t1ZUYOxLOr42fRwMKuEWIZ+D1zRIcn0jUVBv7I2NT3F6FmcHMGE/Mop3zGKSQ/J4R9Lg1DUgHAdzs6ikdgNXderLCnK/r9VBuR+k35hp0jPcs9B6mWPH7Fg4uoxZDgT438nDMj1F/gn222+vMKjf6x1vQlcB9OqzCabhVzAhAV3Q187RO/fNtVBJ+K/j2I+Bsa1dWFuWsmknieGZ7vdaJne0BcO1KJ+A3VkvC2DAsJlcR9PABdBgQesNzMONmXTkLkPwP0FWrQ1g2Wk6poeBX3NVKkN0/uNwq4hsXOf5ITiO5ZnOeGXF+LPQ9OKh8jXeaqBnRYXAHI2qHmqavNv7qLtygi8tFBIMjPHG3y4bfX3IhGhQFdreYLx/Rclse1Ohkp/FzI+5YVsTc7t6aegSHvRUsZX7KcroTvOT/Bbr0b/JUbuacrh3esQ+OsxWNGtImSe/YJHl7Hgdjz3VONC2QauizMswXT77ITbwl5xc7RBJ0eDKgKhoBdLrJfyOuhDxuWV7xn4R2Z86PgfQ3OfE6ngbaH4MswD/+NK3e6Ss48LHWI/tmrk6z16O4S5UztIt6IeaQvwn2jxN1qbI7KrZk9QZ1ZseLv4ELjbbU0inAJqzIYXxV3rQoMFWasBKTI/csjhnpMCmbQHTjeJvl+TECNqsj4Ay/D4roJVgLLlmWDiMrDDMFJuSr+ZaMnxWid3WOKI5vbcq2bOLS+y2hnwyKWIaM9JEWV3YMmwmy24Jv60dLMVx8xzBn0pvFBmHDG61bAQiE0rs3qrdRjBdS30ZIvCz4o0GmRdbEn6jOkmtnjkuntcq5fhpb3zIU8vrQ47aLBYM4GyTV8YWjwSzfD9B+BNZe8thwcE4OL04gC/eMJ8Fc+ep"
"YE3UEp6A3cw1GZcVF38ZhQfqfXodvFOuuv9YcXHw4XT00MZ7S9vJesnW2hw1nglRMm91paGQR+JiCI5Q+Z2xWNKbRJRe+b/RZ7gY1+Lw0jo7PGZmPAFTrBa2pUQPMnE6j+sLY/2FoR9YpnUN5jEmFQZK+qnx7aULRfSrAvT9bntcpntSxt1WzMvAygAneAojiV27vSb2K7XUj3G4pUqdJD1N2rLIPCXWcqyuB3f85QvpxHAskXcZoC7y9Wn/S1KGQRuvsLn9l30IR5J/lImUh4v3YOHy8JZM6G1Jd83iVT//jd8nw="
"1; mode=block"
"nosniff"
"SAMEORIGIN"
"1.1 vegur"
"DYNAMIC"
"max-age=604800, report-uri=\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\""
"cloudflare"
"5548ccbc1f9cd8a1-CPH"
Traceback (most recent call last):
7: from -e:1:in `<main>'
6: from /usr/local/bundle/gems/http-5.0.0.pre/lib/http/chainable.rb:20:in `get'
5: from /usr/local/bundle/gems/http-5.0.0.pre/lib/http/client.rb:34:in `request'
4: from /usr/local/bundle/gems/http-5.0.0.pre/lib/http/redirector.rb:61:in `perform'
3: from /usr/local/bundle/gems/http-5.0.0.pre/lib/http/redirector.rb:98:in `redirect_to'
2: from /usr/local/bundle/gems/http-5.0.0.pre/lib/http/request.rb:106:in `redirect'
1: from /usr/local/lib/ruby/2.6.0/forwardable.rb:230:in `join'
/usr/local/bundle/gems/addressable-2.7.0/lib/addressable/uri.rb:1884:in `join': Can't convert Array into String. (TypeError)