`FileTest.send(command, ...)` allows to call not only FileTest-related
methods but also any method that belongs to Kernel, Object, etc.
patched by <mame@ruby-lang.org>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@67820 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
WEBrick: prevent response splitting and header injection
This is a follow up to d9d4a28.
The commit prevented CRLR, but did not address an isolated CR or an
isolated LF.
Co-Authored-By: NARUSE, Yui <naruse@airemix.jp>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@67819 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Loop with String#scan without creating substrings
Create the substrings necessary parts only, instead of cutting the
rest of the buffer. Also removed a useless, probable typo, regexp.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@67818 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Fix for wrong fnmatch patttern
* dir.c (file_s_fnmatch): ensure that pattern does not contain a
NUL character. https://hackerone.com/reports/449617
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@67817 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
io.c: chomp CR at the end of read buffer
* io.c (rb_io_getline_fast): chomp CR followed by LF but separated
by the read buffer boundary. [ruby-core:91707] [Bug #15642]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@67391 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
tmpdir.rb: permission of user given directory
* lib/tmpdir.rb (Dir.mktmpdir): check if the permission of the
parent directory only when using the default temporary
directory, and no check against user given directory. the
security is the user's responsibility in that case.
[ruby-core:91216] [Bug #15555]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@67148 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
parse.y: unindent continued line
* parse.y (tokadd_string): stop at continued line in dedented here
documents, to dedent for each lines before removing escaped
newlines. [ruby-core:86236] [Bug #14621]
parse.y: terminator at continued line
* parse.y (here_document): a continuing line is not the
terminator. [ruby-core:86283] [Bug #14621]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@67147 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
cont.c: set th->root_fiber to current fiber at fork
Otherwise, th->root_fiber can point to an invalid Fiber,
because Fibers do not live across fork. So consider
whatever Fiber is running the root fiber.
[ruby-core:88723] [Bug #15041]
cont.c (rb_fiber_atfork): th->root_fiber may not exist
Otherwise, bootstraptest/test_fork.rb fails with -DVM_CHECK_MODE=2
[Bug #15041]
Fixes: r64589 "cont.c: set th->root_fiber to current fiber at fork"
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@66968 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Normalize month-mday before finding epoch
Especially over the year 2038, 30 Feb and so on may cause odd behavior
on validating found epoch with given year-month-day [Bug #15340]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@66967 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Mark array as "going to be modified" in `Array#reject!`
Before this patch, if `reject!` is called on a shared array it can
mutate the shared array rather than a copy. This patch marks the array
as "going to be modified" so that the shared source array isn't
mutated.
[Bug #15479] [ruby-core:90781]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@66966 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
infect taint flag on Array#pack and String#unpack
with the directives "B", "b", "H" and "h".
* pack.c (pack_pack, pack_unpack_internal): infect taint flag.
* test/ruby/test_pack.rb: add test for above.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@65129 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Fix Kernel#singleton_method with Module#Prepend
* proc.c (rb_obj_singleton_method): search the method entry from
the origin class, for fix prepended modules. [Bug #14658]
From: Vasiliy Ermolovich <younash@gmail.com>
proc.c: fix segfault when no singleton class
* proc.c (rb_obj_singleton_method): bail out if the receiver does
not have the singleton class without accessing the origin class
not to segfault. [Bug #14658]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@65118 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Just a shebang is valid code
[ruby-core:89240] [Bug #15190]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@65113 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
ruby.c: taint ARGV on Windows
* ruby.c (external_str_new_cstr): strings come from the external
should be tainted. [ruby-dev:50596] [Bug #14941]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@64563 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
thread.c (do_select): fix leak on exception
When do_select is interrupted and raise happens from
RUBY_VM_CHECK_INTS_BLOCKING, the original FD sets we copied
do not get freed, leading to a memory leak. Wrap up all the
FD sets into a Ruby object to ensure the GC can release an
allocations made for rb_fdset_t.
This leak existed since Ruby 2.0.0 (r36430)
[Bug #14929]
increase timeout seconds.
* test/ruby/test_io.rb (test_select_leak): increase timeout seconds
to pass this test on a high-load machine.
60 sec is not enough at all
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@64561 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
variable.c: fix receiver on private constant
* variable.c (rb_const_search): fix NameError :receiver attribute
on private constant, should raise with the included module, not
the ICLASS.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@64559 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
time.rb: fix parsing time zone in iso8601
* lib/time.rb (Time.xmlschema): a colon in time zone designator
can be omitted. [ruby-core:87277] [Bug #14790]
time.rb: fix parsing time zone in iso8601
* lib/time.rb (Time.xmlschema): the minute in time zone designator
can be omitted together with the preceding colon.
[ruby-core:87277] [Bug #14790]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@64148 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
error.c: check redefined backtrace result
* error.c (rb_get_backtrace): check the result of `backtrace` even
if the method is redefined. [ruby-core:87013] [Bug #14756]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@64147 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
vm.c: fix infinite loop
* vm.c: use EXEC_EVENT_HOOK_AND_POP_FRAME. While exception handling, if an exception
is raised in hooks, need to pop current frame and raise this raised exception by hook.
[ruby-dev:50582] [Bug #14865]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@64129 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
test_time_tz.rb: Kiritimati tzdata fix
* test/ruby/test_time_tz.rb (TestTimeTZ#test_pacific_kiritimati):
fix the expected data at the skip of New Year's Eve 1994.
[Bug #14655]
test_time_tz.rb: Kiritimati tzdata fix
* test/ruby/test_time_tz.rb (gen_zdump_test): fix the expected
data at the Kiritimati's skip of New Year's Eve 1994.
[Bug #14655]
test_time_tz.rb: Lisbon tzdata fix
* test/ruby/test_time_tz.rb (gen_variational_zdump_test): Update
Lisbon zdump data, which fixed the 1912-01-01 transition for
Portugual and its colonies. [Bug #14655]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@63808 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
file.c: infect from arguments
* file.c (rb_check_realpath_internal): infetct the result with
arguments, no taint if none are tainted and cwd is not used.
[ruby-core:83583] [Bug #14060]
file.c: infect from arguments
* file.c (rb_check_realpath_internal): infetct the result with
arguments, no taint if none are tainted and cwd is not used.
[ruby-core:83583] [Bug #14060]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@63807 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
win32/file.c: relative path with drive letter
* win32/file.c (IS_ABSOLUTE_PATH_P): home directory should not be
a relative path regardless a drive letter. PathIsRelativeW
returns FALSE on such path. [ruby-core:86356] [Bug #14638]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@63806 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
webrick: support Proc objects as body responses
* lib/webrick/httpresponse.rb (send_body): call send_body_proc
(send_body_proc): new method
(class ChunkedWrapper): new class
* test/webrick/test_httpresponse.rb (test_send_body_proc): new test
(test_send_body_proc_chunked): ditto
[Feature #855]
webrick/httpresponse: IO.copy_stream for regular files
Remove the redundant _send_file method since its functionality
is unnecessary with IO.copy_stream. IO.copy_stream also allows
the use of sendfile under some OSes to speed up copies to
non-TLS sockets.
Testing with "curl >/dev/null" and "ruby -run -e httpd" to
read a 1G file over Linux loopback reveals a reduction from
around ~0.770 to ~0.490 seconds on the client side.
* lib/webrick/httpresponse.rb (send_body_io): use IO.copy_stream
(_send_file): remove
[Feature #14237]
webrick: use IO.copy_stream for single range response
This is also compatible with range responses generated
by Rack::File (tested with rack 2.0.3).
* lib/webrick/httpresponse.rb (send_body_io): use Content-Range
* lib/webrick/httpservlet/filehandler.rb (make_partial_content):
use File object for the single range case
* test/webrick/test_filehandler.rb (get_res_body): use send_body
to test result
test/webrick/test_filehandler.rb: stricter multipart range test
We need to ensure we generate compatibile output in
the face of future changes
* test/webrick/test_filehandler.rb (test_make_partial_content):
check response body
webrick: quiet warning for multi-part ranges
Content-Length is ignored by WEBrick::HTTPResponse even if we
calculate it, so instead we chunk responses to HTTP/1.1 clients
and terminate HTTP/1.0 connections.
* lib/webrick/httpservlet/filehandler.rb (make_partial_content):
quiet warning
webrick/httpresponse: make ChunkedWrapper copy_stream-compatible
The .write method needs to return the number of bytes written
to avoid confusing IO.copy_stream.
* lib/webrick/httpresponse.rb (ChunkedWrapper#write): return bytes written
(ChunkedWrapper#<<): return self
webrick: use IO.copy_stream for multipart response
Use the new Proc response body feature to generate a multipart
range response dynamically. We use a flat array to minimize
object overhead as much as possible; as many ranges may fit
into an HTTP request header.
* lib/webrick/httpservlet/filehandler.rb (multipart_body): new method
(make_partial_content): use multipart_body
get rid of test error/failure on Windows introduced at r62955
* lib/webrick/httpresponse.rb (send_body_io): use seek if NotImplementedError
is raised in IO.copy_stream with offset.
* lib/webrick/httpservlet/filehandler.rb (multipart_body): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@63012 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
webrick/httprequest: limit request headers size
We use the same 112 KB limit started (AFAIK) by Mongrel, Thin,
and Puma to prevent malicious users from using up all the memory
with a single request. This also limits the damage done by
excessive ranges in multipart Range: requests.
Due to the way we rely on IO#gets and the desire to keep
the code simple, the actual maximum header may be 4093 bytes
larger than 112 KB, but we're splitting hairs at that point.
* lib/webrick/httprequest.rb: define MAX_HEADER_LENGTH
(read_header): raise when headers exceed max length
webrick/httpservlet/cgihandler: reduce memory use
WEBrick::HTTPRequest#body can be passed a block to process the
body in chunks. Use this feature to avoid building a giant
string in memory.
* lib/webrick/httpservlet/cgihandler.rb (do_GET):
avoid reading entire request body into memory
(do_POST is aliased to do_GET, so it handles bodies)
webrick/httprequest: raise correct exception
"BadRequest" alone does not resolve correctly, it is in the
HTTPStatus namespace.
* lib/webrick/httprequest.rb (read_chunked): use correct exception
* test/webrick/test_httpserver.rb (test_eof_in_chunk): new test
webrick/httprequest: use InputBufferSize for chunked requests
While WEBrick::HTTPRequest#body provides a Proc interface
for streaming large request bodies, clients must not force
the server to use an excessively large chunk size.
* lib/webrick/httprequest.rb (read_chunk_size): limit each
read and block.call to :InputBufferSize in config.
* test/webrick/test_httpserver.rb (test_big_chunks): new test
webrick: add test for Digest auth-int
No changes to the actual code, this is a new test for
a feature for which no tests existed. I don't understand
the Digest authentication code well at all, but this is
necessary for the subsequent change.
* test/webrick/test_httpauth.rb (test_digest_auth_int): new test
(credentials_for_request): support bodies with POST
webrick/httpauth/digestauth: stream req.body
WARNING! WARNING! WARNING! LIKELY BROKEN CHANGE
Pass a proc to WEBrick::HTTPRequest#body to avoid reading a
potentially large request body into memory during
authentication.
WARNING! this will break apps completely which want to do
something with the body besides calculating the MD5 digest
of it.
Also, keep in mind that probably nobody uses "auth-int".
Servers such as Apache, lighttpd, nginx don't seem to
support it; nor does curl when using POST/PUT bodies;
and we didn't have tests for it until now...
* lib/webrick/httpauth/digestauth.rb (_authenticate): stream req.body
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@63004 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
webrick: prevent response splitting and header injection
Original patch by tenderlove (with minor style adjustments).
* lib/webrick/httpresponse.rb (send_header): call check_header
(check_header): raise on embedded CRLF in header value
* test/webrick/test_httpresponse.rb
(test_prevent_response_splitting_headers): new test
* (test_prevent_response_splitting_cookie_headers): ditto
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@63002 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
dir.c: check NUL bytes
* dir.c (GlobPathValue): should be used in rb_push_glob only.
other methods should use FilePathValue.
https://hackerone.com/reports/302338
* dir.c (rb_push_glob): expand GlobPathValue
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@62999 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Fix setting method visibility on method wrapped with prepend
Ignore prepended modules when looking for already defined methods on a
class to set the visibility on.
[Fix GH-1834]
From: Dylan Thacker-Smith <Dylan.Smith@shopify.com>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@62918 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
thread.c: deadlock in backtrace
* thread.c (unblock_function_set): check interrupts just once
during raising exceptions, as they are deferred since r16651.
[ruby-core:85939] [Bug #14577]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@62917 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
vm_insnhelper.c: rb_autoloading_value flag
* vm_insnhelper.c (vm_get_ev_const): add flag argument of
`rb_autoloading_value`.
* constant.h (rb_autoloading_value): moved the declaration from
vm_core.h for `rb_const_flag_t`. [ruby-core:85516] [Bug #14469]
variable.c: flags at autoloading
* variable.c (const_tbl_update): flags by deprecate_constant /
private_constant set during autoloading should be preserved
after required. [ruby-core:85516] [Bug #14469]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@62916 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
resolv.rb: close socket
* lib/resolv.rb (UnconnectedUDP#lazy_initialize): store new
sockets before binding, so the sockets get closed when the
requester is closing.
* lib/resolv.rb (ConnectedUDP#lazy_initialize): ditto.
* lib/resolv.rb (UnconnectedUDP#close): synchronize to get rid of
race condition.
* lib/resolv.rb (ConnectedUDP#close): ditto.
[ruby-core:85901] [Bug #14571]
From: quixoten (Devin Christensen) <quixoten@gmail.com>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@62915 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
file.c: realpath on special symlink
* file.c (realpath_rec): fallback to symlink path when it is
accessible but the link target is not actual entry on file
systems. [ruby-dev:50487] [Bug #14557]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@62903 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
usa