305 Commits
Author SHA1 Message Date
Hiroshi SHIBATAandClaude Opus 5 10f5cc61c1 Use rb_len_t for lengths and offsets in pack.c
Native `l!` and `L!` directives still pack C `long`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-09 14:00:15 +09:00
Hiroshi SHIBATAandClaude Opus 5.5 ab5a81ebec Remove leftover checks for unsupported platforms
Ruby 1.9.1 dropped DJGPP, Windows CE and VMS, NeXTSTEP, OpenStep and Rhapsody went in 2011, and Ruby 2.3 dropped OS/2, BeOS and Borland C. None of these names appears in RUBY_PLATFORM or host_os on a supported platform.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 07:29:08 +09:00
Peter Zhu 413145a238 Fix crash in Array#pack when format string modified
If the format string is modified during Array#pack (by conversion such
as to_str, to_int, etc.), it can cause an use-after-free since reading
p[-1] may be an use-after-free. We should read that byte before performing
the conversion.

The following script crashes:

    fmt = "a" + " " * 1_000_000
    evil = Object.new
    evil.define_singleton_method(:to_str) { fmt.clear; "A" }
    [evil].pack(fmt)
2026-09-25 17:54:40 +09:00
Peter Zhu 852b24345a Fix use-after-free in String#unpack
If the source string is modified in the block of String#unpack, it can
cause an use-after-free. For example, the following script crashes:

    fmt = "A" * 1_000_000
    str = "C" * 1_000_000
    str.unpack(fmt) { fmt.clear }
2026-09-23 14:53:49 +09:00
Peter Zhu 261685fd25 Fix out-of-bounds in Array#pack for r/R when buffer modified
When the string buffer is modified during the to_int call in Array#pack
with the r/R directive, it can cause an out-of-bounds write. For example,
the following script crashes:

    str = "x" * 10000
    obj = Object.new
    obj.define_singleton_method(:to_int) do
      str.clear
      1
    end
    [obj].pack("R", buffer: str)
2026-09-18 07:36:37 +09:00
Peter Zhu 685144e3d6 Fix crash when source string modified in String#unpack
[Bug #22315]

The following script crashes because there is a use-after-free on the
source string during String#unpack:

    s = "C" * 4000000
    s.unpack("L*") { s.clear }

String#unpack will now instead raise RuntimeError with message "string modified".
2026-09-16 10:22:59 +09:00
Nobuyoshi Nakada e9444aa0da [Feature #22185] Add alignment directives to pack templates 2026-07-15 22:38:32 +09:00
Nobuyoshi Nakada 835ad5c09e Fix a potential out-of-bounds read 2026-07-09 20:46:27 +09:00
Nobuyoshi Nakada edf0ea0355 Fix indents [ci skip] 2026-07-09 20:46:27 +09:00
Nobuyoshi Nakada 1f657355ab pack.c: Convert the mandatory arguments first
Convert optional argument `offset` after the mandatory arguments have
done.
2026-07-08 21:25:45 +09:00
Nobuyoshi Nakada ee09d07d54 pack.c: Expand the result array in advance
Expand the room for all items at once before unpacking them, instead
of extending it one by one.
2026-07-08 21:25:45 +09:00
Nobuyoshi Nakada cf93b84bb9 Make is_bigendian for DYNAMIC_ENDIAN a compile-time invariant 2026-06-26 21:53:14 +09:00
Nobuyoshi Nakada eb053e7446 [Feature #21979] Allow negative offset in unpack 2026-05-11 13:54:09 +09:00
Nobuyoshi Nakada b21043f8ac Constify pack functions 2026-04-15 11:41:59 +09:00
Nobuyoshi Nakada 93f1010f70 Packing the buffer into itself is not possible
Reported at https://hackerone.com/reports/3601645.
2026-04-14 23:42:38 +09:00
Nobuyoshi Nakada 67d4396dc9 Refine Array#pack r/R directives
* remove the temporary buffer object.
* simplify the condition under which an extra byte is required for
  sign extension.
* in the case of `R`, raise an error earlier before packing for the
  negative number.
2026-03-05 21:31:35 +09:00
Nobuyoshi Nakada 98269b6d64 [Feature #21796] unpack variant ^ that returns the final offset (#15647)
[Feature #21796] unpack variant `^` that returns the current offset
2026-02-12 15:40:46 -08:00
Aaron Patterson c61f52a012 [Feature #21785] Add LEB128 again (#16123)
* Revert "Revert pack/unpack support for LEB128"

This reverts commit 77c3a9e447.

* Update specs for LEB128
2026-02-12 23:11:39 +00:00
Chris Hasiński a071078e90 pack.c: remove wasted allocation in BER integer packing
The 'w' format (BER compressed integer) was allocating an empty
string with rb_str_new(0, 0) then immediately overwriting it with
the correctly-sized allocation. Remove the wasted first allocation.

~50% improvement on BER pack benchmarks.
2026-01-11 21:42:37 +09:00
Luke Jahnke 4e0bb58a0a fix underflow 2025-12-29 17:21:33 +09:00
Nobuyoshi Nakada 77c3a9e447 Revert pack/unpack support for LEB128
https://bugs.ruby-lang.org/issues/21785#note-10
> It is too late to introduce it in Ruby 4.0, let's aim for 4.1.

This reverts commits:

* d0b72429a9
  Add support for signed and unsigned LEB128 to pack/unpack.

* 68a900e30b
  add news for pack / unpack directives
2025-12-20 16:59:51 +09:00
Aaron Patterson d0b72429a9 Add support for signed and unsigned LEB128 to pack/unpack.
This commit adds a new pack format command `R` and `r` for unsigned and
signed LEB128 encoding.  The "r" mnemonic is because this is a
"vaRiable" length encoding scheme.

LEB128 is used in various formats including DWARF, WebAssembly, MQTT,
and Protobuf.

[Feature #21785]
2025-12-18 14:42:47 -08:00
Nobuyoshi Nakada 066ac0fdc2 Expand buffer at once then fill it at once 2024-09-29 18:55:15 +09:00
Nobuyoshi Nakada 3b43585a84 Extract common code from pack_pack and pack_unpack_internal 2024-09-29 18:45:47 +09:00
49fcd33e13 Introduce a specialize instruction for Array#pack
Instructions for this code:

```ruby
  # frozen_string_literal: true

[a].pack("C")
```

Before this commit:

```
== disasm: #<ISeq:<main>@test.rb:1 (1,0)-(3,13)>
0000 putself                                                          (   3)[Li]
0001 opt_send_without_block                 <calldata!mid:a, argc:0, FCALL|VCALL|ARGS_SIMPLE>
0003 newarray                               1
0005 putobject                              "C"
0007 opt_send_without_block                 <calldata!mid:pack, argc:1, ARGS_SIMPLE>
0009 leave
```

After this commit:

```
== disasm: #<ISeq:<main>@test.rb:1 (1,0)-(3,13)>
0000 putself                                                          (   3)[Li]
0001 opt_send_without_block                 <calldata!mid:a, argc:0, FCALL|VCALL|ARGS_SIMPLE>
0003 putobject                              "C"
0005 opt_newarray_send                      2, :pack
0008 leave
```

Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com>
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
2024-05-23 12:11:50 -07:00
Nobuyoshi Nakada 232e401b0f Reuse natstr if it equals endstr 2023-10-06 16:33:44 +09:00
Nobuyoshi Nakada f36178006b Fix a typo in BUG message [ci skip] 2023-01-20 00:20:27 +09:00
Peter Zhu 29a5fcd60b Add RBIMPL_ATTR_NORETURN to unknown_directive
Fixes a compiler warning about the function unknown_directive:

pack.c:160:1: warning: function 'unknown_directive' could be declared with attribute 'noreturn' [-Wmissing-noreturn]
2023-01-05 14:55:14 -05:00
Benoit Daloze 03dd37775a Raise an ArgumentError for unknown pack/unpack directive
* [Bug #19150]
2023-01-05 19:05:03 +01:00
Yusuke Endoh ee7a338d2b Unconditionally warn "unknown pack/unpack directive"
[Bug #19150]
2022-12-16 12:15:50 +01:00
Nobuyoshi Nakada 9869bd1d61 [Bug #19108] Check for the encoding of pack/unpack format 2022-12-01 17:38:33 +09:00
Jeremy Evans 1340195e79 Reduce duplication in pack by sharing code for some cases
s/S, i/I, l/L, and q/Q had the same code in both cases, so
combine the cases.

Alternatively, we could actually the size of the unsigned type,
but I doubt there are any platforms where the unsigned type is
a different size than the signed type.
2022-11-24 14:27:19 -08:00
Nobuyoshi Nakada 20b9d7b9fd Use enum over int 2022-11-22 14:20:45 +09:00
Jemma Issroff 199b59f065 Fix bug in array pack with shared strings
If string literals are long and they become shared, we need to make them
independent before we can write to them. [Bug #19116]
2022-11-10 09:26:37 -05:00
Takashi Kokubun 5b21e94beb Expand tabs [ci skip]
[Misc #18891]
2022-07-21 09:42:04 -07:00
Jean Boussier e5319dc985 pack.c: add an offset argument to unpack and unpack1
[Feature #18254]

This is useful to avoid repeteadly copying strings when parsing binary formats
2021-10-26 22:27:30 +02:00
Nobuyoshi Nakada b69c38e62e Refactor associated pointer 2021-10-24 17:50:35 +09:00
Nobuyoshi Nakada 4446942f1a Extract UNPACK_FETCH 2021-10-22 12:47:34 +09:00
Nobuyoshi Nakada c38c2d8ee2 Moved exported symbols in internal/util.h to ruby/util.h
[Feature #18051]
2021-08-24 10:37:41 +09:00
Nobuyoshi Nakada eec45a93ef Escape unprintable chars only, without surrounding quotes 2021-07-24 14:31:41 +09:00
卜部昌平 ff30358d13 RARRAY_AREF: convert into an inline function
RARRAY_AREF has been a macro for reasons.  We might not be able to
change that for public APIs, but why not relax the situation internally
to make it an inline function.
2020-08-15 12:09:26 +09:00
卜部昌平 9e41a75255 sed -i 's|ruby/impl|ruby/internal|'
To fix build failures.
2020-05-11 09:24:08 +09:00
卜部昌平 d7f4d732c1 sed -i s|ruby/3|ruby/impl|g
This shall fix compile errors.
2020-05-11 09:24:08 +09:00
卜部昌平 9e6e39c351 Merge pull request #2991 from shyouhei/ruby.h
Split ruby.h
2020-04-08 13:28:13 +09:00
Nobuyoshi Nakada d7bef803ac Separate builtin initialization calls 2019-12-29 12:34:55 +09:00
卜部昌平 5e22f873ed decouple internal.h headers
Saves comitters' daily life by avoid #include-ing everything from
internal.h to make each file do so instead.  This would significantly
speed up incremental builds.

We take the following inclusion order in this changeset:

1.  "ruby/config.h", where _GNU_SOURCE is defined (must be the very
    first thing among everything).
2.  RUBY_EXTCONF_H if any.
3.  Standard C headers, sorted alphabetically.
4.  Other system headers, maybe guarded by #ifdef
5.  Everything else, sorted alphabetically.

Exceptions are those win32-related headers, which tend not be self-
containing (headers have inclusion order dependencies).
2019-12-26 20:45:12 +09:00
Jeremy Evans ffd0820ab3 Deprecate taint/trust and related methods, and make the methods no-ops
This removes the related tests, and puts the related specs behind
version guards.  This affects all code in lib, including some
libraries that may want to support older versions of Ruby.
2019-11-18 01:00:25 +02:00
Koichi Sasada 3a408aff57 try to remove vm_core.h 2019-11-14 01:48:08 +09:00
Kazuhiro NISHIYAMA 00d56bdf66 Try to fix error on solaris
Add vm_core.h before builtin.h to pack.c for VM_CORE_H_EC_DEFINED.

https://rubyci.org/logs/rubyci.s3.amazonaws.com/solaris11s-sunc/ruby-master/log/20191112T102505Z.fail.html.gz
```
compiling dmyenc.c
linking miniruby
Undefined			first referenced
 symbol  			    in file
rb_vm_lvar_exposed                  pack.o
ld: fatal: symbol referencing errors
gmake: *** [miniruby] Error 2
```
2019-11-13 09:58:20 +09:00
Yusuke Endoh a3141e3c76 Rubified the APIs of pack.c 2019-11-08 23:51:51 +09:00