Commit Graph
100 Commits
Author SHA1 Message Date
TOMITA Masahiro 10a68210b4 [DOC] Fix IO::Buffer document 2025-12-24 11:14:29 +09:00
Nobuyoshi Nakada 61bab18890 Rename to struct rbimpl_size_overflow_tag
This struct is used for addition not only for multiplication, so
remove the word `mul`, and make the member names more descriptive.
2025-12-17 17:35:58 +09:00
Nobuyoshi Nakada 7c402d2c27 IO::Buffer: Warn as experimental at allocation
Previously, warned only in `new` and `map`, but not `for` and
`string`.
2025-12-17 12:35:38 +09:00
Nobuyoshi Nakada c353b62529 [Bug #21787] IO::Buffer: Check addition overflows
https://hackerone.com/reports/3437743
2025-12-17 12:35:38 +09:00
Nobuyoshi Nakada 9519d16381 IO::Buffer: Guard arguments from GC
At least, `string` in `io_buffer_set_string` can be different from
`argv[0]` after `rb_str_to_str` call.  The other cases may not be
necessary.
2025-12-17 12:35:38 +09:00
Peter Zhu 55ea3ec00f Fix strict aliasing warning in rb_int128_to_numeric
If we don't have uint128, then rb_int128_to_numeric emits a strict
aliasing warning:

    numeric.c:3641:39: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
    3641 |         return rb_uint128_to_numeric(*(rb_uint128_t*)&n);
         |                                       ^~~~~~~~~~~~~~~~~
2025-12-08 15:01:05 -08:00
Peter Zhu 2e828dd98f Fix strict aliasing warning in ruby_swap128_int
The following warnings are emitted. We can use type punning to prevent
strict aliasing violations.

    io_buffer.c:1935:23: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
    1935 |     rb_uint128_t u = *(rb_uint128_t*)&x;
         |                       ^~~~~~~~~~~~~~~~~
    io_buffer.c:1937:13: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
    1937 |     return *(rb_int128_t*)&swapped;
         |
2025-12-06 08:31:41 -08:00
Samuel Williams a7dc53b91c Add support for u128, U128, s128 and S128 integers to IO::Buffer. (#15399) 2025-12-06 15:55:32 +13:00
TOMITA Masahiro 2e770cdf77 Fix argument handling in IO::Buffer#each_byte (#15309)
The method incorrectly ignored its first argument and treated the second argument as offset and the third as count.
This change makes the first argument be treated as offset and the second as count.

Also fix incorrect block parameter in comments.
2025-11-27 15:10:43 +13:00
Alexander Bulancov aa9e15cb1e Fix multiple bugs in IO::Buffer.map and update its documentation. (#15264)
- Buffer's size did not account for offset when mapping the file, leading to possible crashes.
- Size and offset were not checked properly, leading to many situations raising EINVAL errors with generic messages.
- Documentation was wrong.
2025-11-21 00:30:42 +00:00
John HawthornandLuke Gruber 3cd2407045 Don't call gc_mark from IO::buffer compact
Previously on our mark_and_move we were calling rb_gc_mark, which isn't
safe to call at compaction time.

Co-authored-by: Luke Gruber <luke.gru@gmail.com>
2025-10-02 13:24:00 -07:00
Kasumi Hanazuki 8aac19d598 io_buffer: Reimplement dcompact for IO::Buffer
The `source` field in IO::Buffer can have a String or an IO::Buffer
object, if not nil.

- When the `source` is a String object. The `base` field points to the
  memory location of the String content, which can be embedded in
  RSTRING, and in that case, GC compaction can move the memory region
  along with the String object.

  Thus, IO::Buffer needs to pin the `source` object to prevent `base`
  pointer from becoming invalid.

- When the `source` is an IO::Buffer, then `base` is a pointer to a
  malloced or mmapped memory region, managed by the source IO::Buffer.
  In this case, we don't need to pin the source IO::Buffer object,
  since the referred memory region won't get moved by GC.

Closes: [Bug #21210]
2025-06-17 07:57:02 +02:00
Kasumi Hanazuki 4cc58c3a6f Revert "Mark rb_io_buffer_type references declaratively"
This reverts commit 6012145299.
2025-06-17 07:57:02 +02:00
Benoit Daloze 2956573b09 Add test for IO::Buffer.for(frozen_string) {} and omit rb_str_{,un}locktmp in that case 2025-06-16 22:59:10 +02:00
Samuel Williams 73c9d6ccaa Allow IO#close to interrupt IO operations on fibers using fiber_interrupt hook. (#12839) 2025-05-23 14:55:05 +09:00
Peter Zhu 6d70847e27 Make free, memsize functions of IO::Buffer static
They are not used outside of io_buffer.c, so we can make them static.
2025-03-31 16:37:44 -04:00
Matt Valentine-House 6012145299 Mark rb_io_buffer_type references declaratively
allowing them to support compaction and be moved.
2025-03-31 17:10:37 +01:00
Peter Zhu 7341a4fc07 [Bug #20933] Fix IO::Buffer overlap calculation
The allocated buffers may be consecutive memory addresses. This will mean
that `b->base == a->base + a->size` even though `a` and `b` are separate
buffers.
2024-12-09 11:52:02 -05:00
Kunshan Wang 8ae7c22972 Annotate anonymous mmap
Use PR_SET_VMA_ANON_NAME to set human-readable names for anonymous
virtual memory areas mapped by `mmap()` when compiled and run on Linux
5.17 or higher.  This makes it convenient for developers to debug mmap.
2024-11-21 13:48:05 -05:00
Samuel Williams 3c0b09ac9e Allow io_buffer_memmove to release the GVL for large buffers. (#12021)
[Feature #20902]
2024-11-20 21:27:16 +13:00
Kasumi Hanazuki 35bf660337 io_buffer.c: Allow copies between overlapping buffers with #copy and #set_string (#11640)
The current implementation of `IO::Buffer#copy` and `#set_string` has
an undefined behavior when the source and destination memory overlaps,
due to the underlying use of the `memcpy` C function.

This patch guarantees the methods to be safe even when copying between
overlapping buffers by replacing `memcpy` with `memmove`,

Fixes: [Bug #20745]
2024-11-06 20:55:39 +00:00
Samuel Williams c50298d7d4 Introduce rb_io_blocking_region which takes struct rb_io argument. (#11795)
This does not change any actual behaviour, but provides a choke point for blocking IO operations.

* Update `IO::Buffer` to use `rb_io_blocking_region`.

* Update `File` to use `rb_io_blocking_region`.

* Update `IO` to use `rb_io_blocking_region`.
2024-10-05 15:10:12 +13:00
Samuel Williams 75c4ac3395 Update IO::Buffer documentation. (#11737)
Update documentation.
2024-10-03 15:33:48 +13:00
Nobuyoshi Nakada 35e124832e [Bug #20755] Frozen string should not be writable via IO::Buffer 2024-10-01 18:46:35 +09:00
Nobuyoshi Nakada 637067440f [Bug #20752] Slice of readonly IO::Buffer also should be readonly 2024-09-30 20:39:14 +09:00
TOMITA Masahiro 48ebd77e59 Fix comment: Buffer.for(string) without block returns readonly buffer 2024-05-19 23:24:50 +09:00
TOMITA Masahiro cb43540d53 Fix IO::Buffer comment 2024-05-19 23:24:50 +09:00
Nobuyoshi Nakada d6564615e7 Fix format specifier for DWORD 2024-05-09 10:11:19 +09:00
Samuel Williams c5cf4d4e12 Improve behavioural consistency of unallocated (zero length) IO::Buffer. (#9532)
This makes the behaviour of IO::Buffer.new(0) and IO::Buffer.new.slice(0, 0) consistent.

Fixes https://bugs.ruby-lang.org/issues/19542 and https://bugs.ruby-lang.org/issues/18805.
2024-01-15 10:47:13 +13:00
Samuel Williams e5a4f757bd Fix Window private file mapping unlink EACCES issue. (#9358)
* Don't return early.

* Add missing `mapping` assignment.

* Make debug logs conditional.
2023-12-27 00:36:56 +13:00
Samuel Williams 260bf60e52 Correctly release the underlying file mapping. (#9340)
* Avoiding using `Tempfile` which was retaining the file preventing it from unlinking.
2023-12-25 14:20:53 +13:00
Samuel Williams 37753f163e IO::Buffer improvements and documentation. (#9329)
* Restore experimental warnings.

* Documentation and code structure improvements.

* Improved validation of flags, clarified documentation of argument handling.

* Remove inconsistent use of `Example:` and add example to `null?`.

* Expose `IO::Buffer#private?` and add test.
2023-12-25 02:03:36 +13:00
Peter Zhu 8e6f63df47 Implement IO::Buffer on VWA 2023-12-19 09:13:20 -05:00
Nobuyoshi Nakada 835f1461f7 [DOC] Add document of IO::Buffer#hexdump 2023-12-18 20:17:44 +09:00
Peter Zhu c5e3d6da9c Implement Write Barriers on IO::Buffer
Benchmark:

```
require "benchmark"

puts(Benchmark.measure do
  ary = 1_000_000.times.map { IO::Buffer.for("") }

  10.times { GC.start(full_mark: false) }
end)
```

Before:

```
14.330119   0.051497  14.381616 ( 14.445106)
```

After:

```
7.481152   0.040166   7.521318 (  7.535209)
```
2023-12-14 12:22:38 -05:00
Peter Zhu 2350c79462 Use xfree for IO::Buffer
Since IO::Buffer is allocated using TypedData_Make_Struct, it must use
xfree to free the buffer otherwise it will cause more major GC to run.

Example:

```
10.times do
  1_000_000.times { IO::Buffer.new(0) }

  puts "oldmalloc_increase_bytes: #{GC.stat(:oldmalloc_increase_bytes)}, major_gc_count: #{GC.stat(:major_gc_count)}"
end
```

Before:

```
oldmalloc_increase_bytes: 14904176, major_gc_count: 3
oldmalloc_increase_bytes: 2399424, major_gc_count: 5
oldmalloc_increase_bytes: 5204640, major_gc_count: 6
oldmalloc_increase_bytes: 2199936, major_gc_count: 7
oldmalloc_increase_bytes: 34199936, major_gc_count: 7
oldmalloc_increase_bytes: 24223360, major_gc_count: 8
oldmalloc_increase_bytes: 5967616, major_gc_count: 9
oldmalloc_increase_bytes: 37967616, major_gc_count: 9
oldmalloc_increase_bytes: 9689792, major_gc_count: 10
oldmalloc_increase_bytes: 41689792, major_gc_count: 10
```

After:

```
oldmalloc_increase_bytes: 117392, major_gc_count: 2
oldmalloc_increase_bytes: 26128, major_gc_count: 2
oldmalloc_increase_bytes: 71600, major_gc_count: 2
oldmalloc_increase_bytes: 117072, major_gc_count: 2
oldmalloc_increase_bytes: 17296, major_gc_count: 2
oldmalloc_increase_bytes: 62768, major_gc_count: 2
oldmalloc_increase_bytes: 108240, major_gc_count: 2
oldmalloc_increase_bytes: 153712, major_gc_count: 2
oldmalloc_increase_bytes: 53936, major_gc_count: 2
oldmalloc_increase_bytes: 99408, major_gc_count: 2
```
2023-12-12 18:47:58 -05:00
Samuel Williams 00467699a9 Don't warn generally for IO::Buffer, only on specific code paths e.g. map. (#9131) 2023-12-06 13:42:45 +13:00
Samuel Williams a095740fed Fix io_buffer_get_string default length computation. (#8427)
* Fix `io_buffer_get_string` default length computation.

When an offset bigger than the size is given, the resulting length will be
computed incorrectly. Raise an argument error in this case.

* Validate all arguments.
2023-09-14 20:37:43 +12:00
Nobuyoshi Nakada 19346c2336 [Bug #19754] Make IO::Buffer#get_string check offset range (#8016) 2023-09-13 09:45:26 +12:00
Samuel Williams f0b43597ff [DOC] Improved documentation. (#8319) 2023-08-29 23:53:28 +12:00
Samuel Williams bf1bc5362e Improve read/write/pread/pwrite consistency. (#7860)
* Documentation consistency.

* Improve consistency of `pread`/`pwrite` implementation when given length.

* Remove HAVE_PREAD / HAVE_PWRITE - it is no longer optional.
2023-05-27 18:48:47 +09:00
Samuel Williams 135a0d26a1 Improvements to IO::Buffer read/write/pread/pwrite. (#7826)
- Fix IO::Buffer `read`/`write` to use a minimum length.
2023-05-24 10:17:35 +09:00
Samuel Williams bd786e7896 Fix mutation on shared strings. (#7837) 2023-05-22 12:58:17 +09:00
Samuel Williams bcb16eb1e5 Rename data -> buffer for better readability. (#7836) 2023-05-22 12:27:20 +09:00
Samuel Williams 648870b5c5 Support IO#pread / IO#pwrite using fiber scheduler. (#7594)
* Skip test if non-blocking file IO is not supported.
2023-03-31 00:48:55 +13:00
Samuel Williams 466aa8010f Fix incorrect usage of rb_fiber_scheduler_io_(p)(read|write). (#7593) 2023-03-25 18:36:27 +13:00
Kasumi Hanazuki 09295ea796 IO::Buffer#resize: Free internal buffer if new size is zero (#7569)
`#resize(0)` on an IO::Buffer with internal buffer allocated will
result in calling `realloc(data->base, 0)`. The behavior of `realloc`
with size = 0 is implementation-defined (glibc frees the object
and returns NULL, while BSDs return an inaccessible object). And
thus such usage is deprecated in standard C (upcoming C23 will make it
UB).

To avoid this problem, just `free`s the memory when the new size is zero.
2023-03-25 12:12:23 +13:00
John Bampton 2f7270c681 Fix spelling (#7389) 2023-02-27 09:56:06 -08:00
Nobuyoshi Nakada ef00c6da88 Adjust else style to be consistent in each files [ci skip] 2023-02-26 13:20:43 +09:00
Samuel Williams d0031db9b3 Prefer RB_NUM2LONG for string length. (#7379) 2023-02-25 23:04:10 +13:00