Nobuyoshi Nakada
d2264185fa
[ruby/json] [ruby/json] Run have_func with the header providing the declarations
...
https://github.com/ruby/json/commit/95fb084027
https://github.com/ruby/json/commit/9d080765cc
2025-07-15 09:23:42 -07:00
Jean Boussier
91efe7b64f
[ruby/json] Release 2.9.1
...
https://github.com/ruby/json/commit/f745ec145e
2024-12-19 08:45:31 +09:00
Naohisa Goto
979b19b741
[ruby/json] Add support for Solaris 10 which lacks strnlen()
...
Check for existence of strnlen() and use alternative code if it is missing.
https://github.com/ruby/json/commit/48d4bbc3a0
2024-12-19 08:45:31 +09:00
Jean Boussier
811bc15c75
[ruby/json] Release 2.9.0
...
https://github.com/ruby/json/commit/e1f6456499
2024-12-05 09:16:22 +01:00
Jean Boussier
1510d72bec
[ruby/json] Fix generate(script_safe: true) to not confuse unrelated characters
...
Fix: https://github.com/ruby/json/issues/715
The first byte check was missing.
https://github.com/ruby/json/commit/93a7f8717d
2024-12-05 09:16:22 +01:00
Yusuke Endoh
209f8ba7c4
[ruby/json] Prevent a warning of "a candidate for gnu_printf format attribute"
...
GCC 13 prints the following warning.
https://rubyci.s3.amazonaws.com/ubuntu/ruby-master/log/20241127T001003Z.log.html.gz
```
compiling generator.c
generator.c: In function ‘raise_generator_error’:
generator.c:91:5: warning: function ‘raise_generator_error’ might be a candidate for ‘gnu_printf’ format attribute [-Wsuggest-attribute=format]
91 | VALUE str = rb_vsprintf(fmt, args);
| ^~~~~
```
This change prevents the warning by specifying the format attribute.
https://github.com/ruby/json/commit/b8c1490846
2024-11-27 23:35:20 +09:00
Jean Boussier and Étienne Barrié
693a793521
JSON::GeneratorError expose invalid object
...
Fix: https://github.com/ruby/json/issues/710
Makes it easier to debug why a given tree of objects can't
be dumped as JSON.
Co-Authored-By: Étienne Barrié <etienne.barrie@gmail.com >
2024-11-26 15:11:05 +09:00
Jean Boussier
6805e88935
[ruby/json] Stop using rb_gc_mark_locations
...
It's using `rb_gc_mark_maybe` under the hood, which isn't what
we need.
https://github.com/ruby/json/commit/e10d0bffcd
2024-11-26 15:11:05 +09:00
Jean Boussier
ee0de3fd4e
[ruby/json] JSON.dump: write directly into the provided IO
...
Ref: https://github.com/ruby/json/issues/524
Rather than to buffer everything in memory.
Unfortunately Ruby doesn't provide an API to write into
and IO without first allocating a string, which is a bit
wasteful.
https://github.com/ruby/json/commit/f017af6c0a
2024-11-26 15:11:05 +09:00
Jean Boussier
9c6217fd05
[ruby/json] Fix the BEWARE documentation in load and unsafe_load.
...
https://github.com/ruby/json/commit/2d62ec449f
2024-11-18 04:23:18 +01:00
Jean Boussier
f3e17a84f4
[ruby/json] Release 2.8.2
...
https://github.com/ruby/json/commit/d5e4a6e3fd
2024-11-18 04:23:18 +01:00
razokulover
62e1469e8b
[ruby/json] Fix redundant to_str call
...
https://github.com/ruby/json/commit/61f022dfbd
2024-11-18 04:23:18 +01:00
Jean Boussier
a4183781ec
[ruby/json] Only use the key cache if the Hash is in an Array
...
Otherwise the likeliness of seeing that key again is really low, and looking up
the cache is just a waste.
Before:
```
== Parsing small hash (65 bytes)
ruby 3.4.0dev (2024-11-13T12:32:57Z fstr-update-callba.. https://github.com/ruby/json/commit/9b44b455b3 ) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 343.049k i/100ms
oj 213.943k i/100ms
Oj::Parser 31.583k i/100ms
rapidjson 303.433k i/100ms
Calculating -------------------------------------
json 3.704M (± 1.5%) i/s (270.01 ns/i) - 18.525M in 5.003078s
oj 2.200M (± 1.1%) i/s (454.46 ns/i) - 11.125M in 5.056526s
Oj::Parser 285.369k (± 4.8%) i/s (3.50 μs/i) - 1.453M in 5.103866s
rapidjson 3.216M (± 1.6%) i/s (310.95 ns/i) - 16.082M in 5.001973s
Comparison:
json: 3703517.4 i/s
rapidjson: 3215983.0 i/s - 1.15x slower
oj: 2200417.1 i/s - 1.68x slower
Oj::Parser: 285369.1 i/s - 12.98x slower
== Parsing test from oj (258 bytes)
ruby 3.4.0dev (2024-11-13T12:32:57Z fstr-update-callba.. https://github.com/ruby/json/commit/9b44b455b3 ) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 54.539k i/100ms
oj 41.473k i/100ms
Oj::Parser 24.064k i/100ms
rapidjson 51.466k i/100ms
Calculating -------------------------------------
json 549.386k (± 1.6%) i/s (1.82 μs/i) - 2.781M in 5.064316s
oj 417.003k (± 1.3%) i/s (2.40 μs/i) - 2.115M in 5.073047s
Oj::Parser 226.500k (± 4.7%) i/s (4.42 μs/i) - 1.131M in 5.005466s
rapidjson 526.124k (± 1.0%) i/s (1.90 μs/i) - 2.676M in 5.087176s
Comparison:
json: 549385.6 i/s
rapidjson: 526124.3 i/s - 1.04x slower
oj: 417003.4 i/s - 1.32x slower
Oj::Parser: 226500.4 i/s - 2.43x slower
```
After:
```
== Parsing small hash (65 bytes)
ruby 3.4.0dev (2024-11-13T12:32:57Z fstr-update-callba.. https://github.com/ruby/json/commit/9b44b455b3 ) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 361.394k i/100ms
oj 217.203k i/100ms
Oj::Parser 28.855k i/100ms
rapidjson 303.404k i/100ms
Calculating -------------------------------------
json 3.859M (± 2.9%) i/s (259.13 ns/i) - 19.515M in 5.061302s
oj 2.191M (± 1.6%) i/s (456.49 ns/i) - 11.077M in 5.058043s
Oj::Parser 315.132k (± 7.1%) i/s (3.17 μs/i) - 1.587M in 5.065707s
rapidjson 3.156M (± 4.0%) i/s (316.88 ns/i) - 15.777M in 5.008949s
Comparison:
json: 3859046.5 i/s
rapidjson: 3155778.5 i/s - 1.22x slower
oj: 2190616.0 i/s - 1.76x slower
Oj::Parser: 315132.4 i/s - 12.25x slower
== Parsing test from oj (258 bytes)
ruby 3.4.0dev (2024-11-13T12:32:57Z fstr-update-callba.. https://github.com/ruby/json/commit/9b44b455b3 ) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 55.682k i/100ms
oj 40.343k i/100ms
Oj::Parser 25.119k i/100ms
rapidjson 51.500k i/100ms
Calculating -------------------------------------
json 555.808k (± 1.4%) i/s (1.80 μs/i) - 2.784M in 5.010092s
oj 412.283k (± 1.7%) i/s (2.43 μs/i) - 2.098M in 5.089900s
Oj::Parser 279.306k (±13.3%) i/s (3.58 μs/i) - 1.356M in 5.022079s
rapidjson 517.177k (± 2.7%) i/s (1.93 μs/i) - 2.626M in 5.082352s
Comparison:
json: 555808.3 i/s
rapidjson: 517177.1 i/s - 1.07x slower
oj: 412283.2 i/s - 1.35x slower
Oj::Parser: 279306.5 i/s - 1.99x slower
```
https://github.com/ruby/json/commit/00c45ddc9f
2024-11-14 11:21:39 +09:00
Jean Boussier
1b6a26e23a
[ruby/json] JSON.load_file: explictly load the file as UTF-8
...
Fix: https://github.com/ruby/json/issues/697
This way even if `Encoding.default_external` is set to a weird value
the document will be parsed just fine.
https://github.com/ruby/json/commit/3a8505a8fa
2024-11-11 09:40:11 +09:00
Jean Boussier
58317328b6
[ruby/json] Rename parse_float into parse_number
...
https://github.com/ruby/json/commit/e51e796697
2024-11-11 09:40:10 +09:00
Aaron Patterson
c991f75c19
[ruby/json] Reduce comparisons when parsing numbers
...
Before this commit, we would try to scan for a float, then if that
failed, scan for an integer. But floats and integers have many bytes in
common, so we would end up scanning the same bytes multiple times.
This patch combines integer and float scanning machines so that we only
have to scan bytes once. If the machine finds "float parts", then it
executes the "isFloat" transition in the machine, which sets a boolean
letting us know that the parser found a float.
If we didn't find a float, but we did match, then we know it's an int.
https://github.com/ruby/json/commit/0c0e0930cd
2024-11-11 09:40:10 +09:00
Jean Boussier
f1be046a1d
[ruby/json] Release 2.8.1
...
https://github.com/ruby/json/commit/49650f7312
2024-11-11 09:40:09 +09:00
Jean Boussier
a78cb959ec
[ruby/json] Fix gemspec to include .jar files
...
Fix: https://github.com/ruby/json/issues/694
This was lost during the .gemspec merge and not noticed
because it was falling back to loading the jars from the
stdlib.
https://github.com/ruby/json/commit/07cf261251
2024-11-11 09:40:08 +09:00
Jean Boussier
e4c15b8ace
[ruby/json] Release 2.8.0
...
https://github.com/ruby/json/commit/8071cc6f09
2024-11-06 23:31:30 +01:00
Jean Boussier
d188a6883f
[ruby/json] Implement a fast path for integer parsing
...
`rb_cstr2inum` isn't very fast because it handles tons of
different scenarios, and also require a NULL terminated string
which forces us to copy the number into a secondary buffer.
But since the parser already computed the length, we can much more
cheaply do this with a very simple function as long as the number
is small enough to fit into a native type (`long long`).
If the number is too long, we can fallback to the `rb_cstr2inum`
slowpath.
Before:
```
== Parsing citm_catalog.json (1727030 bytes)
ruby 3.4.0dev (2024-11-06T07:59:09Z precompute-hash-wh.. https://github.com/ruby/json/commit/7943f98a8a ) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 40.000 i/100ms
oj 35.000 i/100ms
Oj::Parser 45.000 i/100ms
rapidjson 38.000 i/100ms
Calculating -------------------------------------
json 425.941 (± 1.9%) i/s (2.35 ms/i) - 2.160k in 5.072833s
oj 349.617 (± 1.7%) i/s (2.86 ms/i) - 1.750k in 5.006953s
Oj::Parser 464.767 (± 1.7%) i/s (2.15 ms/i) - 2.340k in 5.036381s
rapidjson 382.413 (± 2.4%) i/s (2.61 ms/i) - 1.938k in 5.070757s
Comparison:
json: 425.9 i/s
Oj::Parser: 464.8 i/s - 1.09x faster
rapidjson: 382.4 i/s - 1.11x slower
oj: 349.6 i/s - 1.22x slower
```
After:
```
== Parsing citm_catalog.json (1727030 bytes)
ruby 3.4.0dev (2024-11-06T07:59:09Z precompute-hash-wh.. https://github.com/ruby/json/commit/7943f98a8a ) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 46.000 i/100ms
oj 33.000 i/100ms
Oj::Parser 45.000 i/100ms
rapidjson 39.000 i/100ms
Calculating -------------------------------------
json 462.332 (± 3.2%) i/s (2.16 ms/i) - 2.346k in 5.080504s
oj 351.140 (± 1.1%) i/s (2.85 ms/i) - 1.782k in 5.075616s
Oj::Parser 473.500 (± 1.3%) i/s (2.11 ms/i) - 2.385k in 5.037695s
rapidjson 395.052 (± 3.5%) i/s (2.53 ms/i) - 1.989k in 5.042275s
Comparison:
json: 462.3 i/s
Oj::Parser: 473.5 i/s - same-ish: difference falls within error
rapidjson: 395.1 i/s - 1.17x slower
oj: 351.1 i/s - 1.32x slower
```
https://github.com/ruby/json/commit/3a4dc9e1b4
2024-11-06 23:31:30 +01:00
Nobuyoshi Nakada
29d76d8c8b
[ruby/json] Fix right shift warnings
...
Ignoring `CHAR_BITS` > 8 platform, as far as `ch` indexes
`escape_table` that is hard-coded as 256 elements.
```
../../../../src/ext/json/generator/generator.c(121): warning C4333: '>>': right shift by too large amount, data loss
../../../../src/ext/json/generator/generator.c(122): warning C4333: '>>': right shift by too large amount, data loss
../../../../src/ext/json/generator/generator.c(243): warning C4333: '>>': right shift by too large amount, data loss
../../../../src/ext/json/generator/generator.c(244): warning C4333: '>>': right shift by too large amount, data loss
../../../../src/ext/json/generator/generator.c(291): warning C4333: '>>': right shift by too large amount, data loss
../../../../src/ext/json/generator/generator.c(292): warning C4333: '>>': right shift by too large amount, data loss
```
https://github.com/ruby/json/commit/fb82373612
2024-11-06 23:31:30 +01:00
Jean Boussier
6cea370b23
[ruby/json] parser.rl: parse_string implement a fast path
...
If we assume most string don't contain any escape sequence we can avoid
a lot of costly operations when it holds true.
Before:
```
== Parsing activitypub.json (58160 bytes)
ruby 3.4.0dev (2024-11-06T07:59:09Z precompute-hash-wh.. https://github.com/ruby/json/commit/7943f98a8a ) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 884.000 i/100ms
oj 789.000 i/100ms
Oj::Parser 943.000 i/100ms
rapidjson 584.000 i/100ms
Calculating -------------------------------------
json 8.897k (± 1.3%) i/s (112.40 μs/i) - 45.084k in 5.068520s
oj 7.967k (± 1.5%) i/s (125.52 μs/i) - 40.239k in 5.051985s
Oj::Parser 9.564k (± 1.4%) i/s (104.56 μs/i) - 48.093k in 5.029626s
rapidjson 5.947k (± 1.4%) i/s (168.16 μs/i) - 29.784k in 5.009437s
Comparison:
json: 8896.5 i/s
Oj::Parser: 9563.8 i/s - 1.08x faster
oj: 7966.8 i/s - 1.12x slower
rapidjson: 5946.7 i/s - 1.50x slower
== Parsing twitter.json (567916 bytes)
ruby 3.4.0dev (2024-11-06T07:59:09Z precompute-hash-wh.. https://github.com/ruby/json/commit/7943f98a8a ) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 83.000 i/100ms
oj 64.000 i/100ms
Oj::Parser 77.000 i/100ms
rapidjson 54.000 i/100ms
Calculating -------------------------------------
json 823.083 (± 1.8%) i/s (1.21 ms/i) - 4.150k in 5.043805s
oj 632.538 (± 1.4%) i/s (1.58 ms/i) - 3.200k in 5.060073s
Oj::Parser 769.122 (± 1.8%) i/s (1.30 ms/i) - 3.850k in 5.007501s
rapidjson 548.494 (± 1.5%) i/s (1.82 ms/i) - 2.754k in 5.022153s
Comparison:
json: 823.1 i/s
Oj::Parser: 769.1 i/s - 1.07x slower
oj: 632.5 i/s - 1.30x slower
rapidjson: 548.5 i/s - 1.50x slower
== Parsing citm_catalog.json (1727030 bytes)
ruby 3.4.0dev (2024-11-06T07:59:09Z precompute-hash-wh.. https://github.com/ruby/json/commit/7943f98a8a ) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 41.000 i/100ms
oj 34.000 i/100ms
Oj::Parser 45.000 i/100ms
rapidjson 39.000 i/100ms
Calculating -------------------------------------
json 427.162 (± 1.2%) i/s (2.34 ms/i) - 2.173k in 5.087666s
oj 351.463 (± 2.8%) i/s (2.85 ms/i) - 1.768k in 5.035149s
Oj::Parser 461.849 (± 3.7%) i/s (2.17 ms/i) - 2.340k in 5.074461s
rapidjson 395.155 (± 1.8%) i/s (2.53 ms/i) - 1.989k in 5.034927s
Comparison:
json: 427.2 i/s
Oj::Parser: 461.8 i/s - 1.08x faster
rapidjson: 395.2 i/s - 1.08x slower
oj: 351.5 i/s - 1.22x slower
```
After:
```
== Parsing activitypub.json (58160 bytes)
ruby 3.4.0dev (2024-11-06T07:59:09Z precompute-hash-wh.. https://github.com/ruby/json/commit/7943f98a8a ) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 953.000 i/100ms
oj 813.000 i/100ms
Oj::Parser 956.000 i/100ms
rapidjson 563.000 i/100ms
Calculating -------------------------------------
json 9.525k (± 1.2%) i/s (104.98 μs/i) - 47.650k in 5.003252s
oj 8.117k (± 0.5%) i/s (123.20 μs/i) - 40.650k in 5.008283s
Oj::Parser 9.590k (± 3.2%) i/s (104.27 μs/i) - 48.756k in 5.089794s
rapidjson 6.020k (± 0.9%) i/s (166.10 μs/i) - 30.402k in 5.050155s
Comparison:
json: 9525.3 i/s
Oj::Parser: 9590.1 i/s - same-ish: difference falls within error
oj: 8116.7 i/s - 1.17x slower
rapidjson: 6020.5 i/s - 1.58x slower
== Parsing twitter.json (567916 bytes)
ruby 3.4.0dev (2024-11-06T07:59:09Z precompute-hash-wh.. https://github.com/ruby/json/commit/7943f98a8a ) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 87.000 i/100ms
oj 64.000 i/100ms
Oj::Parser 75.000 i/100ms
rapidjson 55.000 i/100ms
Calculating -------------------------------------
json 866.563 (± 0.8%) i/s (1.15 ms/i) - 4.350k in 5.020138s
oj 643.567 (± 0.8%) i/s (1.55 ms/i) - 3.264k in 5.072101s
Oj::Parser 777.346 (± 3.5%) i/s (1.29 ms/i) - 3.900k in 5.023933s
rapidjson 557.158 (± 0.7%) i/s (1.79 ms/i) - 2.805k in 5.034731s
Comparison:
json: 866.6 i/s
Oj::Parser: 777.3 i/s - 1.11x slower
oj: 643.6 i/s - 1.35x slower
rapidjson: 557.2 i/s - 1.56x slower
== Parsing citm_catalog.json (1727030 bytes)
ruby 3.4.0dev (2024-11-06T07:59:09Z precompute-hash-wh.. https://github.com/ruby/json/commit/7943f98a8a ) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 41.000 i/100ms
oj 35.000 i/100ms
Oj::Parser 40.000 i/100ms
rapidjson 39.000 i/100ms
Calculating -------------------------------------
json 429.216 (± 1.2%) i/s (2.33 ms/i) - 2.173k in 5.063351s
oj 354.755 (± 1.1%) i/s (2.82 ms/i) - 1.785k in 5.032374s
Oj::Parser 465.114 (± 3.7%) i/s (2.15 ms/i) - 2.360k in 5.081634s
rapidjson 387.135 (± 1.3%) i/s (2.58 ms/i) - 1.950k in 5.037787s
Comparison:
json: 429.2 i/s
Oj::Parser: 465.1 i/s - 1.08x faster
rapidjson: 387.1 i/s - 1.11x slower
oj: 354.8 i/s - 1.21x slower
```
https://github.com/ruby/json/commit/96bd97c61e
2024-11-06 23:31:30 +01:00
Nobuyoshi Nakada
8254f6492c
[ruby/json] Categorize deprecated warning
...
https://github.com/ruby/json/commit/1acce7aceb
2024-11-06 23:31:30 +01:00
Jean Boussier
9987298654
Update depend files
2024-11-05 18:00:36 +01:00
Jean Boussier
ca8f21ace8
[ruby/json] Resync
2024-11-05 18:00:36 +01:00
Jean Boussier
f664e7eaab
[ruby/json] Add tests for the behavior of JSON.generate with base types subclasses
...
Ref: https://github.com/ruby/json/pull/674
Ref: https://github.com/ruby/json/pull/668
The behavior on such case it quite unclear, the goal here is to
figure out whatever was the behavior on Cext version of `json 2.7.0`
and get all implementations to converge.
We can then decide to make them all behave differently if we so wish.
https://github.com/ruby/json/commit/614921dcef
2024-11-05 18:00:36 +01:00
Jean Boussier
2f84a02ad5
[ruby/json] Use rb_str_new_frozen
...
https://github.com/ruby/json/commit/90c8aaaa6a
2024-11-05 18:00:36 +01:00
Jean Boussier
ed22e68379
[ruby/json] JSON::Ext::Parser mark the name cache entries when not on the heap
...
This is somewhat dead code as unless you are using `JSON::Parser.new`
direcltly we never allocate `JSON::Ext::Parser` anymore.
But still, we should mark all its reference in case some code out there
uses that.
Followup: #675
https://github.com/ruby/json/commit/8bf74a977b
2024-11-05 18:00:36 +01:00
Jean Boussier
ee4fa4ccee
[ruby/json] json_string_unescape: Use the returned RString as buffer
...
Rather than to copy into a buffer to unescape and then copy that
buffer into the final string, we can directly copy into the final
string.
The downside is that if the string contains a lot of escaping, we
end up returning a string that's larger than strictly necessary, but
it's probably fine.
Before:
```
== Parsing twitter.json (567916 bytes)
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec ) +YJIT [arm64-darwin23]
Warming up --------------------------------------
json 56.000 i/100ms
oj 58.000 i/100ms
oj strict 74.000 i/100ms
Oj::Parser 76.000 i/100ms
rapidjson 52.000 i/100ms
Calculating -------------------------------------
json 556.659 (± 2.9%) i/s (1.80 ms/i) - 2.800k in 5.034719s
oj 604.077 (± 3.8%) i/s (1.66 ms/i) - 3.016k in 5.001546s
oj strict 706.942 (± 3.5%) i/s (1.41 ms/i) - 3.552k in 5.030954s
Oj::Parser 752.917 (± 3.2%) i/s (1.33 ms/i) - 3.800k in 5.052707s
rapidjson 546.470 (± 3.5%) i/s (1.83 ms/i) - 2.756k in 5.049855s
Comparison:
json: 556.7 i/s
Oj::Parser: 752.9 i/s - 1.35x faster
oj strict: 706.9 i/s - 1.27x faster
oj: 604.1 i/s - 1.09x faster
rapidjson: 546.5 i/s - same-ish: difference falls within error
== Parsing citm_catalog.json (1727030 bytes)
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec ) +YJIT [arm64-darwin23]
Warming up --------------------------------------
json 29.000 i/100ms
oj 32.000 i/100ms
oj strict 38.000 i/100ms
Oj::Parser 42.000 i/100ms
rapidjson 38.000 i/100ms
Calculating -------------------------------------
json 317.858 (± 3.1%) i/s (3.15 ms/i) - 1.595k in 5.023245s
oj 348.168 (± 2.6%) i/s (2.87 ms/i) - 1.760k in 5.058431s
oj strict 394.599 (± 2.8%) i/s (2.53 ms/i) - 1.976k in 5.012073s
Oj::Parser 403.771 (± 3.0%) i/s (2.48 ms/i) - 2.058k in 5.101578s
rapidjson 383.441 (± 3.7%) i/s (2.61 ms/i) - 1.938k in 5.061355s
Comparison:
json: 317.9 i/s
Oj::Parser: 403.8 i/s - 1.27x faster
oj strict: 394.6 i/s - 1.24x faster
rapidjson: 383.4 i/s - 1.21x faster
oj: 348.2 i/s - 1.10x faster
```
After:
```
== Parsing twitter.json (567916 bytes)
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec ) +YJIT [arm64-darwin23]
Warming up --------------------------------------
json 56.000 i/100ms
oj 62.000 i/100ms
oj strict 73.000 i/100ms
Oj::Parser 76.000 i/100ms
rapidjson 54.000 i/100ms
Calculating -------------------------------------
json 561.009 (± 7.5%) i/s (1.78 ms/i) - 2.800k in 5.039548s
oj 601.124 (± 4.3%) i/s (1.66 ms/i) - 3.038k in 5.064686s
oj strict 707.455 (± 3.4%) i/s (1.41 ms/i) - 3.577k in 5.062540s
Oj::Parser 751.799 (± 3.1%) i/s (1.33 ms/i) - 3.800k in 5.059509s
rapidjson 535.641 (± 3.2%) i/s (1.87 ms/i) - 2.700k in 5.045816s
Comparison:
json: 561.0 i/s
Oj::Parser: 751.8 i/s - 1.34x faster
oj strict: 707.5 i/s - 1.26x faster
oj: 601.1 i/s - same-ish: difference falls within error
rapidjson: 535.6 i/s - same-ish: difference falls within error
== Parsing citm_catalog.json (1727030 bytes)
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec ) +YJIT [arm64-darwin23]
Warming up --------------------------------------
json 30.000 i/100ms
oj 32.000 i/100ms
oj strict 36.000 i/100ms
Oj::Parser 42.000 i/100ms
rapidjson 39.000 i/100ms
Calculating -------------------------------------
json 313.248 (± 7.3%) i/s (3.19 ms/i) - 1.560k in 5.014118s
oj 341.977 (± 4.1%) i/s (2.92 ms/i) - 1.728k in 5.063332s
oj strict 387.062 (± 6.2%) i/s (2.58 ms/i) - 1.944k in 5.045961s
Oj::Parser 400.423 (± 4.0%) i/s (2.50 ms/i) - 2.016k in 5.044513s
rapidjson 379.046 (± 6.1%) i/s (2.64 ms/i) - 1.911k in 5.064461s
Comparison:
json: 313.2 i/s
Oj::Parser: 400.4 i/s - 1.28x faster
oj strict: 387.1 i/s - 1.24x faster
rapidjson: 379.0 i/s - 1.21x faster
oj: 342.0 i/s - same-ish: difference falls within error
```
https://github.com/ruby/json/commit/5e1ec4a268
2024-11-01 13:04:24 +09:00
Jean Boussier
b85a7a44fa
[ruby/json] Trigger write barrier when setting Generator::State configs
...
Followup: https://github.com/ruby/json/commit/6382c231b0b8
https://github.com/ruby/json/commit/0c797b4a11
2024-11-01 13:04:24 +09:00
Jean Boussier
ef5565f5d1
JSON.generate: call to_json on String subclasses
...
Fix: https://github.com/ruby/json/issues/667
This is yet another behavior on which the various implementations
differed, but the C implementation used to call `to_json` on String
subclasses used as keys.
This was optimized out in e125072130229e54a651f7b11d7d5a782ae7fb65
but there is an Active Support test case for it, so it's best to
make all 3 implementation respect this behavior.
2024-11-01 13:04:24 +09:00
Jean Boussier
b8b33efd4d
[ruby/json] Remove String#-@ check in extconf.rb
...
Now that older rubies have been droped, we no longer need to check
for all that.
https://github.com/ruby/json/commit/35cf2b84e0
2024-11-01 13:04:24 +09:00
Jean Boussier
165cc6cf40
[ruby/json] json_string_unescape: assume the string doesn't need escaping
...
If that assumption holds true, then we don't need to copy the
string into a buffer to unescape it. For small string is just saves
copying, but for large ones it also saves a malloc/free combo.
Before:
```
== Parsing twitter.json (567916 bytes)
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec ) +YJIT [arm64-darwin23]
Warming up --------------------------------------
json 52.000 i/100ms
oj 61.000 i/100ms
oj strict 70.000 i/100ms
Oj::Parser 71.000 i/100ms
rapidjson 55.000 i/100ms
Calculating -------------------------------------
json 510.111 (± 2.9%) i/s (1.96 ms/i) - 2.548k in 5.000029s
oj 610.232 (± 3.1%) i/s (1.64 ms/i) - 3.050k in 5.003725s
oj strict 713.231 (± 3.2%) i/s (1.40 ms/i) - 3.570k in 5.010902s
Oj::Parser 762.598 (± 3.0%) i/s (1.31 ms/i) - 3.834k in 5.033130s
rapidjson 553.029 (± 7.4%) i/s (1.81 ms/i) - 2.750k in 5.022630s
Comparison:
json: 510.1 i/s
Oj::Parser: 762.6 i/s - 1.49x faster
oj strict: 713.2 i/s - 1.40x faster
oj: 610.2 i/s - 1.20x faster
rapidjson: 553.0 i/s - same-ish: difference falls within error
== Parsing citm_catalog.json (1727030 bytes)
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec ) +YJIT [arm64-darwin23]
Warming up --------------------------------------
json 28.000 i/100ms
oj 33.000 i/100ms
oj strict 37.000 i/100ms
Oj::Parser 43.000 i/100ms
rapidjson 38.000 i/100ms
Calculating -------------------------------------
json 303.853 (± 3.6%) i/s (3.29 ms/i) - 1.540k in 5.076079s
oj 348.009 (± 2.0%) i/s (2.87 ms/i) - 1.749k in 5.027738s
oj strict 396.679 (± 3.3%) i/s (2.52 ms/i) - 1.998k in 5.042271s
Oj::Parser 406.699 (± 2.2%) i/s (2.46 ms/i) - 2.064k in 5.077587s
rapidjson 393.463 (± 3.3%) i/s (2.54 ms/i) - 1.976k in 5.028501s
Comparison:
json: 303.9 i/s
Oj::Parser: 406.7 i/s - 1.34x faster
oj strict: 396.7 i/s - 1.31x faster
rapidjson: 393.5 i/s - 1.29x faster
oj: 348.0 i/s - 1.15x faster
```
After:
```
== Parsing twitter.json (567916 bytes)
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec ) +YJIT [arm64-darwin23]
Warming up --------------------------------------
json 56.000 i/100ms
oj 62.000 i/100ms
oj strict 72.000 i/100ms
Oj::Parser 77.000 i/100ms
rapidjson 55.000 i/100ms
Calculating -------------------------------------
json 568.025 (± 2.1%) i/s (1.76 ms/i) - 2.856k in 5.030272s
oj 630.936 (± 1.4%) i/s (1.58 ms/i) - 3.162k in 5.012630s
oj strict 705.784 (±11.2%) i/s (1.42 ms/i) - 3.456k in 5.006706s
Oj::Parser 783.989 (± 1.7%) i/s (1.28 ms/i) - 3.927k in 5.010343s
rapidjson 557.630 (± 2.0%) i/s (1.79 ms/i) - 2.805k in 5.032388s
Comparison:
json: 568.0 i/s
Oj::Parser: 784.0 i/s - 1.38x faster
oj strict: 705.8 i/s - 1.24x faster
oj: 630.9 i/s - 1.11x faster
rapidjson: 557.6 i/s - same-ish: difference falls within error
== Parsing citm_catalog.json (1727030 bytes)
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec ) +YJIT [arm64-darwin23]
Warming up --------------------------------------
json 29.000 i/100ms
oj 33.000 i/100ms
oj strict 38.000 i/100ms
Oj::Parser 43.000 i/100ms
rapidjson 37.000 i/100ms
Calculating -------------------------------------
json 319.271 (± 3.1%) i/s (3.13 ms/i) - 1.595k in 5.001128s
oj 347.946 (± 1.7%) i/s (2.87 ms/i) - 1.749k in 5.028395s
oj strict 396.914 (± 3.0%) i/s (2.52 ms/i) - 2.014k in 5.079645s
Oj::Parser 409.311 (± 2.7%) i/s (2.44 ms/i) - 2.064k in 5.046626s
rapidjson 394.752 (± 1.5%) i/s (2.53 ms/i) - 1.998k in 5.062776s
Comparison:
json: 319.3 i/s
Oj::Parser: 409.3 i/s - 1.28x faster
oj strict: 396.9 i/s - 1.24x faster
rapidjson: 394.8 i/s - 1.24x faster
oj: 347.9 i/s - 1.09x faster
```
https://github.com/ruby/json/commit/7e0f66546a
2024-11-01 13:04:24 +09:00
Jean Boussier
081689b9e2
[ruby/json] parser.rl: extract build_string
...
https://github.com/ruby/json/commit/7e557ee291
2024-11-01 13:04:24 +09:00
Benoit Daloze
6412e6f6c3
[ruby/json] Use String#encode instead of rb_str_conv_enc()
...
* rb_str_conv_enc() returns the source string unmodified
if the conversion did not work. But we should be consistent with
the generator here and only accept BINARY or convertible to UTF-8.
https://github.com/ruby/json/commit/1344ad6f66
2024-11-01 13:04:24 +09:00
Jean Boussier
3782600f0f
[ruby/json] Emit warnings when dumping binary strings
...
Because of it's Ruby 1.8 heritage, the C extension doesn't care
much about strings encoding. We should get stricter over time.
https://github.com/ruby/json/commit/42402fc13f
2024-11-01 13:04:24 +09:00
Jean Boussier
f2b8829df0
Deprecate unsafe default options of JSON.load
...
[Feature #19528 ]
Ref: https://bugs.ruby-lang.org/issues/19528
`load` is understood as the default method for serializer kind of libraries, and
the default options of `JSON.load` has caused many security vulnerabilities over the
years.
The plan is to do like YAML/Psych, deprecate these default options and direct
users toward using `JSON.unsafe_load` so at least it's obvious it should be
used against untrusted data.
2024-11-01 13:04:24 +09:00
Jean Boussier
cc2e67a138
Elide Generator::State allocation until a to_json method has to be called
...
Fix: https://github.com/ruby/json/issues/655
For very small documents, the biggest performance gap with alternatives is
that the API impose that we allocate the `State` object. In a real world app
this doesn't make much of a difference, but when running in a micro-benchmark
this doubles the allocations, causing twice the amount of GC runs, making us
look bad.
However, unless we have to call a `to_json` method, the `State` object isn't
visible, so with some refactoring, we can elude that allocation entirely.
Instead we allocate the State internal struct on the stack, and if we need
to call a `to_json` method, we allocate the `State` and spill the struct on
the heap.
As a result, `JSON.generate` is now as fast as re-using a `State` instance,
as long as only primitives are generated.
Before:
```
== Encoding small mixed (34 bytes)
ruby 3.3.4 (2024-07-09 revision be1089c8ecbe1089c8ecbe1089c8ecbe1089c8ecbe1089c8ecbe1089c8ec
2024-11-01 13:04:24 +09:00
Jean Boussier
7daa1083c9
[ruby/json] Move State#configure back into C
...
While less nice, this open the door to eluding the State object
allocation when possible.
https://github.com/ruby/json/commit/5c0d428d4c
2024-11-01 13:04:24 +09:00
Jean Boussier
5dc3b15b3c
[ruby/json] generator.c: store pretty strings in VALUE
...
Given we expect these to almost always be null, we might as
well keep them in RString.
And even when provided, assuming we're passed frozen strings
we'll save on copying them.
This also reduce the size of the struct from 112B to 72B.
https://github.com/ruby/json/commit/6382c231b0
2024-11-01 13:04:24 +09:00
Jean Boussier
4a5e44953a
[ruby/json] Make fbuffer_inc_capa easier to inline
...
With the extra logic added for stack allocation, and especially the
memcpy, it became harder for compilers to inline.
This doesn't fully reclaim the speed lost with the stack allocation,
but it's getting closer.
Before:
```
== Encoding twitter.json (466906 bytes)
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec ) +YJIT [arm64-darwin23]
Warming up --------------------------------------
json 160.000 i/100ms
oj 225.000 i/100ms
Calculating -------------------------------------
json 1.577k (± 2.0%) i/s (634.20 μs/i) - 8.000k in 5.075561s
oj 2.264k (± 2.3%) i/s (441.79 μs/i) - 11.475k in 5.072205s
Comparison:
json: 1576.8 i/s
oj: 2263.5 i/s - 1.44x faster
== Encoding citm_catalog.json (500298 bytes)
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec ) +YJIT [arm64-darwin23]
Warming up --------------------------------------
json 101.000 i/100ms
oj 123.000 i/100ms
Calculating -------------------------------------
json 1.033k (± 2.6%) i/s (968.06 μs/i) - 5.252k in 5.087617s
oj 1.257k (± 2.2%) i/s (795.54 μs/i) - 6.396k in 5.090830s
Comparison:
json: 1033.0 i/s
oj: 1257.0 i/s - 1.22x faster
```
After:
```
== Encoding twitter.json (466906 bytes)
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec ) [arm64-darwin23]
Warming up --------------------------------------
json 213.000 i/100ms
oj 230.000 i/100ms
Calculating -------------------------------------
json 2.064k (± 3.6%) i/s (484.44 μs/i) - 10.437k in 5.063685s
oj 2.246k (± 0.7%) i/s (445.19 μs/i) - 11.270k in 5.017541s
Comparison:
json: 2064.2 i/s
oj: 2246.2 i/s - 1.09x faster
== Encoding citm_catalog.json (500298 bytes)
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec ) [arm64-darwin23]
Warming up --------------------------------------
json 133.000 i/100ms
oj 132.000 i/100ms
Calculating -------------------------------------
json 1.327k (± 1.7%) i/s (753.69 μs/i) - 6.650k in 5.013565s
oj 1.305k (± 2.2%) i/s (766.40 μs/i) - 6.600k in 5.061089s
Comparison:
json: 1326.8 i/s
oj: 1304.8 i/s - same-ish: difference falls within error
```
https://github.com/ruby/json/commit/89f816e868
2024-11-01 13:04:24 +09:00
Jean Boussier
59eebeca02
[ruby/json] Allocate the initial generator buffer on the stack
...
Ref: https://github.com/ruby/json/issues/655
Followup: https://github.com/ruby/json/issues/657
Assuming the generator might be used for fairly small documents
we can start with a reasonable buffer size of the stack, and if
we outgrow it, we can spill on the heap.
In a way this is optimizing for micro-benchmarks, but there are
valid use case for fiarly small JSON document in actual real world
scenarios, so trashing the GC less in such case make sense.
Before:
```
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec ) +YJIT [arm64-darwin23]
Warming up --------------------------------------
Oj 518.700k i/100ms
JSON reuse 483.370k i/100ms
Calculating -------------------------------------
Oj 5.722M (± 1.8%) i/s (174.76 ns/i) - 29.047M in 5.077823s
JSON reuse 5.278M (± 1.5%) i/s (189.46 ns/i) - 26.585M in 5.038172s
Comparison:
Oj: 5722283.8 i/s
JSON reuse: 5278061.7 i/s - 1.08x slower
```
After:
```
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec ) +YJIT [arm64-darwin23]
Warming up --------------------------------------
Oj 517.837k i/100ms
JSON reuse 548.871k i/100ms
Calculating -------------------------------------
Oj 5.693M (± 1.6%) i/s (175.65 ns/i) - 28.481M in 5.004056s
JSON reuse 5.855M (± 1.2%) i/s (170.80 ns/i) - 29.639M in 5.063004s
Comparison:
Oj: 5692985.6 i/s
JSON reuse: 5854857.9 i/s - 1.03x faster
```
https://github.com/ruby/json/commit/fe607f4806
2024-11-01 13:04:24 +09:00
Jean Boussier
d329896fb5
[ruby/json] Fix a memory leak in #to_json methods
...
Fix: https://github.com/ruby/json/issues/460
The various `to_json` methods must rescue exceptions
to free the buffer.
```
require 'json'
data = 10_000.times.to_a << BasicObject.new
20.times do
100.times do
begin
data.to_json
rescue NoMethodError
end
end
puts `ps -o rss= -p #{$$}`
end
```
```
20128
24992
29920
34672
39600
44336
49136
53936
58816
63616
68416
73232
78032
82896
87696
92528
97408
102208
107008
111808
```
https://github.com/ruby/json/commit/d227d225ca
2024-11-01 13:04:24 +09:00
Peter Zhu
e077be119b
[ruby/json] Remove double semicolon at end of line in parser
...
https://github.com/ruby/json/commit/f6d6ca3c17
2024-10-30 10:13:49 +09:00
Jean Boussier
f2e51146f8
[ruby/json] Remove dead cases from convert_UTF8_to_* functions
...
https://github.com/ruby/json/commit/d54063a790
2024-10-30 10:13:49 +09:00
Jean Boussier
5d176436ce
[ruby/json] Allocate the FBuffer struct on the stack
...
Ref: https://github.com/ruby/json/issues/655
The actual buffer is still on the heap, but this saves a pair
of malloc/free.
This helps a lot on micro-benchmarks
Before:
```
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec ) +YJIT [arm64-darwin23]
Warming up --------------------------------------
Oj 531.598k i/100ms
JSON reuse 417.666k i/100ms
Calculating -------------------------------------
Oj 5.735M (± 1.3%) i/s (174.35 ns/i) - 28.706M in 5.005900s
JSON reuse 4.604M (± 1.4%) i/s (217.18 ns/i) - 23.389M in 5.080779s
Comparison:
Oj: 5735475.6 i/s
JSON reuse: 4604380.3 i/s - 1.25x slower
```
After:
```
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec ) +YJIT [arm64-darwin23]
Warming up --------------------------------------
Oj 518.700k i/100ms
JSON reuse 483.370k i/100ms
Calculating -------------------------------------
Oj 5.722M (± 1.8%) i/s (174.76 ns/i) - 29.047M in 5.077823s
JSON reuse 5.278M (± 1.5%) i/s (189.46 ns/i) - 26.585M in 5.038172s
Comparison:
Oj: 5722283.8 i/s
JSON reuse: 5278061.7 i/s - 1.08x slower
```
Bench:
```ruby
require 'benchmark/ips'
require 'oj'
require 'json'
json_encoder = JSON::State.new(JSON.dump_default_options)
test_data = [1, "string", { a: 1, b: 2 }, [3, 4, 5]]
Oj.default_options = Oj.default_options.merge(mode: :compat)
Benchmark.ips do |x|
x.config(time: 5, warmup: 2)
x.report("Oj") do
Oj.dump(test_data)
end
x.report("JSON reuse") do
json_encoder.generate(test_data)
end
x.compare!(order: :baseline)
end
```
https://github.com/ruby/json/commit/72110f7992
2024-10-30 10:13:48 +09:00
Jean Boussier
2e43621806
[ruby/json] Optimize fbuffer_append_long
...
Ref: https://github.com/ruby/json/issues/655
Rather than to write the number backward, and then reverse
the buffer, we can start from the back of the buffer and write
the number in the proper direction.
Before:
```
== Encoding integers (8009 bytes)
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec ) +YJIT [arm64-darwin23]
Warming up --------------------------------------
json 8.606k i/100ms
oj 9.598k i/100ms
Calculating -------------------------------------
json 86.059k (± 0.8%) i/s (11.62 μs/i) - 430.300k in 5.000416s
oj 97.409k (± 0.6%) i/s (10.27 μs/i) - 489.498k in 5.025360s
Comparison:
json: 86058.8 i/s
oj: 97408.8 i/s - 1.13x faster
```
After:
```
== Encoding integers (8009 bytes)
ruby 3.3.4 (2024-07-09 revision https://github.com/ruby/json/commit/be1089c8ec ) +YJIT [arm64-darwin23]
Warming up --------------------------------------
json (reuse) 9.500k i/100ms
json 9.359k i/100ms
oj 9.722k i/100ms
Calculating -------------------------------------
json (reuse) 96.270k (± 0.4%) i/s (10.39 μs/i) - 484.500k in 5.032777s
json 94.800k (± 2.2%) i/s (10.55 μs/i) - 477.309k in 5.037495s
oj 97.131k (± 0.7%) i/s (10.30 μs/i) - 486.100k in 5.004822s
Comparison:
json (reuse): 96270.1 i/s
oj: 97130.5 i/s - same-ish: difference falls within error
json: 94799.9 i/s - same-ish: difference falls within error
```
https://github.com/ruby/json/commit/0655b58d14
2024-10-29 13:25:01 +09:00
Hiroshi SHIBATA
caa946f2de
Restore ext/json/extconf.rb
2024-10-26 18:44:15 +09:00
Jean Boussier
e136e552b6
[ruby/json] Instantiate Parser with a kwsplat
...
Prior to 2.7.3, `JSON::Ext::Parser` would only take kwargs.
So if json_pure 2.7.4 is loaded with `json <= 2.7.2` (or stdlib)
it blows up.
Ref: https://github.com/ruby/json/issues/650
Fix: https://github.com/ruby/json/issues/651
https://github.com/ruby/json/commit/4d9dc98817
2024-10-26 18:44:15 +09:00
Jean Boussier
8018a3121f
[ruby/json] Workaround being loaded alongside a different json_pure version
...
Fix: https://github.com/ruby/json/issues/646
Since both `json` and `json_pure` expose the same files, if the
versions don't match, the native extension may be loaded with Ruby
code that don't match and is incompatible.
By doing the `require json/ext/generator/state` from C we ensure
we're at least loading that.
But this is a dirty workaround for the 2.7.x branch, we should
find a better way to fully isolate the two gems.
https://github.com/ruby/json/commit/dfdd4acf36
2024-10-26 18:44:15 +09:00
Nobuyoshi Nakada