There is no separate math library with MSVC: the math functions are part
of the C runtime. libjxl.pc already leaves -lm out on MSVC (#4314), but
libjxl_cms.pc and libjxl_threads.pc still list it, which gives link
errors in client applications that use these files with MSVC.
Add PKGCONFIG_MATH_LIB next to PKGCONFIG_CXX_LIB and use it for both
files. This is the same change vcpkg carries for its libjxl port
(microsoft/vcpkg#53773).
Fixes#4966
Tested: with AppleClang 15 on macOS, the generated libjxl.pc,
libjxl_cms.pc and libjxl_threads.pc are byte-identical before and after
this change. A configure-only run with MSVC forced on produces no -lm in
any of the three files.
Co-authored-by: Eugene Kliuchnikov <eustas@google.com>
Netpbm only allows sample values in [0, MaxVal], but the PNM reader only
checked MaxVal itself and then copied the raster as-is. It also sets
JXL_BIT_DEPTH_FROM_CODESTREAM, whose documented input range is
0..(1 << bits_per_sample) - 1, and nothing downstream enforces that range.
So a P6 with MaxVal 4095 and a sample of 65535 was encoded with
bits_per_sample = 12 and modular_16_bit_buffer_sufficient = true, while the
codestream holds values that do not fit in 16-bit signed buffers.
libjxl's decoder uses int32 buffers and clamps on output, so djxl shows
4095. Decoders that use 16-bit buffers when the header says they are
sufficient (jxl-rs, used by Chrome) wrap samples above 32767 to negative
values, which render as black.
Check the samples in DecodeImagePNM and in ChunkedPNMDecoder::Init
(cjxl --streaming_input) and fail if any is above MaxVal. MaxVal is always
2^n - 1 here, so a sample is too large iff its most significant byte has a
bit set above MaxVal; the check ORs the raster 8 bytes at a time and tests
the bytes that hold those bits. It is skipped for MaxVal 255 and 65535,
where every value is valid, and for PFM. On a 6000x4000 12-bit PPM, cjxl
-d 0 -e 1 goes from about 0.10 s to 0.11 s.
Tested:
- New CodecTest.PNMSampleAboveMaxVal: P5/P6/P7 inputs with samples equal to
MaxVal decode; a sample above MaxVal (4/12-bit, colour and alpha, at
every position of a 19-sample row) fails. It fails without the pnm.cc
change.
- ctest (full suite).
- cjxl -d 0, -d 1 and -d 0 --streaming_input on random PGM/PPM/PAM inputs
with MaxVal 1, 15, 255, 1023, 4095, 16383 and 65535 (plus all-MaxVal
images) give byte-identical output to the unpatched cjxl; inputs with a
sample above MaxVal are now rejected.
Co-authored-by: Eugene Kliuchnikov <eustas@google.com>
JxlDecoderSkipCurrentFrame() resets image_out_buffer_set but keeps
extra_channel_output, unlike the path that finishes a frame, which clears
both. Extra channel buffers apply only to the current frame (see
JxlDecoderSetExtraChannelBuffer), so a client that skips a frame and then
requests only color output for the next frame has the decoder write that
frame's extra channel into the buffer it was given for the skipped frame.
The client may already have released that buffer, and in non-coalesced mode
the next layer can be larger than the skipped one; the bounds checks in the
write stage are debug-only, so in release builds this is a use-after-free or
heap-buffer-overflow write of decoded pixel data.
Clear the extra channel outputs together with the image output buffer, as is
done when a frame is finished.
DecodeTest.SkipCurrentFrameForgetsExtraChannelBuffers sets color and alpha
outputs for the first frame of an animation, skips it, decodes the second
frame with color output only, and checks the skipped frame's alpha buffer is
untouched. Before this change the decoder overwrote 21576 of its 21600 bytes.
Co-authored-by: Eugene Kliuchnikov <eustas@google.com>
LowMemoryRenderPipeline keeps the output of every kInOut stage in a ring
buffer indexed by row number modulo a power of two. Its size was
"2 * border_y of the next stage + rows produced per call", which is the
exact distance between the newest row a stage produces and the oldest row
its consumer still reads only if every stage runs exactly border_y rows
ahead of its consumer.
Since #4713 the virtual padding of each stage is rounded up to its row
sampling factor, so a vertically subsampled stage can run up to one extra
output row per sampling step ahead of its consumer. The ring buffers were
not resized, so a consumer can read a row that has already been
overwritten by a later one, and whatever was in that slot (for the first
rows of a group, never-written memory) reaches the output. This is what
OSS-Fuzz 501356662 and 511400706 report (MSan use-of-uninitialized-value,
regressed 2026-04-08/09); the regression shipped in v0.12.0.
Compute the ring size by replaying the scheduling conditions of RenderRect
(virtual padding, input shift, border_y, rows per call) over enough virtual
rows to pass the start-up transient and cover several periods of all
sampling factors, and keep the largest distance between the newest row
produced and the oldest row still read by any consumer. A JXL_DASSERT
checks the result is never below the old formula.
In debug builds (hence in the OSS-Fuzz builds) every ring slot now
remembers which row it holds: rows about to be written are fetched with
GetOutputBuffer, rows to be read with GetInputBuffer, rows reused from the
previous invocation are re-checked, so reading a row that was never
produced or already overwritten fails at the read instead of returning
stale memory. Release builds only change the buffer sizes.
Testing: new RenderPipelineSchedulingTest compares the low-memory and the
simple pipeline for every combination of per-channel chroma upsampling,
Gaborish and 1x/2x/4x/8x upsampling at 6x5 and 17x229 (the two OSS-Fuzz
geometries): 317 of 1024 configurations fail on main, all pass with this
change. Full ctest passes as on main in Release, in ASan, and with
JXL_IS_DEBUG_BUILD (ci.sh opt), where the new row-ownership check runs on
every decode in the suite. The djxl_fuzzer seed corpus replays without new
findings under the OSS-Fuzz ASan build. The two OSS-Fuzz testcases are
clean under the OSS-Fuzz MSan build of djxl_fuzzer and reproduce without
this change. With the stage-combination harness from #4785 (frame
dimensions made consistent with the stages, all three input channels
initialized) and every pipeline buffer pre-filled with NaN so that any read
of unwritten memory is deterministic, 820 of 2048 configurations fail on
main and none with this change. ./ci.sh lint passes.
Co-authored-by: Eugene Kliuchnikov <eustas@google.com>
* Reset JPEG reconstruction state when the decoder is rewound or reset
JxlDecoderRewind() and JxlDecoderReset() forget the image, extra channel
and box output buffers, but not the JPEG reconstruction decoder. Its output
pointer (from JxlDecoderSetJPEGBuffer), the partly read jbrd box and the
parsed JPEGData all survive into the next decode.
The API documents the JPEG buffer as used "until JxlDecoderReleaseJPEGBuffer
is called or the decoder is destroyed or reset", so a caller may free it
after a reset. When the same decoder is then reused to reconstruct another
JPEG, JxlDecoderSetJPEGBuffer() fails with "Already set JPEG buffer", and
the decoder writes the new JPEG into the freed buffer anyway. With ASan this
is a heap-use-after-free WRITE in JxlToJpegDecoder::WriteOutput(). Rewinding
while a jbrd box is only partly read has a related problem: the next pass
appends the box to the stale partial copy, and the decode fails.
Add JxlToJpegDecoder::Reset() and call it from
JxlDecoderRewindDecodingState(), next to the other per-decode output state.
Tests:
- DecodeTest.JPEGReconstructionBufferNotKeptAcrossReset reconstructs a JPEG,
resets without releasing the buffer, reconstructs again into a new buffer,
and checks the first buffer was not written. Before this change
JxlDecoderSetJPEGBuffer() returned JXL_DEC_ERROR on the second pass.
- DecodeTest.JPEGReconstructionRewindInsideBox stops a few bytes into the
jbrd box, rewinds, and checks the full decode reconstructs the original
JPEG. Before this change it returned JXL_DEC_ERROR.
* Initialize JxlToJpegDecoder through Reset()
Call Reset() from the constructor instead of repeating the initial values
as default member initializers, so the initial state is written in one
place and stays in sync with what a rewind or reset restores.
With prefix codes a single-symbol histogram costs nothing per symbol and one
with more symbols at least a bit per symbol. The fast clustering's Shannon
distance missed that and merged a mostly-zero 2-symbol histogram into large
all-zero ones (a 715 B jxl_from_tree image became 27 KB). Only applies when
force_huffman is set.
EncodeBlockCtxMap only used the 1-bit "default" code when the context map
had 21 entries, the size of the default map when there were 7 order
classes. With 13 order classes the default map has 39 entries, so the
default map was always written out in full (~22 bytes).
Tested: cjxl output with the default block context map (e.g. effort 3, or
small images at any effort) is 22-23 bytes smaller with identical decoded
pixels in djxl and jxl-rs; other files are unchanged.
jxl_test expectations for the four tests with the default map are 23-29
bytes lower.
* Allow non-444 Smoothing and fix JPEG reconstruction
* Enable JPEG DC smoothing
* Move reconstruction fix to a separate PR
* Add back transcode fix
Realised even with it being put in another PR, this one will just merge with it anyway, but now it'll run tests correctly too.
* Add a flag for JPEG LF Smoothing
* Add a flag for JPEG LF Smoothing
* Add a flag for JPEG LF Smoothing
* Add a flag for JPEG LF Smoothing
* Add a flag for JPEG LF Smoothing
* Enable 444 smoothing by default
* Enable 444 smoothing by default
* Enable 444 smoothing by default
* Enable 444 smoothing by default
PatchDictionary is reused across frames. Decode() cleared positions_ but
appended to ref_positions_ and blendings_, so in a later frame the blending
entries (indexed by position index) could still be the previous frame's:
patches were blended with a previous frame's blend modes. Clear() had the
same problem.
Tested: new PatchDictionaryTest.DecodeReplacesPreviousFrameBlendModes (fails
without the fix: the second frame's kReplace patch was blended with the first
frame's kAdd), and multi-frame files whose frames use patches with different
blend modes (made with jxl_from_tree): djxl now matches jxl-rs, which was
already correct.
* Update kRenderPipelineXOffset on ARM to respect actual maximum padding
Thanks to Apple for reporting.
Ref: AA1
* Limit "extra_zero_runs" in JPEG metadata
Since there are at most 64 coeffs in DCT block, and each "extra_zero_run" covers 16 values,
there is no valid JPEG that have more than 4 of those.
Thanks to Anthropic for discovery
Ref: AF1
* Update MaxEncodedBitsPerSample with actual values
Thanks to Anthropic for discovery.
Ref: AF4
* Deal with box_size type size mismatch on 32-bit platforms
Thanks to Ali Raza (@locus-x64) for reporting
Ref: AR1
* Fix upsampling input/output offset
Ref: MY05
* Even if EC is not used, require consistent (late) upsampling when Patches are applied
Ref: MY11
* Fix FileDirName returning empty string for bare filenames
Fixes#1872
* Drop benchmark FileDirName test
---------
Co-authored-by: Eugene Kliuchnikov <eustas@google.com>
Reminder: the grand plan is get rid of CodecIo, when possible
Drive-by: in some places we set "quality" option, that is no-op; set "q" instead (parsed by "jpg" encoder)
* warn user when GIF disposal mode is not supported
* add pseudo to Authors file
* Apply suggestion from @jonsneyers
Co-authored-by: Jon Sneyers <jon@cloudinary.com>
---------
Co-authored-by: Eugene Kliuchnikov <eustas@google.com>
Co-authored-by: Jon Sneyers <jon@cloudinary.com>
