This was missed when the hwaccel_picture_private was made a buffer.
Fixes: YMGIFa63VNkS
Fixes: AISLE-2026-0150-00681
Fixes: memleak
Memleak Replicated through UnModified FFmpeg with ASAN
Found-by: Joshua Rogers (AISLE)
Cherry-picked from https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24652 where
the commit is part of the Vulkan MJPEG hwaccel. The Fixes, replication and
Found-by lines were added when applying it for the security report; the
start_frame() call keeps passing NULL as the preceding commits of that pull
request are not included.
The decoder carries FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM, so
avformat_find_stream_info() invokes it with skip_frame = AVDISCARD_ALL
just to fill the stream parameters, yet it unpacked the whole frame
regardless and discarded it, tens of megabytes for a high-resolution
scan. Return as soon as the dimensions, pixel format and color
properties are set, before allocating the buffer, like the other
decoders with the capability.
Supports RawFormat 8, used by the S1II, S5II and GH6. Outputs the Bayer
mosaic, with the sensor borders as cropping and the black and white
levels as raw color params side data.
The as-shot white balance (tags 0x24 - 0x26) is exported as
AVRW2RawColorParams, and the TIFF and EXIF tags of IFD0 and its sub-IFDs
as EXIF side data, gathered during the single header walk and attached
without re-parsing. The orientation becomes a display matrix; Panasonic's
raw parameters, the previews and the strip pointers are left out.
The whole file is given as one packet.
Bit-exact with LibRaw on 801 S1II files, at 3x its speed per thread.
av_frame_apply_cropping() returned AVERROR_BUG when the plane offset was
more aligned than crop_left, which is always the case for single-plane
formats with more than a byte per pixel (gray16, rgb48, 16-bit Bayer,
p010). The rounding that follows already handles it.
Convert the decoder to the receive_frame API (one codestream per
packet; skipped frames pull the next packet instead of returning
empty-handed), finish threading setup before the tile decode, and do
not leave a partially initialized frame on the output on error.
Move the tag-tree size, exp2fi and band stepsize helpers into the
shared header as ff_-prefixed inlines so code outside jpeg2000.c can
derive the same geometry and quantization parameters.
XYZ formats carry no YCbCr matrix, so decode and encode them like RGB
instead of falling into the unspecified colorspace path, which applied
a BT.470BG matrix and marked the graph as incomplete.
Note that the new code does not yet convert between XYZ and RGB.
Pinning per-packet sizes of the float NMR coder's output is not portable:
two builds of the same tree emit a few different packets at -q:a 8. Use
the decoded-PCM stddev comparison the other AAC encode tests use.
-aac_rc cbr|abr (default cbr). ABR runs the nd-target quality solve with
the set-point owned by a slow rate servo, and joins VBR as a
quality-target mode in the psy model (fixed-reference PE reduction,
quality attack map, broadband mask floor).
The servo integrates the log rate error but applies it to the nd
set-point in rare, discrete, calm-gated steps. The set-point must be
quasi-static: any drift on content timescales moves lambda, and with it
band/PNS/scalefactor state, tracking sections in antiphase and
coarsening exactly what needs bits - measured 2x worse than a fixed
target at equal mean rate. Locally this mode IS fixed-target VBR; rate
honesty converges on the minutes scale.
Bootstrap. The set-point is seeded from the VBR calibration anchor (one
constant, NMR_VBR_ANCHOR: the two are required to agree, and moving only
one copy made ABR overshoot its rate target by up to 9%). After ~2s of
fast rate measurement off a representative window (an all-transient
head reads over-target and would coarsen the very content that needs
bits), an open-loop jump over the measured loop gain (~ -0.24 log2 rate
per target unit) corrects the seed's per-content error, applied at
1.2x - the estimator's own inverse warm-up factor, since the EMA is
seeded on target (a cold start must not read as a fake deficit) and so
recovers only ~82% of the error. One correction is not enough whatever
the gain estimate, because the 0.24 loop gain is only nominal: the boot
re-arms while the rate is still off, up to a bounded number of times,
re-measuring each time, and re-seeds the estimator on target rather than
to zero (a zero estimate pins the integrator at its clip via log2f(0)
and reads as a 100% deficit to the fill in the same frame). Its settle
window sits below the stepper's hold, or the stepper resets the shared
counter first and the re-arm never fires.
Every set-point correction is glided in at ~0.5 log2-units/s rather than
stepped: a one-frame noise-floor jump at a fixed stream time (the boot
lands at ~2s in every stream) is audible against revealing content even
at step sizes the corpus cannot see.
Bits-fill. Under the long-run target, an easy frame is topped up toward
the ask by moving lambda finer only - the quality target is a floor,
never traded away. It spends the rate the user asked for on sub-mask
margin the nd statistic cannot see (it clips at -4/band); without it,
stat-transparent content caps the rate below any ask. The fill runs from
frame one and fades continuously as the rate EMA converges (a boot-gated
fill flipped regimes at ~2s, an audible quality step), follows the psy
PE demand shape, and is slew-bounded, writing the lambda continuity
state from long frames only instead of free-diving up to 64x per frame.
The un-booted stream head behaves CBR-like until the servo takes over
(love.flac: >6k energy had swung 0.61-1.83x vs the source at frame rate
while the head ran 25% under the ask).
The reachable-range clamps do not track the calibration anchor: their
floors are set by the stat's -4/band sub-mask clip. A floor at -1.65
railed the servo on easy content (male_speech at a 128k ask pinned at
the clamp with a 10% standing deficit); -3.0 keeps 384k asks reachable
without the stall. Easy content can still fall short of very high asks
(main_theme: 196k at a 256k ask) - there is no padding. The coarse
clamp is a quality guard: set-points past it buy little rate for a lot
of quality (2.5 -> 6.0 at a 48k ask: -5% rate, +32% Zimtohrli), so asks
below about 56 kbps stereo land above the ask (24k/32k asks: ~40k).
The documentation points those rates at CBR.
Rate accuracy against the -b:a ask:
corpus bias mean|err| worst
IgorC-3 -1.3% 2.2% -11.8%
IgorC-1 +1.1% 2.0% +5.0%
32 random -0.3% 2.2% -10.0%
A 4.5-minute concatenation of the IgorC-3 samples lands at 128.2 kbps
on a 128k ask. castanets/velvet ABR beat CBR; ABR corpus 0.00060 at
126k.
-aac_rc is documented, and fate-aac-nmr-abr-encode pins the per-packet
sizes through the servo's boot.
-q:a with the NMR coder becomes a quality-domain VBR: bisect the pooled
lambda until the mean log2 dist/real-mask over coded bands (floored at
-4/band, so a flat sf-chain over-coding quiet bands cannot let a coarser
lambda read as a finer statistic) meets a target. The target is anchored
so -q:a 1 lands near 128 kbps stereo; higher q is finer, as with the
other coders, at -1.2 per q doubling (about a third more rate), clamped
at a reachable floor that q above about 6 reaches: the stat clips at
-4/band, targets below the clamp are asymptotically unreachable, the
bisect rails at the finest lambda and the frame exceeds the decoder
buffer. The anchor is calibrated with the broadband quality-mode mask
floor from earlier in the series; that floor measured -8.8% / -5.0% /
-0.9% Zimtohrli (ViSQOL +0.013 / +0.018 / +0.004) at matched rate on
three corpora, worst sample -34%, per-sample quality spread 2.41 ->
1.74.
The psy model runs its PE reduction at a fixed reference quality for
this coder (psy.unbounded_pe): thresholds keep their perceptual shaping
but stay independent of -q:a, which otherwise moves thr_real under the
coder's own statistic and inverts the quality ladder.
Frame classes the statistic mishandles:
- Short frames: the grouped-band statistic is inflated, so they bisect
against an offset target and never write solver state. A fixed +3
offset was a content-class policy in disguise - broadband beats
(velvet) need it, LF/mid tonal decays audibly do not and paid 1.7-2.8x
the CBR error through every post-hit decay. The offset is measured
per frame instead: evaluate the grouped stat at the long-anchor lambda
(continuity says true quality is on target there, the excess is this
content's inflation), weighted by the frame's HF energy share - the
stat reads inflated on both classes, audibility does not. Post-hit
decays 1.67x -> 1.13x CBR error (worst 2.82x -> 1.44x).
- Continuous shorts: the offset-bisect assumes isolated transients
between long anchors; on dense onsets the per-frame stat is bisect
noise and lambda careened across decades (0.16 -> 82 -> 2 -> 43), an
audible whoosh. Shorts hold to the CBR run slew, clamped after the
last solve (PASS 2 re-bisects on a fresh +-32x bracket and otherwise
rails at exactly lam*32); dives toward finer stay free.
- Operating points: only bisected long frames whose stat actually
landed within 0.5 of the target write slew/warm-start state. Cheap
frames (room tone, gaps) rail fine at no cost; the first frames of a
quiet channel rail coarse from no warm state - writing either builds
a slew prison the channel climbs out of one rung per frame (a muffled
stream head on fade-ins).
- Decoupled solo solves keep per-channel slew state: a shared lam_slew
clamped each channel against the OTHER channel's operating lambda and
landed the damage asymmetrically.
The window history the slew rules read (previous-frame short, frames
since the last short run) used to advance only inside the CBR reservoir
bookkeeping; it now advances once per frame in every rate-control mode,
so the run slew and the pooled short-TNS hysteresis see real history
here too. Measured neutral (IgorC-1 VBR q=1: -0.3% Zimtohrli).
Legality is absolute: a frame may never exceed the decoder buffer.
- The cap is the format limit regardless of -q:a. It used to scale with
the outer lambda, i.e. with -q:a itself, so the finest settings had
the smallest cap (531 bits/channel at q=0.1 under the old direction)
and lost bands where they wanted the most bits.
- Dense short frames at high rates can exceed 6144 bits/channel even at
the coarsest candidate grid (the 1e4 rail still emitted 11-13k
spectral bits); the legality path sheds the highest bands across all
window groups (act[] runs group by group) until the frame fits,
keeping the scalefactor-chain anchor.
- The bits a solve reports for the next frame's side-information
estimate count only the surviving bands, and PNS filters the active
list instead of rebuilding it from all bands. Counting shed bands
drove the estimate negative (one frame emitting 1256 bits counted
13825), and a negative estimate enlarged the next frame's cap: 13056
payload bits against a 12288 limit. A negative estimate can no longer
enlarge the cap.
- The outer loop no longer skips the size check under a constant
quality: an oversized frame is retried at a lower lambda, which
shrinks the coders' caps. This applies to every coder, and only to
frames that would otherwise be illegal.
A fine-quality encode test (fate-aac-nmr-vbr-encode) pins the per-packet
sizes; its largest frame is 11560 payload bits.
Encoder side: the bandwidth ladder takes the expected per-channel rate
from the quality ladder (q=1 ~ 64.5 kbps/ch, x1.27 per doubling), and
Qavg reports the coder's quality-mode slew state: the lambda of the last
long operating-point solve. The encoder documentation describes the
NMR meaning of -q:a.
CBR output is unchanged.
M/S balance gate. M/S has no coding gain on a hard-panned band
(|S| ~ |M|) and its quantization noise decorrelates across the unfold,
smearing the panned source into the far channel; require channel
balance within ~6dB for M/S candidacy.
Perceptual intensity stereo, on by default. Zimtohrli penalizes I/S
reconstruction even when the image error sits below the masks, so the
metric cannot govern it; ears above ~8kHz do not hear interaural fine
structure, and they can. Engage I/S on perceptual criteria alone: above
8kHz, inter-channel correlation over 0.85 (sticky), long windows only,
with the image-error budget opened for qualifying bands regardless of
rate pressure. The gates select content like a scalpel - Angels 9.3% of
bands, Enola 6.7%, trumpet 3.0% - and exactly 0% on decorrelated-HF
material (velvet, Berling) where I/S would damage the image. Corpus cost
is metric noise (+0.6% Zim at 128k, +1.4% at 64k - Zimtohrli penalizes
I/S reconstruction the ear does not hear); freed bits improve the
sub-8kHz spectrum 10% on engagement-heavy content. -aac_is 0 disables
I/S altogether.
Measured and not included: lifting the both-channels-noise-like
requirement for stereo PNS (engagement moves less than 0.2% even at
15-18% band usage, corpus flat, and noise in one ear against coded
texture in the other is an imaging risk), and M/S threshold, BMLD side
reduction, M/S equivalence and I/S floor/surgical-candidacy sweeps.
The id3v2 re-encode refs follow the default NMR output; the fast-coder
encode tests are unaffected.
Three fixes to the CBR corridor's handling of the reservoir.
Bound the coarse-pass cap re-solve. When the coarse pass's trellis bits
exceeded the reservoir cap, its re-solve bisected lambda in [lam, 1e4]
on the coarse candidate grid, whose steppy bit curve can demand a jump
of orders of magnitude - a one-frame excursion that potato-codes the
burst recovery frame: on bass-heavy content the frame after an isolated
transient came out 20-40dB coarser below 500Hz, an audible digital
scratch (ENC_TEST_SAMPLE_05 at 2.1s and 2.8s, with every coding tool
off; err 0.0038 -> 0.0004). Coarsen at most 3x past the corridor there.
The fine pass then works from the bounded lambda, and the reservoir cap
is still enforced on the fine grid after it, where the fit only moves
lambda as far as the frame needs (1.03-1.2x on the test clips; larger
moves happen only at a stream head whose corridor bootstrapped near
lossless, where the cap is what keeps the head at the requested rate).
Corpus neutral at 128k, -2% at 64k, transient panel unchanged.
Escalate it under sustained overage. The debt ledger clips at -rc_bmax:
once saturated, further overage was silently forgiven, and sustained
transient content rode hot forever (castanets 143 kbps on a 128k CBR ask
- its opening roll spent 215 kbps for a full second; tightening the
bound made it worse, since less fit means more forgiven overage). A
one-frame hard fit fixes the rate but re-creates the potato-frame click
the bound exists to prevent (You look good 64k click +53%). The
enforcement now escalates with consecutive saturated frames, counted
once per frame across element groups (any frame without a saturated
overage ends the run): each one widens the allowed coarsening by another
3x step, so a chronic roll converges within a few frames while isolated
bursts never see more than one escalation. Castanets 142.9 -> 134.1
kbps, You-look click at baseline, the E5 recovery-frame scratch stays
fixed, corpus unchanged at both rates.
Bucket-full spend floor. Bits saved beyond the reservoir's remaining
headroom are simply lost (the bucket is capped), so with a full bucket
the plan must never sit below nominal minus what can still be banked.
Without this, a quiet intro poisons the corridor centre high and the
loud entrance rate-limits through it for over a second - listening
feedback flagged a muffled first second on a fade-in sample while our
rate crawled from 47 to 128 kbps over 0.9s where reference encoders
enter flat. Applied in the corridor and in the pre-bootstrap phase, so a
fade-in bootstraps lam_rc at a spending operating point; finer-only,
true silence still codes nothing. Fade-in head now enters at full rate
from the first content frame; corpus standings and the E5 artifact
fixes unchanged.
The id3v2 re-encode refs follow the default NMR output.
Allocation and band-selection changes to the NMR coder.
Allocation law. The law moves from the Apple-derived (0.443, 0.111) to
(0.50, 0.18), swept on the repaired measurement harness: the old
constants describe Apple's bit distribution, not this trellis's optimum,
and the harness defect that anchored them charged full-scale integer
wraps on hot masters to the louder encoder. Corpus Zimtohrli -38% /
ViSQOL +0.06 at 128k stereo, -15% / +0.05 at 64k, all listening guards
flat or better.
HF precision taper. Bitstream comparison at matched rate shows Apple
coding 7.5-12.6kHz about 4 scalefactor steps coarser and 12.6kHz+ about
19 steps coarser (with 22% of those bands zeroed) than a flat allocation
gives, funding their LF/mid SNR edge; the ear tolerates coarse HF
texture far better than mid-band noise. A quadratic threshold rise above
the knee (0.08 dB * ((f - 8kHz)/1kHz)^2) reproduces the rolloff: -5.4%
corpus Zim at 128k stereo for -0.014 ViSQOL, -2.3% at 64k, -1.2% at 96k.
A spread-coupled variant measured flat and is not included. Bands the
taper pushes more than 2dB past their real mask are kept out of the
nd_ema statistic: their deficit is by design, and on HF-rich content it
would otherwise ramp the pressure state into noise-class rate control on
clean tonal signals.
Group-window integrity. A coded group-band no longer mutes individual
borderline windows: they share the group scalefactor (near-free to
keep) and a 3-6ms in-band mute right before an attack reads as a gated
crunch on every beat.
PNS. Hole-fill fires at any comfortable lambda: noise-like bands whose
chosen rendition leaves most energy uncoded turn to sparse spectral
lines - structure damage the NMR objective cannot see (measured:
13-16kHz flatness 0.45 in source, 0.07 coded, 0.47 with noise). The
debounce closes two churn doors: near-masked entry now needs two
consecutive wants and the loud-band guard exits only after two loud
frames; bands no longer flip coded/noise at frame rate.
Not included: skipping the short-window threshold flatten on
LF-dominated windows. Such a window's HF bins hold sidelobe leakage of
the LF (phase-locked, acoustically silent), and the flatten hands them
the LF-sized window-mean threshold, so the residue unmasks as an HF
smear trailing slow LF attacks (SombreDetune 1.7s: 21x the reference
HF). The skip takes it to 5.7x and improves the 128k transient panel,
but the flatten is real anti-starvation protection at 64k and under
pressure, and no gate tried separates the two classes cleanly.
Encoder side:
- The sectioning trellis receives the coder's real operating lambda
rather than the static outer-loop 120 (measured neutral, but wrong on
principle).
- Exact levels: the clip-avoidance pre-attenuation scales every
near-full-scale frame by up to 0.95, a permanent -0.45dB level error
on hot masters (dense tonal content: -0.46dB mean partial level error,
Apple -0.03dB). Float decoders never clip and int16 playback saturates
the rare overshoot gracefully: -5.1% corpus Zim at 128k stereo.
Scoped to the NMR coder.
- The bandwidth ladder was tuned by metric sweeps into Zimtohrli's blind
spot: it rewards HF presence and cannot hear HF graininess, so sweeps
always push wide - while coarse HF at these rates reads as
beat-synchronous crunch. Ear-tuned down to 15/16/19.5 kHz at 48/64/96
kbps per channel (FDK sits at 14k, Apple at ~16k for 64k/ch).
Measured and not included: an LF precision tilt in either direction, a
tonality term in the law, power-mean pooling of group thresholds (the
current sum is p=1; corpus-flat at every exponent) and a START-frame HF
tilt.
The id3v2 re-encode refs follow the default NMR output.
PE demand shaping. The 3GPP bit-demand curve modulates per-frame budgets
far too gently for the trellis coder: amplifying the deviation from
unity by 2.5x is worth -2.5% Zimtohrli at 128k stereo and -13.5% at 64k
(ViSQOL up at both), with a sharp cliff beyond ~2.8 as budgets start
swinging into the clamps. Listening guards flat, pulse-train sample
stays click-free.
The amplified factor goes negative in high-PE frames on a starved
reservoir - 15-30% of CBR channel-frames. The desired PE is then zero or
below, and the reduction raises every threshold as far as the min-SNR
and hole rules allow: those frames are shaped for constant SNR rather
than by the mask. This is load-bearing, not an accident: flooring the
returned demand at the documented frame_bits/8 costs +5.2% / +3.3%
Zimtohrli at 64k on the two ABC/HR corpora (mixed at 128k).
The reservoir-fill correction of the desired PE is clamped
symmetrically in the log domain, [1/1.15, 1.15] instead of
[0.85, 1.15]: a negative previous demand sits on the lower bound, and
the asymmetric 0.85 there costs 2.4% Zimtohrli at 64k.
Signal-relative mask floor. 3GPP 5.6.1.3.3 exempts bands that are quiet
against the spread energy from the min-SNR floor ("holes allowed
here"), and nothing downstream bounds them: measured at 128k, the
published mask ends up only 1.5-2 dB below the band's own energy across
the whole 6-16 kHz range, on every sample tested. The model is asserting
the top of the spectrum is inaudible, and every consumer believes it -
the allocator buys those bands nothing, and a quality-target solver sees
a mask it cannot fail.
A band we still choose to code may now not have its mask come within
PSY_THRFL_* dB of its own energy. The two rate-control families want
different shapes, and measurably so, because the mask does a different
job in each: in quality-target modes it IS the rate authority, so the
floor is broadband (10 dB); in CBR the budget is fixed and the mask only
ranks bands against each other, so a broadband floor merely shuffles
bits and costs ~2% Zim on random content, while restricting it to above
8 kHz (6 dB) keeps the win. Here the quality-target branch covers -q:a;
the ABR mode later in the series joins it.
CBR at matched rate, Zimtohrli / ViSQOL over the two ABC/HR corpora and
two random-clip corpora: -14% / +0.017, 0% / +0.008, +1.4% / +0.003,
0% / +0.004. ViSQOL improves on every corpus. The +1.4% Zim is on the 16
canonical clips only; a fresh 32-clip random corpus is flat there, and
the ungated floor costs +1.8% on that same set, so the frequency gate is
what makes CBR safe.
Threshold-shape experiments from the harness sweeps - spreading slopes,
the min-SNR clamp, the tonality-dependent TMN/NMT offsets and the TNS
gain bars - all measured at or worse than the flat 3GPP values,
confirming the earlier static-threshold verdict, and are not included.
The fast-coder encode tests move with the thresholds and are re-pinned:
ln-encode-128k 622 -> 618, is-encode 500 -> 503, ms-encode 550 -> 563.
The id3v2 re-encode refs follow the default NMR output.
Three changes to the LAME-style window switch and short-window grouping.
Gap-onset detection. Two attack classes were invisible to the
fs/4-highpass ratio test: stop-gap slams with no content above the
filter (or a rise too gradual for the 2-sub-block comparison), and both
classes of pulse-train repeat. Detect gap-onsets on broadband and HP
sub-block peak envelopes against a decaying program-level reference:
fire when a sustained quiet stretch ends adjacent to a candidate
comparable to the recent program level. Pulse trains cannot fire it -
their inter-pulse floor never drops far enough below their own running
peak.
HF-novelty veto. A bass onset under sustained playing trips the attack
detector, but a long window codes it better: pre-echo is masked by the
bass itself, while the short excursion is the audible event - the
shorts frame mutes the HF floor, then the stop frame codes the attack's
HF spray into 21ms of silent-HF background at SNR~0 (ENC_TEST_SAMPLE_05
~2.07s, 2x the reference HF energy, rate-independent; Apple stays long
through the same passage). Candidates are gated on a first-difference
sub-block peak envelope, and the veto fires only when all three agree:
- absolutely small (2000): no HF content worth protecting;
- relatively unremarkable (4x the recent derivative envelope):
a quiet transient rising out of near-silence has a small derivative
but maximal novelty, and its long-window pre-echo lands on silence
('rustle of a falling leaf', fatboy 4.35s: 3x the un-vetoed error);
- pre-echo is masked: the minimum envelope over the ~5ms before the
attack is no more than 8x below the candidate. Stop-gap
slams (kick+chug into a beat gap) pass both other bars, but the dip
right before them means the smear lands on quiet.
Gap-detector attacks fire downstream and are unaffected. E5 2.07s
scratch gone (HF ratio 5.13 -> 0.64, Apple 0.63); castanets -11%,
trumpet -17% Zim (their low-derivative spurious shorts were hurting);
velvet/applaud/Linchpin and both corpus rates neutral; artifacts.flac
click-free.
Level-homogeneous grouping. The fixed attack-position grouping patterns
force windows of very different level into one group, where they share
scalefactors and the group's error placement smears across the level
step - the mechanism behind the castanets/velvet transient deficit.
Group instead by level runs: a new group starts where adjacent lookahead
windows differ by more than 2.5x in peak level, with group
length capped at 4; the pair path merges the two channels'
level maps so the grouping (and with it common_window) stays shared.
Castanets -15% Zim, velvet -9%, Linchpin -5.5%, 64k corpus -4.5%, 128k
corpus neutral; applause pays +3% for the fragmentation. Forcing extra
boundaries at detected attacks reversed the wins and is not done -
fragmentation, not boundary placement, is the enemy. A looser 2.0x/cap-8
rule buys castanets and velvet another 20% but costs the corpus at both
rates.
Rejected along the way: capping the group that starts at an attack (the
FDK 2-window discipline) and isolating the pre-attack window close the
pre-attack window gap to FDK on the per-window attack-profile
instrument, but the overhead costs fatboy/velvet corpus Zim.
The window decisions change for every coder: tns-encode re-pinned
637 -> 619. The id3v2 re-encode refs follow the default NMR output.
Formats with a _420 or _422 suffix require even dimensions, but frames
contexts of subsampled formats with odd dimensions still got a single
multiplane image, whose chroma planes the drivers rounded down: 320x240
instead of 321x241 at 641x481. FFV1 decoding through the Vulkan hwaccel
corrupted the chroma of such frames, and ffv1_vulkan encoding was not
lossless, on both RADV and ANV.
Use one image per plane for them instead. Hardware decoders are not
affected, as they set the image format and align the frames context
dimensions themselves. Decoding broke when 713e3c4f91 stopped aligning
the frames of decoders without a video profile.
For formats whose main and fallback Vulkan formats are the same (planar
RGB and YUVA), feats_secondary was never set. With multiplane disabled,
vkfmt_from_pixfmt2() returns through the fallback branch, which derives
the supported usage from it, so frames contexts got usage flags from
leftover data. With disable_multiplane=1, gbrp frames on an RX 6900 XT
lost VK_IMAGE_USAGE_TRANSFER_DST_BIT.
The product of the run length and the current multiplier fits in 32
bits for any valid stream, so avoid the 64-bit multiplication.
Also, the encoder uses 32-bit bits for this step.
Range coded slices were decoded and encoded by a subgroup of exactly 32
invocations, one for each of the states of a context, and devices which
could not provide one were refused. The number of invocations now
follows the subgroup size: 32 if the subgroup size can be set to it,
and the largest subgroup size below it otherwise. Each invocation holds
32 divided by that number of the states of a context and of the bytes
of the decoder's window, and lines are coded in chunks of as many
samples. The encoder keeps its queue of 64 output bytes, and computes
the slice CRC over as many segments. Devices whose subgroups are all
wider than 32 invocations run the 32 invocations as part of one, and
are no longer refused either. The ballot quantizers need 32
invocations, and are only used with them.
With 32 invocations, the instructions executed per sample and the speed
of both the decoder and the encoder are unchanged. Decoding and encoding
were checked with 1 to 16 invocations on an RX 6900 XT, running the
workgroups as partial subgroups, and decoding with the subgroups of 8
invocations of lavapipe.
Dispatch the slices in descending order of their size in the previous
frame, as the decoder does with the actual sizes. Several waves share a
SIMD and the issue arbiter favours the oldest, so the slices that take
the longest get the priority, and the frame no longer waits on a heavy
slice that was started late.
The order is written by the host into a small buffer read by the encode
shader, which now keys the RGB line cache rows by the slice row rather
than the workgroup row. The first frame is encoded in raster order.
The output is unchanged. Encoding a 6464x4852 16-bit RGB frame with
1024 slices on an RX 6900 XT, with the frame in VRAM, goes from
44.2/40.9 ms to 40.8/37.0 ms with context model 1/0. It was
202.5/178.5 ms before this series.
The CRC of a range coded slice was computed a byte at a time by a
single invocation once the slice was coded.
All invocations now compute it: each takes a segment of the slice, and
the CRCs of the segments are combined, from the first, with a GF(2)
matrix that advances a CRC past a segment of zero bytes, whose columns
are computed alongside. The table is read from shared memory.
The output is unchanged. Encoding a 6464x4852 16-bit RGB frame with
1024 slices on an RX 6900 XT, with the frame in VRAM, goes from
52.9/50.1 ms to 44.2/40.9 ms with context model 1/0.
The zero flag, the ones of the unary prefix and its terminator can only
shrink the range, so they are now coded without renormalisation checks,
and the range is checked once after the terminator. When it underflowed
on the way, they are coded again from the saved coder state, with the
checks, which happens for about a quarter of the symbols.
The output is unchanged. Encoding a 6464x4852 16-bit RGB frame with
1024 slices on an RX 6900 XT, with the frame in VRAM, goes from
53.2/51.2 ms to 52.9/50.1 ms with context model 1/0.
Every invocation computed the context, the prediction and the residual
of each sample in turn, from neighbours kept in the line cache and
loaded from the image, and the states of a sample were only loaded
once it was reached.
The invocations now compute them for 32 samples at once, one sample
each, together with the masks of the states each sample will use, and
the image loads of the next 32 samples are issued before the current
ones are coded. The states of the next sample are loaded while the
current one is coded, and forwarded when both samples use the same
context. The byte queue is checked in the loop condition, the extended
lookup becomes a specialisation constant, as the encoder uses a single
quant table, and RGB rows are preloaded four samples per invocation at
a time.
The output is unchanged. Encoding a 6464x4852 16-bit RGB frame with
1024 slices on an RX 6900 XT, with the frame in VRAM, goes from
126.6/101.6 ms to 53.2/51.2 ms with context model 1/0.
Large slices gain the most: with 24 slices, the same frame now takes
1.5 s to encode, where the previous code ran into the GPU's job
timeout and lost the device.
A symbol was coded by loops over its unary prefix and its mantissa,
reading every state from its invocation just before its decision and
updating it right after.
Each exponent now has its own straight-line code, reached through a
small branch tree, which reads all the states the symbol uses from
their invocations up front, so the chain of decisions never waits on
them. Once the symbol is coded, all invocations adapt their state at
once, from masks of the states the symbol used and of the decisions
that were ones. Escaped symbols adapt the two states they repeat in
scalar registers. The renormalisation check stays a branch instead of
being turned into selects.
The output is unchanged. Encoding a 6464x4852 16-bit RGB frame with
1024 slices on an RX 6900 XT, with the frame in VRAM, goes from
170.1/144.8 ms to 126.6/101.6 ms with context model 1/0.
A renormalisation of the range coder now only records the low end of
the interval before the shift, in a queue of up to 64 entries held by
the invocations of the subgroup. The outstanding byte logic runs over
the queue once more than 42 entries are pending, outside the symbol
coding, and writes the bytes of the whole batch at once, one per
invocation, when none of them is deferred and no carry is pending.
Escaped symbols flush the queue while coding their long runs.
The output is unchanged. Encoding a 6464x4852 16-bit RGB frame with
1024 slices on an RX 6900 XT, with the frame in VRAM, goes from
179.0/154.9 ms to 170.1/144.8 ms with context model 1/0.
A range coded slice is encoded by a workgroup of 32 invocations, but
only the first one ran the range coder, on a coder state and context
states kept in shared memory, with two barriers per sample.
The slice is now encoded by a single subgroup. Each invocation holds
one of the 32 states of the current context, and the range coder runs
on uniform values, in a new encoder section of rangecoder_subgroup.glsl.
A decision reads its state from the invocation that holds it and
updates it there, so no barriers are needed. The coder keeps the top of
its interval instead of the bottom, which coding a one leaves unchanged.
The line cache of the range coded encoder is kept per invocation, as in
the decoder, and subgroups of 32 invocations are required.
The output is unchanged. Encoding a 6464x4852 16-bit RGB frame with
1024 slices on an RX 6900 XT, with the frame in VRAM, goes from
202.5/178.5 ms to 179.0/154.9 ms with context model 1/0.
The context base of each chunk of 32 samples is built from inputs 1, 2
and 4 of the quant table, which every invocation read from the uniform
buffer. When the slice starts, those inputs of the tables of the first
plane and of the chroma planes are now copied to shared memory as
int16, and the chunks read them from there. At 3 KiB, the copy leaves
enough shared memory for every slice of a frame to stay resident. Any
other plane reads the uniform buffer as before.
Decoding a 6464x4852 16-bit RGB frame with 1024 slices on an RX 6900
XT, with the bitstream in VRAM, goes from 50.7/40.6/39.1 ms to
50.3/40.1/38.7 ms with context model 1/0/2. It was 177.1/146.6/142.9 ms
before this series.
Several waves share a SIMD and the issue arbiter favours the oldest, so
the slices that take the longest get the priority instead of the luck
of raster order, and the frame no longer waits on a heavy slice that
was started late. The host sorts the slices by their size and the
shader reads its slice index from that order.
Decoding a 6464x4852 16-bit RGB frame with 1024 slices on an RX 6900
XT, with the bitstream in VRAM, goes from 53.1/43.8/43.2 ms to
50.7/40.6/39.1 ms with context model 1/0/2.
Whether the quant table has an extended lookup is known when the slice
starts, so the line decoder is instantiated once for each case instead
of testing it per sample.
Decoding a 6464x4852 16-bit RGB frame with 1024 slices on an RX 6900
XT, with the bitstream in VRAM, goes from 53.5/45.2/44.7 ms to
53.1/43.8/43.2 ms with context model 1/0/2.
The context of the next sample, and with it the load of its states, are
issued as soon as the value is known, before the states of the current
sample are adapted and written back, and the states are forwarded when
both samples use the same context. The renormalisation left over by the
sign decision moves behind the load.
The parts of the context and of the predictor that only depend on the
row above are computed 32 samples at a time, one per invocation, with
the row of the next chunk loaded during the current one, and decoded
samples are written once per chunk. The median predictor becomes a
clamp of the sample above between bounds computed per chunk.
Decoding a 6464x4852 16-bit RGB frame with 1024 slices on an RX 6900
XT, with the bitstream in VRAM, goes from 99.1/72.4/69.8 ms to
53.5/45.2/44.7 ms with context model 1/0/2.
While the unary prefix of a symbol decodes ones, the distance
range - low - 1 does not change, and the range of each level is the
range of the level before scaled by its state. The ranges of all ten
levels can therefore be computed before any of them is decided. They
only shrink, so the levels whose range stays above
max(range - low - 1, 0xff), those that decode a one without a
renormalisation, form a prefix.
The first five ranges are computed back to back, and a search that
starts at the fifth level finds the end of the prefix with two or
three compares instead of a branch per level. The upper five ranges
are only computed when the prefix reaches past the fifth level. The
zero flag is folded into the first level: its subtraction from the
distance wraps when the flag is set, so the same compares reject that
case. A level that needs a renormalisation refills and restarts the
search with the levels already decided neutralised.
Each exit decodes the mantissa and the sign in straight-line code
specialised for its exponent, so the masks of the states a symbol used
are constant expressions of it. The sign decision leaves its
renormalisation to the caller.
Decoding a 6464x4852 16-bit RGB frame with 1024 slices on an RX 6900
XT, with the bitstream in VRAM, goes from 126.6/106.2/104.7 ms to
99.1/72.4/69.8 ms with context model 1/0/2.
Each slice is still decoded by one workgroup of 32 invocations holding
one byte of the current context's states each, but it now runs as a
single subgroup: the range decoder runs on uniform values, in the new
rangecoder_subgroup.glsl, the states of a symbol are read with subgroup
broadcasts, and the states it used are adapted by every invocation at
once from two masks, instead of going through shared memory and a
barrier on each side of the symbol.
The bitstream is read through a 32-byte window held by the subgroup,
with the next byte prefetched. The decoder tracks range - low - 1, so
that a decision is one compare and two selects, and the states are
pre-shifted so that each decision is a single 32-bit high multiply.
Escapes adapt their states in scalar registers, and the prefix is capped
at 32 ones as in the software decoder, which the previous code did not
do.
Range coded slices now require subgroups of 32 invocations.
Decoding a 6464x4852 16-bit RGB frame with 1024 slices on an RX 6900
XT, with the bitstream in VRAM, goes from 168.9/144.1/140.2 ms to
126.6/106.2/104.7 ms with context model 1/0/2.
The two context inputs that depend on the sample to the left are the
only ones on the dependency chain between decoding a sample and loading
the states of the next one. When a quant table is a staircase over the
8-bit difference, with at most 32 steps of the same size, its value is
the number of thresholds the difference passes: one threshold per
invocation, a comparison, a ballot and a popcount, with no memory access.
The host checks the tables and stores the thresholds next to them in the
constants buffer, and the decoder requests a subgroup of exactly
CONTEXT_SIZE invocations when it uses them.
get_pred is split into the part that depends on the row above and the
part that depends on the sample to the left.
Decoding a 6464x4852 16-bit RGB frame with 1024 slices on an RX 6900
XT, with the bitstream in VRAM, goes from 171.6/143.7/139.6 ms to
168.9/144.1/140.2 ms with context model 1/0/2: only context model 1
has an extended lookup, and the current decoder is bound by shared
memory and barriers. The rewrite of the range coded decoder relies on
it.
The tables were uploaded as int16, so every lookup in the shaders was a
dword load followed by an extract and a sign extension. Converting them
on upload makes each lookup a single load. The buffer is shared by the
encoder and the decoder, which now bind exactly the range the tables
occupy.
Decoding a 6464x4852 16-bit RGB frame with 1024 slices on an RX 6900
XT, with the bitstream in VRAM, goes from 177.1/146.6/142.9 ms to
171.6/143.7/139.6 ms with context model 1/0/2.
Add a third context model, selected with -context 2, using only the two
nearest gradients coarsely quantized, for 13 contexts per plane.
Every slice starts from fresh probability states, and with many slices
there is too little data per slice for the larger context sets to adapt.
On a 6464x4852 16-bit RGB film scan coded with 1024 slices, this codes
87.99 MB per frame, against 90.29 MB with -context 0 and 100.79 MB with
-context 1, while needing 832 bytes of probability state per slice
instead of 23 KB and 324 KB. The same holds on 10-bit and 8-bit content
at high slice counts; with few slices, -context 0 remains best.
The table is stored in the extradata like the existing ones, so decoders
need no changes.
816d74e0bb made frames without psy analysis at EOF silent. The frame
that follows the last input frame to flush its MDCT overlap is always
such a frame, so the last CELT_OVERLAP samples of the stream were lost
whenever it was needed, i.e. when the length of the stream is a
multiple of frame_size or less than CELT_OVERLAP samples short of one.
It was also coded as a one byte packet, which the MPEG-TS muxer
rejects.
Encode it like any other frame instead. Its psy analysis data is
zeroed, which leaves it with the default frame parameters.
Fixes: #24196
Silent frames were coded with zero bytes, which decoders take as a lost
frame and conceal by extrapolating the preceding audio, rather than
outputting silence. A packet of such frames is also only one or two
bytes long, and the MPEG-TS muxer rejects packets shorter than two.
Code the silence flag in a two byte frame like libopus does, and keep
the encoder state in sync with the decoder after it: the decoder
predicts the band energies of the next frame from CELT_ENERGY_SILENCE,
and the pre-emphasis filter state at the start of the next overlap is
zero. As silent frames no longer have a size of zero, leave them out of
the rate control explicitly.
The trailing CELT_OVERLAP samples of a frame are filtered again as the
overlap of the next frame, so the filter state saved for the next frame
has to be the one from right before them. The code relied on packets
being assembled from 120 sample input frames for this, which is no
longer the case since c3aea7628c, and saved the state from the end of
the previous frame instead. The first overlap sample of every frame was
then filtered differently than when it was encoded as part of the
previous frame.
The overlap of a frame is the tail of the previous frame as it was
encoded, which was zero padded if the input frame was shorter than
frame_size. The last CELT_OVERLAP input samples were copied instead,
so when the last input frame was short and a frame followed it to flush
its overlap, the end of the stream was distorted.
The padding of a short input frame was also written at the start of the
sample buffer instead of after the input frame's samples.
When the range coded data and the raw bits of a frame did not meet, the
bytes between them were copied from the stale contents of the range
coder buffer, and when there were no raw bits, the bytes after the range
coded data were left as whatever the packet buffer held. Zero them
instead, like libopus does.
Always create a compute pipeline instead.
The issue is Mesa does not cache compute shader objects, which wastes upwards of a
second on every single compute shader initialization, instead of just on the first.
They were read one at a time from device-local memory through the BAR,
an uncached read over the bus each, which took 70-90% of the encoder's
CPU time. 1080p on RADV goes from 290 to 625 fps, 4K from 91 to 260 fps.
The frame was inserted into the fragment without a reference, so CBS
never freed it, leaking 35 KB per packet. Keep it in the context, as
the other CBS-based encoders do.
The frame header always left it absent, so every stream decoded as
unspecified with limited range. Take it from each frame, and only mark
it present when it differs from the inferred defaults.
The trellis summed the estimated slice sizes into a separate host buffer
for the CPU to step over the payload with, and wrote each slice's start
offset, which nothing has read since slices went into fixed slots. The
gather writes the packed size of the slices after their sizes; use that,
and drop the buffer and the prefix sums.
The barrier after the trellis now covers the quantisers the encoder
reads, rather than the frame size buffer.
The second field follows the first field's payload, whose size is only
known on the GPU, so it was gathered into a scratch buffer and copied
into the packet by the CPU. Let the gather add an offset it reads from
GPU memory, and point it at the size the first field's gather wrote.
The FFv1, APV and ProRes encoders each drove their execution pool by
hand, and either submitted a frame or returned a packet on each call,
never both. With a depth of N that filled N contexts and then drained
them, leaving the GPU idle for the drain, and the default depth of 1
was fully synchronous.
FFVkEncodeLoop keeps up to pool_size frames in flight, and returns the
oldest packet as soon as its execution completes, or once the pool is
full. It also carries the packet metadata, and flushes what is in
flight, which the encoders advertised but did not implement. ProRes
now honours async_depth and no longer leaks its per-context data on
close.
1080p 4:2:2 10-bit, depth 2: ProRes on RADV goes from 44 to 61 fps and
APV on ANV from 194 to 235 fps. Output is unchanged at every depth.
The packet wraps the pooled readback buffer, so the bytes after the
payload were whatever the previous frame left there, and at the worst
case size the padding fell outside the buffer.
Host-mapped planes start wherever the software frame's do, but
buffer-image copies need offsets aligned to the texel block size, and
on transfer-only queue families to 4 bytes unless maintenance11 is
enabled. Fall back to the staging buffer when they are not.
The FFv1, APV and ProRes encoders each carried a copy of the same
segment gather. Move it into ff_vk_seg_gather(), which also writes the
packed size after the segment sizes, all that FFv1 needed from them,
and makes the output visible to the host, which none of them did.
Gather straight into the host-visible output, as ProRes already did.
FFv1 and APV gathered into VRAM and copied the worst case out, but on a
compute queue that copy is just another shader writing to system memory,
after a round trip through VRAM. This also drops a worst-case VRAM
buffer per frame in flight.
At depth 1 on RADV, APV goes from 12 to 40 fps at 4K and from 44 to
175 fps at 1080p, and FFv1 from 42 to 131 fps at 1080p.
The estimate stored its results at index 0, where the tables held the
constant quantizer's matrix, but the encoder looked both up at the
quantizer itself. The slice sizes never matched the coded slices, and
below 16 every coefficient was divided by an empty matrix.
Fill the tables with every quantizer's matrix, estimate with the
constant quantizer while keeping index 0, and skip the overquant search.
The slides truncated the offset from the opposite end, landing one row
or column behind. The circles used the squared distance ratio. Every
edge now derives from the reversed progress the same way vf_xfade does,
and pixelize samples at integer coordinates.
Slides and wipes are now bit-exact. Fade, circles and pixelize remain
within 1 LSB, as the software filter truncates when storing.
At the first transition frame the offset equals the frame size, so the
first row or column had a remainder of zero and was still pushed up by
one size, reading past the plane.
The bitonic sort needed a 64 KiB shared buffer, more than some devices
have, and for larger slices sorted in place in an unqualified buffer
with a memoryBarrierBuffer() after the barrier, which under the Vulkan
memory model does not make the writes visible to other invocations. It
also takes log2(N)^2/2 barriers, 91 for the UHD slices, and a slice of
millions of pixels times out the GPU.
Replace it with an LSD radix sort, 4 bits per pass, between the pairs
region of a plane and a scratch region. Each subgroup owns a block of
rows of elements, one per lane, and ranks them with ballots from the
digit counts of the blocks, which take up to 16 KiB of shared memory.
The bitmaps of the planes now follow the pairs of all planes, as the
setup shader only fills them once the sort is done, so their region
serves as the scratch and the buffer keeps its size.
At UHD with the default slicing on an RX 6900 XT, the sort of a frame
goes from 14 ms to 3.4 ms.
The encoder kept its own bookkeeping of the last coded value and of a
pending run of consecutive values, and ended a run by coding its length
followed by a zero-length run. The decoder however skips one value after
every run and stays in run mode, so a value exactly two above the last
value of a run is coded as the start of the next run, not as a step from
the value that was skipped. The encoder coded a wrapped-around step of
0xFFFFFFFF for it instead, which sent the decoder past the end of the
value range and corrupted the rest of the slice. Slices with enough
distinct values for this to happen only occur from around 3K upwards.
Rewrite the coder to walk the decoder's state machine, coding every
symbol relative to the decoder's cursor, and only code the multiplier
after the first value when the decoder is going to read it.
The decoder converted the 32-bit output of its transform to 16 bits
through an empirically found offset, and added one to every DC
coefficient to compensate for the bias of the result. Neither
reproduces the transform well: against a bit-exact model of the
reference decoder, most samples are off, with a quantizer-dependent
bias and errors of up to 357 out of 65535 at the extremes of the range,
and the DC offset only helps on HQ material.
Use the 16-bit output variant of simple_idct instead, with the
dequantized coefficients saturated to 16 bits as the reference does,
and the centered transform output biased in the output stage.
Against an exact transform, the error drops from 5.6 LSB RMS with a
maximum of 21 to 0.49 LSB RMS with a maximum of 2.3 and no bias, on par
with the reference decoder's 0.29 and 2.4. The decoded output agrees
with the reference decoder on 95% of the samples of a 5.7K RAW HQ frame
with a log curve, within 3.5 LSB of a 14-bit sensor in its top stop and
within 1 LSB elsewhere. Speed is unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016LFnYMstwe4woBzpxfzDzg
For coefficients and output that both span 16 bits, like those of ProRes
RAW. The template's 16-bit intermediates cannot hold more fractional
bits than the output needs, and the constants of the 12-bit transform,
scaled to 15 bits, make W4 32767 rather than 32768, which alone costs
0.8 LSB RMS of a 16-bit output.
The new variant uses the 10-bit transform's constants, scaled to 14
bits and exact at W4, with a row shift of 13 and a column shift of 15,
the most fractional bits the 32-bit sums of a 16-bit result allow.
Against an exact transform, it is within 0.49 LSB RMS and 2.3 LSB peak
on real content, with no bias, and passes every IEEE 1180 criterion in
the standard's 8-bit units by three orders of magnitude.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016LFnYMstwe4woBzpxfzDzg
Each picture started a recording on the same execution context, and the
second start waited on the value the first had claimed. Start and submit
once around both pictures, and discard the recording when either fails.
The estimate sums the planes of a slice with subgroup shuffles, so
their lanes must share a subgroup, which the workgroup size derived
from the maximum subgroup size did not guarantee. Size the workgroup
to a single subgroup and pin its size where the device allows it.
The overquant search also ran the shuffles inside a loop the slices
leave at different iterations; ANV then sank the unused sum out of the
loop, and every overquant slice scored zero. Keep the loop
subgroup-uniform instead.
Output on ANV is now identical to RADV.
The shared scan and tail hand-off arrays assumed subgroups of at least
32 lanes, but ANV and lavapipe run the tile workgroups with 8 or 16,
and the extra subgroups overran the arrays into each other. The output
on Intel varied from run to run as a result; it now matches RADV.
Each invocation loads one row of its block into shared memory, and the
first DCT pass then transforms a column, which spans the rows of the
other invocations.
The trellis and the encoder sum the bits of all four planes, but
without an alpha plane nothing wrote the fourth. In VRAM this produced
garbage slice sizes, and a frame size that crashed the packet assembly.
Inter frames carry the slice state from one frame to the next in a
single buffer, but the execution contexts rotate across the compute
queues of the device, and nothing ordered one frame's reads and writes
against the previous frame's. Two frames submitted back to back, as
after a flush, corrupted each other on RADV.
Wait on the previous frame's execution before an inter frame.
None of the transfer paths handled a bottom-up software frame. The
staging buffer path stored the linesize in the unsigned row length, so
the buffer size wrapped to several gigabytes per plane, and passed it as
the byte width of av_image_copy_plane(), a negative length for memcpy.
The host image copy path did the same with memoryRowLength, and host
mapping rejected the frame, which only led to the staging buffer.
Use the absolute linesize for the buffer row length and copy width, as
av_image_copy_plane() walks a negative stride itself, and skip host
copies and host mapping for such frames, as neither can address rows
downwards from the plane pointer.
Since the decoder gained an update_thread_context() for the hwaccel
pixel format handoff, the framework no longer calls
ff_thread_finish_setup() on its behalf, and the decoder never called it
itself, so every frame thread waited for the previous frame to be
fully decoded before starting. With no slice threading to fall back
on, the decoder ran on one core whatever the thread count.
Signal the end of the setup once the frame buffer and the hwaccel
private data are allocated. Decoding a 3840x2160 10-bit sequence goes
from no speedup at all to 1.5x on 12 threads, bound by the memory
bandwidth of the unpacking, with identical output.
The decoder advertises frame threading but never called
ff_thread_finish_setup(), so the framework only let the next frame
thread start once the current frame was fully decoded. As frame
threading takes precedence over slice threading, the default thread
configuration decoded one frame at a time on one core.
Signal the end of the setup once the frame buffer and the hwaccel
private data are allocated, as the ProRes decoder does. A 4264x2408
RAW HQ clip goes from 8.4 to 49 fps with the default threading on 12
cores, with identical output.
The exec already waits on the frame's semaphore, which makes earlier
writes available and visible. Inheriting the old flags put shader
accesses on transfer-only queues, flagged as
VUID-VkImageMemoryBarrier2-srcAccessMask-03909 and -07454 on every
hwdownload after a compute filter.
Both were dropped in the compile-time SPIR-V port: the shader kept
planes = 0 and never wrote a frame, and each descriptor binding held
a single plane, so multi-plane updates overflowed into the next one.
Same situation as ProRes RAW: with the packet host-mapped, the VLD is
bound by fetch latency over the bus.
Make GET_BITS_SMEM a specialization constant: 32 lines per invocation
on discrete GPUs reading host-mapped packets, 4 otherwise. The 8x8
workgroup makes that 32 KB of shared memory; 64 does not fit, and 40
or 48 measured slower. An integrated GPU loses a third of its speed at
32, so it keeps 4. The depth is clamped to maxComputeSharedMemorySize
for 32 KB devices such as MoltenVK.
5.7K ProRes HQ on an RX 6900 XT: 85 -> 120 fps, bit-identical output.
The 8x8 scale table was 64 specialization constants indexed per
coefficient. The index is divergent, so nothing folds and every
coefficient costs a load from the constant-data segment.
The table is an outer product, s[y]*s[x], so pass the eight 1D factors
instead: the x factor hoists out of the coefficient loop and the y
factor folds once the loop is unrolled. ProRes RAW's gain of 8 becomes
a literal in its shader.
Reassociation moves a few samples in ten thousand by an LSB or so,
well below the existing float iDCT error; PSNR versus software is
unchanged.
slice_reset_contexts, remap and remap_count were only set on version 4
streams. The slice context buffer is pooled and never cleared, so on
older streams the reset shader read stale memory as the reset flag and
wiped the range coder state carried over from the previous frame.
Non-keyframes of pre-version-4 streams intermittently decoded to
garbage, with the slice CRCs still passing.
With the packet host-mapped, the decode shader is bound by fetch
latency over the bus: 5.49 ms per frame for the VLD alone, versus
0.21 ms from a device-local buffer.
Make GET_BITS_SMEM a specialization constant: 64 lines (1 KB) per
invocation on discrete GPUs reading host-mapped packets, 4 otherwise.
Integrated GPUs get slower with every extra line.
5.7K ProRes RAW on an RX 6900 XT: 145 -> 249 fps (non-HQ), 98 -> 154
fps (HQ), bit-identical output. Intel Iris Xe unchanged.
FILL_SMEM read up to 16*GET_BITS_SMEM bytes past the end of the buffer.
At the default depth of 4 that is exactly AV_INPUT_BUFFER_PADDING_SIZE,
so it only worked by coincidence; deeper windows faulted.
Clamp lines past the end onto the last line; nothing reads them. Issuing
each line as its own load is also ~3% faster than one indexed fetch.
The reference decoder reads every symbol from a 32-bit window that is
zero-filled past the end of the component's data, and stops decoding the
component as soon as that window is all zero. Both our decoders checked for
remaining bits instead, and near the end of the data decoded a zero symbol
from a short window and stored spurious coefficients of magnitude 1 at the
end of the last blocks of a tile. The software decoder also ran one
coefficient past the end of the block array. Decode the way the reference
does, in both, so their coefficients match it, and each other.
The decoder returned before parsing anything when skip_frame was
AVDISCARD_ALL, yet advertises FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM, so it
never set the pixel format and dimensions while probing. Probing gives up
after probesize, and ProRes RAW frames are larger than the default, so
ffmpeg failed to open such files at all with "Could not find codec
parameters ... unspecified pixel format". Parse up to the tile table, which
sets every stream parameter, before skipping the frame.
A frame_num gap dummy shares the duplicated picture's hwaccel state, and
with it the DPB slot that picture was decoded into, which can differ from
the dummy's own index into h->DPB: the dummy and the picture it duplicates
are distinct H264Picture entries. Deriving the Vulkan slot from the DPB
index, as before, could then bind the dummy at an index nothing was ever
decoded into, or one currently in use by an unrelated picture.
Assign the slot once, when a picture is first decoded, to the lowest index
no current reference holds, and keep it for as long as the picture (or a
dummy sharing its state) remains referenced. ff_ctz(0) is undefined, so
guard the case where every slot is taken rather than fall into it.
A slot can still end up requested twice in the same scope: once for the
original picture, once for a dummy sharing its state. Detect that by
identity on the shared hwaccel state rather than by DPB index, since the
two no longer coincide.
A Vulkan DPB slot only becomes active once a decode operation sets it up,
and from then on holds exactly the picture resource that was decoded into
it. The H.264 decoder however hands us references that never went through
the hwaccel: the dummies it synthesizes to fill frame_num gaps and to stand
in for missing references, and fields lost to a seek. Binding those as
active slots fails validation (VUIDs 07239, 07265, 07266, 07151), and since
the slot lookup for long-term references only searched the first 16
pictures and fell back to slot 0, a resource could end up bound twice
(07238, 07264).
Track, per picture, which fields were actually decoded into its slot, and
bind only the references, and only the fields of them, that were:
vk_h264_fill_pict returns 1 when it bound the picture and 0 when it cannot
be bound, and the reference loops advance by that. Use the picture's index
in the DPB as its slot directly instead of searching for it, and describe
the setup slot by the structure of the picture being decoded rather than by
its reference marking, which does not describe the slot's contents while
decoding the second field of a pair.
When decoding the second field of a pair, the picture is already bound as
an active reference at its DPB slot, since the first field made it one.
Appending it again as an inactive slot duplicates the picture resource
within the coding scope, which is not allowed
(VUID-vkCmdBeginVideoCodingKHR-pPictureResource-07238).
Lynne