SEI NAL units following the last VCL NAL unit start a new access unit,
but some files keep them in the same packet as the preceding picture,
e.g. YouTube encodes carry an unregistered user data SEI after the slice
data. With frame threading such SEI was skipped and a "Late SEI is not
implemented" warning was printed for every affected packet, because the
SEI context is copied to the next frame thread in
ff_h264_update_thread_context() and must not be modified after
ff_thread_finish_setup(). Single threaded decoding decoded the SEI and
kept its persistent state, like the alternative transfer characteristics
or the x264 build, for the following pictures.
Decode such SEI right before finishing the setup instead. The current
picture is already set up at this point, so the SEI does not apply to
it, matching the single threaded behavior.
Fixes: https://code.ffmpeg.org/FFmpeg/FFmpeg/issues/21348
Fixes: https://github.com/mpv-player/mpv/issues/18104
Signed-off-by: Kacper Michajłow <kasper93@gmail.com>
When skip_pred and skip_idct are both AVDISCARD_ALL, no pixel of any
frame is wanted, and since slices are self-contained NAL units the
macroblock layer is not needed to stay in sync with the bitstream.
Parse only the slice headers, run the reference and DPB bookkeeping
and output frames with valid metadata and uninitialized pixels.
Error resilience, film grain synthesis, the gray fill of frames
synthesized for frame_num gaps and of chroma planes for CODEC_FLAG_GRAY
and the duplication of a missing field write or read pixels, so all of
them are disabled in this mode. The mode is not entered with
AV_CODEC_FLAG2_CHUNKS, whose frame completion tracking needs the
macroblock position of the decoded slices, nor when error detection,
motion vector export, video encoding parameter export or macroblock
debugging is requested, as these need the slice data the mode does not
parse.
Decoding a 4690 frame 720x576 stream single threaded on x86_64
(gcc, mean +- stddev over 12 interleaved runs each):
full decode: 2.973 +- 0.014 s
metadata only: 0.058 +- 0.007 s
51x faster, p = 5e-37 (Welch)
An inter neighbour must not contribute its coefficient count to nC for
an intra macroblock when constrained intra prediction and data
partitioning are both in use; otherwise decoding partitions A and B
alone would depend on data that lives in partition C.
The neighbour is kept available with nN = 0, which is what JM does and
what the streams it produces expect. FF_BUG_H264_DP_NNZ selects that
and is autodetected on the first partition A, while -bug 0 drops the
neighbour from the average as 9.2.1 has it. Streams without data
partitioning decode identically either way.
This replaces a FIXME older than any data partitioning support.
Signed-off-by: guy-with-a-why <aaron@iterationlab.io>
Syntax elements of category 2 stay in partition A, category 3 (intra
residual) moves to partition B and category 4 (inter residual) to
partition C, so CAVLC has to read the residual of each macroblock from
the partition that matches its type. A partition that never arrived is
reported instead of being read from A.
B and C are taken from the NAL units immediately following their A and
matched to it by slice_id. Arbitrary slice order may separate them
(7.4.1.2.5); that is not supported, and such a stream reports the
missing partition instead. Attaching runs while the slice is queued,
before any slice is executed, so no partition can arrive too late to
be used. ff_h264_queue_decode_slice() reports which context it queued,
as a slice opening a second field is moved into slot zero once the
pending field has been decoded.
Attaching and routing land together on purpose: enabling the partition
A path on its own would decode partitioned streams incorrectly instead
of reporting them unsupported. hwaccel formats are not offered for a
partitioned picture and CODEC_FLAG2_CHUNKS is refused, as both assume
one self-contained slice NAL.
Signed-off-by: guy-with-a-why <aaron@iterationlab.io>
Add CUARRAY hwaccel structs (ff_<codec>_nvdec_cuarray_hwaccel) and the
corresponding AV_PIX_FMT_CUARRAY entries in each decoder's pixel format
negotiation list. This allows decoders to advertise CUARRAY as a
supported hardware output format alongside AV_PIX_FMT_CUDA.
Codecs: H.264, HEVC, AV1, MPEG-1, MPEG-2, MPEG-4, VC-1, WMV3, VP8, VP9.
Signed-off-by: Diego de Souza <ddesouza@nvidia.com>
(This also stops zero-allocating er_temp_buffer for H.264,
reverting back to the behavior from before commit
0a1dc817237b8546189960efd523257b4f62e1ab.)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This commit adds a reference to the buffer as an argument to
start_frame, and adapts all existing code.
This allows for asynchronous hardware accelerators to skip
copying packet data by referencing it.
It is pointless to try to report the true number because
decode_simple_internal() always treats the buffer as
completely consumed for video codecs anyway.
(Apart from that: The h263dec code would report to only
have consumed the header data in case decoding is aborted
due to skip_frame which makes no sense.)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This fixes timeout issues and seems like it was intended
since the line emits an error log.
Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Marth64 <marth64@proxyid.net>
AVFilmGrainAFGS1Params, the offending struct, is using sizeof(AVFilmGrainParams)
when it should not. This change also forgot to make the necessary changes to the
frame threading sync code.
Both of these will be fixed by the following commit.
H274FilmGrainDatabase will be handled later.
This reverts commit 08b1bffa49.
Signed-off-by: James Almer <jamrial@gmail.com>
Film grain support adds a huge amount of overhead to the H264Context
structure for a feature that is rarely used. On low end devices or
pages that have lots of media this bloats memory usage rapidly.
This changes the static film grain metadata allocations to be dynamic
which reduces the H264Context size from 851808 bytes to 53444 bytes.
Bug: https://crbug.com/359358875
Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Niklas Haas <git@haasn.dev>
The earlier code had two problems:
1. For reference frames that are not directly output (happens unless
low_delay is set), the mb skip values referred to the next reference
frame to be decoded.
2. For non-reference frames, every macroblock was always considered
skipped.
This makes the output (worse than) useless; that no one ever
complained about this shows that this feature is not really used.
It is therefore removed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Before commit f025b8e110,
every frame-threaded decoder used ThreadFrames, even when
they did not have any inter-frame dependencies at all.
In order to distinguish those decoders that need the AVBuffer
for progress communication from those that do not (to avoid
the allocation for the latter), the former decoders were marked
with the FF_CODEC_CAP_ALLOCATE_PROGRESS internal codec cap.
Yet distinguishing these two can be done in a more natural way:
Don't use ThreadFrames when not needed and split ff_thread_get_buffer()
into a core function that calls the user's get_buffer2 callback
and a wrapper around it that also allocates the progress AVBuffer.
This has been done in 02220b88fc
and since that commit the ALLOCATE_PROGRESS cap was nearly redundant.
The only exception was WebP and VP8. WebP can contain VP8
and uses the VP8 decoder directly (i.e. they share the same
AVCodecContext). Both decoders are frame-threaded and VP8
has inter-frame dependencies (in general, not in valid WebP)
and therefore the ALLOCATE_PROGRESS cap. In order to avoid
allocating progress in case of a frame-threaded WebP decoder
the cap and the check for the cap has been kept in place.
Yet now the VP8 decoder has been switched to use ProgressFrames
and therefore there is just no reason any more for this check
and the cap. This commit therefore removes both.
Also change the value of FF_CODEC_CAP_USES_PROGRESSFRAMES
to leave no gaps.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
When starting on a SEI recovery point close enough to the end of
the stream that draining starts before the recovery point's frame
is output, there can be non-recovered frames in the delayed picture
buffer that would currently cause the decoder to fail to output a
frame. This commit skips such frames and outputs the first recovered
frame, if there exists one.
Signed-off-by: arch1t3cht <arch1t3cht@gmail.com>
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
When decoding starts at a SEI recovery point very shortly before the
end of the video stream, there can be frames which are decoded before
the recovery point's frame is output and which will only be output once
the draining has started. Previously, these frames would never be set
as recovered. This commit copies the logic from h264_select_output_frame
to send_next_delayed_frame to properly mark such frames as recovered.
Fixes ticket #10936.
Signed-off-by: arch1t3cht <arch1t3cht@gmail.com>
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
There are lots of files that don't need it: The number of object
files that actually need it went down from 2011 to 884 here.
Keep it for external users in order to not cause breakages.
Also improve the other headers a bit while just at it.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Unnecessary since acf63d5350adeae551d412db699f8ca03f7e76b9;
also avoids relocations.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
The implementation is based on:
https://learn.microsoft.com/en-us/windows/win32/medfound/direct3d-12-video-overview
With the Direct3D 12 video decoding support, we can render or process
the decoded images by the pixel shaders or compute shaders directly
without the extra copy overhead, which is beneficial especially if you
are trying to render or post-process a 4K or 8K video.
The command below is how to enable d3d12va:
ffmpeg -hwaccel d3d12va -i input.mp4 output.mp4
Signed-off-by: Wu Jianhua <toqsxw@outlook.com>
Signed-off-by: Tong Wu <tong1.wu@intel.com>
It involves less allocations and therefore has the nice property
that deriving a reference from a reference can't fail.
This allows for considerable simplifications in
ff_h264_(ref|replace)_picture().
Switching to the RefStruct API also allows to make H264Picture
smaller, because some AVBufferRef* pointers could be removed
without replacement.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
It is unnecessary since the removal of non-thread-safe callbacks
in e0786a8eeb. Since then, the
AVCodecContext has only been used as logcontext.
Removing ff_thread_release_buffer() allowed to remove AVCodecContext*
parameters from several other functions (not only unref functions,
but also e.g. ff_h264_ref_picture() which calls ff_h264_unref_picture()
on error).
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
When using multi-threaded decoding, every decoding thread
has its own DBP consisting of H264Pictures and each of these
points to its own AVFrames. They are synced during
update_thread_context via av_frame_ref() and therefore
the threads actually decoding (as well as all the others)
must not modify any field that is copied by av_frame_ref()
after ff_thread_finish_setup().
Yet this is exactly what happens when an error occurs
during decoding and the AVFrame's decode_error_flags are updated.
Given that these errors only become apparent during decoding,
this can't be set before ff_thread_finish_setup() without
defeating the point of frame-threading; in practice,
this meant that the decoder did not set these flags correctly
in case frame-threading was in use. (This means that e.g.
the ffmpeg cli tool fails to output its "corrupt decoded frame"
message in a nondeterministic fashion.)
This commit fixes this by adding a new H264Picture field
that is actually propagated across threads; the field
is an AVBufferRef* whose data is an atomic_int; it is
atomic in order to allow multiple threads to update it
concurrently and not to provide synchronization
between the threads setting the field and the thread
ultimately returning the AVFrame.
This unfortunately has the overhead of one allocation
per H264Picture (both the original one as well as
creating a reference to an existing one), even in case
of no errors. In order to mitigate this, an AVBufferPool
has been used and only if frame-threading is actually
in use. This expense will be removed as soon as
a proper API for refcounted objects (not based upon
AVBuffer) is in place.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Add a pointer parameter that if supplied will be used to return
the updated decode_error_flags. This will allow to fix several
races when using frame-threading; these resulted from AVFrame
that the earlier code updated concurrently being used as source
in an av_frame_ref() call in the decoder's update_thread_context.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This commit is the AVHWAccel analogue of commit
20f9727018: It moves the private fields
of AVHWAccel to a new struct FFHWAccel extending AVHWAccel
in an internal header (namely hwaccel_internal.h).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
For encoding, this field is entirely redundant with
AVCodecContext.framerate.
For decoding, this field is entirely redundant with
AV_CODEC_PROP_FIELDS.
It reduces typing: Before this patch, there were 11 callbacks
that exceeded the 80 char line length limit; now there are zero.
It also allows to remove ONLY_IF_THREADS_ENABLED() in
libavutil/internal.h.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Kacper Michajłow