Commits 0cd75dbfa0 and 27e94281d1 added QueryInterface()
implementations that compare the requested interface against
IID_IUnknown. The Linux SDK defines it in LinuxCOM.h and the Windows
SDK gets it from the platform COM headers, but the macOS SDK uses
CoreFoundation's COM types from <CoreFoundation/CFPlugInCOM.h>, which
only provide IUnknownUUID as a CFUUIDRef. Since then, building with
--enable-decklink on macOS fails:
libavdevice/decklink_dec.cpp:138:43: error: use of undeclared
identifier 'IID_IUnknown'
libavdevice/decklink_dec.cpp:603:43: error: use of undeclared
identifier 'IID_IUnknown'
libavdevice/decklink_enc.cpp:120:39: error: use of undeclared
identifier 'IID_IUnknown'
libavdevice/decklink_enc.cpp:181:39: error: use of undeclared
identifier 'IID_IUnknown'
On macOS, REFIID is CFUUIDBytes, so derive IID_IUnknown from
IUnknownUUID with CFUUIDGetUUIDBytes(), the same definition used by
the macOS samples shipped with the DeckLink SDK. This avoids
duplicating the GUID bytes, and CFUUIDGetConstantUUIDWithBytes()
returns a constant that must not be released. The definition is
limited to __APPLE__, so Linux and Windows builds are unchanged.
Tested with DeckLink SDK 16.0 and Apple clang 21 on macOS arm64.
Signed-off-by: Jake Fineman <jake@wave.online>
With input_resizable enabled, get_output_ov() created partial_shape with
ov_partial_shape_create() and then passed the same variable to
ov_shape_to_partial_shape(), which allocates a new dims array and
overwrites the struct. The first allocation was never freed, so every
call leaked it, on the success path and on the two error returns in
between. The dims it was built from were never used.
Drop the ov_partial_shape_create() call and the dims array; the partial
shape is built from the model's input shape by ov_shape_to_partial_shape()
and freed after ov_model_reshape_single_input() as before.
The leak was introduced with the OpenVINO API 2.0 support in e79bd1f1b1.
Signed-off-by: Jake Fineman <jake@wave.online>
ff_proc_from_dnn_to_frame() and ff_proc_from_frame_to_dnn() each declare a
single `void *middle_data` local and, on the DL_NCHW layout branch, take its
address (`dst_data = &middle_data;` / `src_data = &middle_data;`) and pass
that as sws_scale()'s dst/src plane-array argument after casting to
`uint8_t *const *`. sws_scale()'s dst/src parameters are a 4-element array
(uint8_t *const dst[4]/src[4]), and scale_internal() copies all four
pointers (memcpy of srcSlice/dstSlice into src2/dst2) regardless of how many
planes the actual conversion uses. Since `&middle_data` only ever points at
one pointer-sized stack slot, that copy reads past it into whatever is next
on the stack: an out-of-bounds stack read on
every dnn_processing call whose model uses NCHW layout output or input into
an RGB24/BGR24 frame.
Fix both by giving dst_data/src_data real 4-element backing storage
(dst_data_buf/src_data_buf, zero-initialized) with the allocated buffer in
slot 0, instead of aliasing the single pointer.
Tested on arm64 macOS 27.0 (M1 Ultra), clang-asan toolchain, with
--enable-libopenvino (2026.4.0), --enable-libonnxruntime (1.30.0) and
--enable-libtensorflow, against unmodified upstream master (ff2059afd9) and
against this commit applied on top of the same base:
- Reproduced the stack-buffer-overflow on unmodified master two ways:
- ff_proc_from_dnn_to_frame(): `ffmpeg -threads 1 -i puppies.ogv -an -vf
"fps=2,format=rgb24,dnn_processing=dnn_backend=openvino:
model=proc_valid_720.onnx:input=input:output=output" -f framemd5 out.md5`
aborts under ASan on the first frame (offset-72 read into the
`middle_data`/`planar_data` stack slots, from the OpenVINO async
inference callback thread).
- ff_proc_from_frame_to_dnn(): the same dnn_processing filter with two real
ONNX Model Zoo fast-neural-style models (candy-9.onnx,
sha256 9d11a3529d1e547da6ae07201d93484dbab2ec0a3614535752c8f40f0fe2968a;
mosaic-9.onnx,
sha256 fa646dedade881243f8d5a2ceb7de2b93675b21fc24f7482894ac4851a9a0a47;
both NCHW 1x3x224x224 in/out) via dnn_backend=onnx aborts under ASan the
same way, this time with the overflow located directly in
ff_proc_from_frame_to_dnn's `middle_data` frame (offset 72), confirming
the sibling instance flagged at review is independently live, not just
structurally identical.
With this commit applied, all three of the above runs are clean of the
stack-buffer-overflow report (an unrelated, pre-existing OpenVINO/TBB
at-exit SEGV under ASan remains on this machine/toolchain/OpenVINO version;
it reproduces identically before and after this commit, and even on plain
`ffmpeg -version` with no DNN filter at all, so it is unrelated to this fix
and not addressed here).
- Whole-clip (puppies.ogv, 589 frames) framemd5 output from a release
(non-ASan) build is byte-for-byte identical before/after this commit for:
the NCHW OpenVINO identity model (proc_valid_720.onnx) into rgb24, and for
both candy-9.onnx and mosaic-9.onnx into rgb24 via dnn_backend=onnx. This
is expected: the aliasing read pulls garbage bytes that sws_scale never
writes to a packed destination/never reads for these plane counts, so the
visible output is unaffected; only ASan's redzone instrumentation makes the
read observable.
- Unchanged paths are also byte-for-byte identical before/after: the NHWC
TensorFlow path (tf_rgb_valid.pb into rgb24, which never takes the
DL_NCHW/middle_data branch) and the planar/gray output path (tf_y_valid.pb
into gray8, which uses frame->data directly and never allocates
middle_data).
- `make fate-source` passes with this commit. This tree has no fate-dnn*
tests to run.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Jake Fineman <jake@wave.online>
ff_proc_from_dnn_to_frame()'s AV_PIX_FMT_RGB24/BGR24 case assumes the
model's output has exactly 3 channels, but never checks it.
The first sws_scale() in that case reads from output->data with a
hardcoded source stride of frame->width * 3 * src_datatype_size over
frame->height rows, i.e. frame->width * 3 * frame->height *
src_datatype_size bytes. For a model whose output has fewer channels
than that -- e.g. the single-channel alpha matte an RGB-in
background-removal model produces -- the real tensor only holds
frame->width * frame->height * channels * src_datatype_size bytes, so
the call reads roughly 3x past the end of it. This affects both
DL_NCHW and DL_NHWC output, since it happens before the layout-specific
planar-to-packed branch.
For DL_NCHW there is a second, independent overflow on the write side:
middle_data is allocated plane_size * output->dims[1] bytes, but
linesize[0] is set to frame->width * 3 unconditionally, so the same
sws_scale() also writes past that allocation. This reproduces as a
SIGABRT (glibc "free(): invalid pointer", or a malloc.c assertion,
depending on allocator state) inside dnn_processing, after inference
has already completed successfully in the backend.
The input side is guarded -- check_modelinput_inlink() in
vf_dnn_processing.c -- but the output side is not. dnn_processing only
supports channel-preserving models for these pixel formats by design,
so reject a mismatched channel count with a clear error instead of
reading and writing out of bounds. The channel dimension is resolved
via dnn_get_channel_idx_by_layout() so the check covers both layouts.
This does not add support for rendering a 1- or 2-channel model output
through this filter; that would need a different pixel-format target
and is left as future work.
Signed-off-by: Jake Fineman <jake@wave.online>
get_output_tf() (dnn_backend_tf.c) and get_output_ov()
(dnn_backend_openvino.c) both declare `TaskItem task;` uninitialized
and then unconditionally free task.in_frame/task.out_frame on their
error path:
ret = ff_dnn_fill_gettingoutput_task(&task, ...);
if (ret != 0)
goto err;
...
err:
av_frame_free(&task.out_frame);
av_frame_free(&task.in_frame);
ff_dnn_fill_gettingoutput_task() (dnn_backend_common.c) allocates
in_frame first and, only if that succeeds, allocates out_frame and
assigns both pointers into exec_params (and from there into the
task via ff_dnn_fill_task()). If the first av_frame_alloc() fails,
the function returns AVERROR(ENOMEM) immediately, before task.in_frame
or task.out_frame is ever set. The two probes then reach `err:` with
task's frame fields holding whatever garbage was on the stack, and
av_frame_free() dereferences those pointers.
Fix by zero-initializing the declaration, matching get_output_onnx()
(dnn_backend_onnx.c) and get_output_th() (dnn_backend_torch.cpp),
which already use `TaskItem task = { 0 };` and are not affected.
Reported by Forgejo_Fairy, review 8971 on #24761, which also
confirmed the defect predates that series and is already present at
6246f7a4cd.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Jake Fineman <jake@wave.online>
Jake Fineman