mirror of
https://github.com/opencv/opencv.git
synced 2026-10-28 08:35:30 +08:00
Prasad Ayush Kumar
f7ca3a382a
support dynamic int8 ConvInteger and MatMulInteger - #30416 Requires: https://github.com/opencv/opencv_extra/pull/1446 Models quantized with ONNX Runtime's dynamic quantization use DynamicQuantizeLinear followed by ConvInteger or MatMulInteger. 5.x has no importer for either op, so none of these models load. ### Models this enables The int8 ONNX files of these models load and run with this PR: - Segmentation: [SAM 3 tracker](https://huggingface.co/onnx-community/sam3-tracker-ONNX), [SAM 2.1 hiera-tiny](https://huggingface.co/onnx-community/sam2.1-hiera-tiny-ONNX), [SegFormer-B3 ADE20K](https://huggingface.co/onnx-community/segformer-b3-finetuned-ade-512-512-ONNX) - Detection: [Grounding DINO tiny](https://huggingface.co/onnx-community/grounding-dino-tiny-ONNX), [OWLv2 base](https://huggingface.co/onnx-community/owlv2-base-patch16-ensemble-ONNX), [RT-DETR R18vd](https://huggingface.co/onnx-community/rtdetr_r18vd), [RF-DETR base](https://huggingface.co/onnx-community/rfdetr_base-ONNX), [YOLO26n](https://huggingface.co/onnx-community/yolo26n-ONNX) - Depth and features: [Depth Anything V2 small](https://huggingface.co/onnx-community/depth-anything-v2-small), [DINOv2 small](https://huggingface.co/onnx-community/dinov2-small), [ViT-B/16](https://huggingface.co/onnx-community/vit-base-patch16-224-ONNX) - Text: [TrOCR base printed](https://huggingface.co/onnx-community/trocr-base-printed-ONNX), [BERT base uncased](https://huggingface.co/Xenova/bert-base-uncased) ### Changes - **ConvInteger** ([opset 10](https://onnx.ai/onnx/operators/onnx__ConvInteger.html)): imported as `Conv2Int8` in a new int32-output mode. The int8 conv kernels (AVX-VNNI, AVX2, NEON, RVV and the scalar path) can now store the int32 accumulators instead of requantizing. Weights and zero points stay inputs, so constant and runtime weights both work. The first commit also fixes the NEON 1x1 fast path, which ignored padding and assumed at least 8 output positions; those cases now take the general path. - **MatMulInteger** ([opset 10](https://onnx.ai/onnx/operators/onnx__MatMulInteger.html)): imported as `MatMulInt8` in a new int32-output mode, backed by a dedicated int8 GEMM in `int8layers/matmul_int8_kernels` (dispatched for AVX2, AVX_VNNI and NEON_DOTPROD, with a universal-intrinsics baseline). A constant B is packed once, and zero points are applied as a correction term so the inner loop is a plain int8 dot product. B can be N-D with broadcast batch dimensions; b_zero_point can be a scalar, one value per column, or per batch and column. A per-row a_zero_point is rejected, as in ONNX Runtime. - ONNX conformance tests `test_basic_convinteger`, `test_convinteger_with_padding`, `test_convinteger_without_padding` and `test_matmulinteger` are enabled (skipped for the OpenVINO backend). ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [ ] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake
OpenCV: Open Source Computer Vision Library
Resources
- Homepage: https://opencv.org
- Courses: https://opencv.org/courses
- Docs: https://docs.opencv.org/5.x/
- Q&A forum: https://forum.opencv.org
- previous forum (read only): http://answers.opencv.org
- Issue tracking: https://github.com/opencv/opencv/issues
- Additional OpenCV functionality: https://github.com/opencv/opencv_contrib
- Donate to OpenCV: https://opencv.org/support/
Contributing
Please read the contribution guidelines before starting work on a pull request.
Summary of the guidelines:
- One pull request per issue;
- Choose the right base branch;
- Include tests and documentation;
- Clean up "oops" commits before submitting;
- Follow the coding style guide.
Additional Resources
- Submit your OpenCV-based project for inclusion in Community Friday on opencv.org
- Subscribe to the OpenCV YouTube Channel featuring OpenCV Live, an hour-long streaming show
- Follow OpenCV on LinkedIn for daily posts showing the state-of-the-art in computer vision & AI
- Apply to be an OpenCV Volunteer to help organize events and online campaigns as well as amplify them
- Follow OpenCV on Mastodon in the Fediverse
- Follow OpenCV on X
- OpenCV.ai: Computer Vision and AI development services from the OpenCV team.