mirror of
https://github.com/nodejs/node.git
synced 2026-10-28 18:25:13 +08:00
Philipp Dunkel
df2ea49e00
Add zlib.crc32c(data[, value]), which computes the CRC-32C checksum (Castagnoli polynomial, RFC 3720 section 12.1). CRC-32C has better error detection properties than the CRC-32 of zlib.crc32() and is the checksum required by many current protocols, formats and services, among them: - iSCSI header and data digests (RFC 7143) - SCTP packet checksums (RFC 9260) - iWARP MPA framing (RFC 5044) - NVMe/TCP header and data digests - ext4, Btrfs and XFS metadata checksums - the Snappy framing format - Apache Kafka record batches - LevelDB and RocksDB block checksums - TensorFlow TFRecord files - Google Cloud Storage and Amazon S3 object checksums Implementing any of these in JavaScript currently needs a userland CRC-32C, which is far slower than native code. The function mirrors zlib.crc32(): it accepts a string (encoded as UTF-8) or an ArrayBufferView, takes an optional 32-bit unsigned starting value so a checksum can be continued across chunks, and has a V8 fast API path. The checksum is computed by absl::ExtendCrc32c() from the bundled Abseil, which is already built and linked into node, so this adds no new dependency and no new checksum implementation. absl_crc32c is added to the libraries linked with --shared-abseil. Abseil selects its hardware-accelerated CRC-32C path (SSE4.2 and PCLMULQDQ on x64, the CRC32 and PMULL extensions on arm64) at compile time, so with node's default compiler flags the portable table-driven implementation is used. It runs at about the same speed as zlib.crc32() (about 6.5 GB/s on a recent x64 machine), while the hardware path reaches about 38 GB/s. Builds whose target CPU baseline already includes those instructions get the hardware path without changes. Enabling it for generic builds is left out on purpose: on x64, Abseil uses the accelerated code for unrecognized CPUs without a runtime feature check, so compiling it in unconditionally would fault on CPUs without SSE4.2. Signed-off-by: Philipp Dunkel <pip@pipobscure.com> PR-URL: https://github.com/nodejs/node/pull/66828 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Paolo Insogna <paolo@cowtech.it> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>