lzma: treat empty input as a no-op, not end of stream (#8752)

decompress_chunks reports whether the codec signalled LZMA_STREAM_END,
and it short-circuits when the chunker has nothing left to feed. That
early return answered "the stream ended" for the case where no input was
supplied at all, so LZMADecompressor.decompress(b"") claimed a finish the
decoder had never signalled.

The caller latches that answer with `self.eof |= stream_end`, which only
ever flips eof on. A single empty call therefore finished the object for
good: needs_input went false and every later call was rejected by the
`if self.eof` guard with EOFError, before a single compressed byte had
been seen. CPython leaves eof false and needs_input true, and accepts
compressed data handed over by a later call.

  d = lzma.LZMADecompressor()
  d.decompress(b"")   # d.eof was True, now False
  d.decompress(blob)  # was EOFError, now b'data'

The zlib engine returns false from the same short-circuit; lzma had
drifted away from it. bz2 was fixed incidentally by #8638 when its engine
moved into crates/common, so only lzma needed a code change.

Verified on macOS arm64 against CPython 3.14.7: the script from the issue
now prints the documented "False True" and b'data' for both
decompressors. test_decompressor_chunks_empty in Lib/test/test_lzma.py
covers exactly this and no longer needs its expectedFailure marker. The
added engine test is named after the zlib one that already existed and is
mirrored into bz2, so all three engines stay pinned to the same
behaviour. test_lzma, test_bz2, test_zlib, test_gzip, test_tarfile,
cargo test --workspace, cargo fmt --check and cargo clippy
-p rustpython-common --all-features --all-targets -D warnings are clean.

Fixes #8637

Assisted-by: Claude Code:claude-opus-5
This commit is contained in:
hyoinandout authored and GitHub committed 2026-09-20 17:22:27 +09:00
1 parent f759c18a0e
commit 4c7cc0a30c
3 files changed
+35 -2

No files matched your search

-1
View File
@@ -143,7 +143,6 @@ class CompressorDecompressorTestCase(unittest.TestCase):
self.assertTrue(lzd.eof)
self.assertEqual(lzd.unused_data, b"")
@unittest.expectedFailure # TODO: RUSTPYTHON; EOFError: End of stream already reached
def test_decompressor_chunks_empty(self):
lzd = LZMADecompressor()
out = []
+17
View File
@@ -312,6 +312,23 @@ mod tests {
assert!(decompressor.eof());
}
#[test]
fn empty_input_does_not_finish_a_stream() {
let mut compressor = Compressor::new(9).unwrap();
let mut encoded = compressor.compress(b"later input").unwrap();
encoded.extend(compressor.flush().unwrap());
let mut decompressor = Decompressor::new();
assert_eq!(decompressor.decompress(b"", None).unwrap(), b"");
assert!(!decompressor.eof());
assert!(decompressor.needs_input());
assert_eq!(
decompressor.decompress(&encoded, None).unwrap(),
b"later input"
);
assert!(decompressor.eof());
}
#[test]
fn bad_data_latches_failure() {
let mut decompressor = Decompressor::new();
+18 -1
View File
@@ -378,8 +378,10 @@ fn decompress_chunks(
stream: &mut LzmaStream,
max_length: Option<usize>,
) -> Result<(Vec<u8>, bool), XzError> {
// No input to feed the decoder: not a stream end, so `eof` stays false and
// the caller can still supply compressed data later.
if chunks.is_empty() {
return Ok((Vec::new(), true));
return Ok((Vec::new(), false));
}
let max_length = max_length.unwrap_or(usize::MAX);
let mut output = Vec::new();
@@ -622,6 +624,21 @@ impl Compressor {
mod tests {
use super::*;
#[test]
fn empty_input_does_not_finish_a_stream() {
let mut compressor = Compressor::new(FORMAT_XZ, CHECK_CRC64, PRESET_DEFAULT, None).unwrap();
let mut encoded = compressor.compress(b"data").unwrap();
encoded.extend(compressor.flush().unwrap());
let mut decompressor = Decompressor::new(FORMAT_AUTO, None, None).unwrap();
assert_eq!(decompressor.decompress(b"", None).unwrap(), b"");
assert!(!decompressor.eof());
assert!(decompressor.needs_input());
assert_eq!(decompressor.decompress(&encoded, None).unwrap(), b"data");
assert!(decompressor.eof());
}
#[test]
fn encode_lzma1_properties_rejects_invalid_lclppb() {
for (lc, lp, pb) in [(5, 0, 0), (0, 5, 0), (4, 1, 0), (0, 0, 5)] {