Files
RustPython/Lib/profiling/sampling/binary_reader.py
T
Jeong, YunWonandyouknowdot f0b1e82652 Add tracing profiler and Python 3.15 offline profiling tools (#9271)
* Add deterministic profiler and canonical Python 3.15 tools

Use the merged operation critical sections with interpreter-bound profiler
entry checks and short payload borrows. Trace owned code references for GC.
Import the canonical tracing and offline profiling output closure via the
repository updater, retaining existing test markers.

Assisted-by: dot:GPT-6.1

* Clarify standalone profiling license cross-reference

Preserve license terms and copyright notices; point to the reproduced
Apache text below instead of CPython's original OpenSSL section above.

Assisted-by: dot:GPT-6.1

* Defer unsupported sampling package from profiler extraction

Keep the deterministic tracing profiler and its canonical tools together.
Defer the entirely new sampling production, tests, assets and their license
as one closure until native record and binary-collector support is ready.
Retain canonical pstats for deterministic empty-profile handling; its
upstream sampled-file sorting issue is not patched here.

Assisted-by: dot:GPT-6.1

* Stage profiling root as a namespace until sampling is available

Omit the whole newly added canonical package initializer rather than
advertise an absent sampling module or hand-edit canonical __all__.
Explicit profiling.tracing and cProfile imports remain available.
The namespace root intentionally lacks canonical metadata/automatic exports.

This addresses test___all__ failures in the prior Linux, macOS and Windows
CI runs. No test assertions, skips or canonical file contents are changed.

Assisted-by: dot:GPT-6.1

* Match canonical static builtin profiler labels and metadata

Normalize the raw builtin receiver instead of its call-argument accessor.
Snapshot static method module metadata at construction, matching CPython,
so explicit module mutation and deletion remain observable. Avoid a
profiler-time fallback that would obscure deleted metadata.

Correct float.__getformat__ to CPython's classmethod calling convention,
including subclass binding, rather than special-casing its profiler label.

Assisted-by: dot:GPT-6.1

* Support collision-safe frozen package containers

Generate empty frozen packages only for source namespace directories with
real frozen descendants. Preserve genuine modules and packages across
frozen-library registrations while retaining real-real replacement order.

Record the container marker in FrozenModule and its library envelope;
legacy module/package encodings remain unchanged. This is closed-tree
frozen packaging, not full PEP 420 namespace merging.

Add structural codec, mock generator, and registration-priority checks.

* Revert frozen package containers before restoring canonical profiling root

The container implementation changed filesystem namespace precedence.
Restore the existing importer and frozen envelope unchanged; the following
offline sampling closure restores the canonical regular package root.

This reverses d5a8049234.

Assisted-by: dot:GPT-6.1

* Restore canonical profiling package with offline sampling codec

Provide native records, status constants and genuine uncompressed binary
reader/writer APIs for canonical offline tools. Keep the unsupported live
process capability absent and preserve existing canonical test gates.
Restore the complete selected sampling production/test/assets closure via
update_lib, including its asyncio frame-schema dependency. The prior frozen
container extension is reverted separately; canonical packages work with
both filesystem and frozen imports without changing namespace semantics.

Assisted-by: dot:GPT-6.1

* Keep platform-dependent C long normalization explicit

Allow only the c_long-to-i32 conversion lint: Windows uses the same type,
while LP64 requires the existing truncation to match CPython normalization.
No runtime behavior or test assertions change.

Assisted-by: dot:GPT-6.1

* Include profiling notices in library and desktop packages

Bundle a byte-identical copy of the root profiling notice in rustpython-pylib
and include the root notice in configured desktop resources. Avoid a symlink
so Windows checkouts preserve the actual notice text. Terms are unchanged.

Assisted-by: dot:GPT-6.1

* Keep profiling asset license notice in pylib crate

* Serve frozen profiling assets through native resource readers

* Preserve canonical profiling asset line endings on checkout

---------

Co-authored-by: youknowdot <dot@youknowone.org>
2026-10-22 11:18:03 +09:00

149 lines
5.3 KiB
Python
Vendored

"""Thin Python wrapper around C binary reader for profiling data."""
import _remote_debugging
from .gecko_collector import GeckoCollector
from .stack_collector import FlamegraphCollector, CollapsedStackCollector
from .jsonl_collector import JsonlCollector
from .pstats_collector import PstatsCollector
from .binary_collector import CAPTURE_FEATURES
class BinaryReader:
"""High-performance binary reader using C implementation.
This reader uses memory-mapped I/O (on Unix) for fast replay of
profiling data from binary files.
Use as a context manager:
with BinaryReader('profile.bin') as reader:
info = reader.get_info()
reader.replay_samples(collector, progress_callback)
"""
def __init__(self, filename):
"""Create a new binary reader.
Args:
filename: Path to input binary file
"""
self.filename = filename
self._reader = None
def __enter__(self):
self._reader = _remote_debugging.BinaryReader(self.filename)
return self
def __exit__(self, exc_type, exc_val, exc_tb):
if self._reader is not None:
self._reader.close()
self._reader = None
return False
def get_info(self):
"""Get metadata about the binary file.
Returns:
dict: File metadata including:
- sample_count: Number of samples in the file
- sample_interval_us: Sampling interval in microseconds
- start_time_us: Start timestamp in microseconds
- string_count: Number of unique strings
- frame_count: Number of unique frames
- compression: Compression type used
- mode: Profiling mode, or None if not recorded
- capture_config: Capture feature mapping, or None if not
recorded
"""
if self._reader is None:
raise RuntimeError("Reader not open. Use as context manager.")
info = self._reader.get_info()
capture_features = info.pop("capture_features")
info["capture_config"] = (
None if capture_features is None else {
name: bool(capture_features & bit)
for name, bit in CAPTURE_FEATURES.items()
}
)
return info
def replay_samples(self, collector, progress_callback=None):
"""Replay samples from binary file through a collector.
This allows converting binary profiling data to other formats
(e.g., flamegraph, pstats) by replaying through the appropriate
collector.
Args:
collector: A Collector instance with a collect() method
progress_callback: Optional callable(current, total) for progress
Returns:
int: Number of samples replayed
"""
if self._reader is None:
raise RuntimeError("Reader not open. Use as context manager.")
return self._reader.replay(collector, progress_callback)
@property
def sample_count(self):
if self._reader is None:
raise RuntimeError("Reader not open. Use as context manager.")
return self._reader.get_info()['sample_count']
def get_stats(self):
"""Get reconstruction statistics from replay.
Returns:
dict: Statistics about record types decoded and samples
reconstructed during replay.
"""
if self._reader is None:
raise RuntimeError("Reader not open. Use as context manager.")
return self._reader.get_stats()
def convert_binary_to_format(input_file, output_file, output_format,
sample_interval_usec=None, progress_callback=None):
"""Convert a binary profiling file to another format.
Args:
input_file: Path to input binary file
output_file: Path to output file
output_format: Target format ('flamegraph', 'collapsed', 'pstats', etc.)
sample_interval_usec: Override sample interval (uses file's if None)
progress_callback: Optional callable(current, total) for progress
Returns:
int: Number of samples converted
"""
with BinaryReader(input_file) as reader:
info = reader.get_info()
interval = sample_interval_usec or info['sample_interval_us']
# Create appropriate collector based on format
if output_format == 'flamegraph':
collector = FlamegraphCollector(interval)
elif output_format == 'collapsed':
collector = CollapsedStackCollector(interval)
elif output_format == 'pstats':
collector = PstatsCollector(interval)
elif output_format == 'gecko':
collector = GeckoCollector(interval)
elif output_format == "jsonl":
collector = JsonlCollector(interval, mode=info.get("mode"))
else:
raise ValueError(f"Unknown output format: {output_format}")
# Replay samples through collector
count = reader.replay_samples(collector, progress_callback)
if hasattr(collector, "set_replay_stats"):
collector.set_replay_stats(info)
if hasattr(collector, "set_mode"):
collector.set_mode(info.get("mode"))
# Export to target format
collector.export(output_file)
return count