mirror of
https://github.com/RustPython/RustPython.git
synced 2026-10-28 10:44:36 +08:00
* Add deterministic profiler and canonical Python 3.15 tools
Use the merged operation critical sections with interpreter-bound profiler
entry checks and short payload borrows. Trace owned code references for GC.
Import the canonical tracing and offline profiling output closure via the
repository updater, retaining existing test markers.
Assisted-by: dot:GPT-6.1
* Clarify standalone profiling license cross-reference
Preserve license terms and copyright notices; point to the reproduced
Apache text below instead of CPython's original OpenSSL section above.
Assisted-by: dot:GPT-6.1
* Defer unsupported sampling package from profiler extraction
Keep the deterministic tracing profiler and its canonical tools together.
Defer the entirely new sampling production, tests, assets and their license
as one closure until native record and binary-collector support is ready.
Retain canonical pstats for deterministic empty-profile handling; its
upstream sampled-file sorting issue is not patched here.
Assisted-by: dot:GPT-6.1
* Stage profiling root as a namespace until sampling is available
Omit the whole newly added canonical package initializer rather than
advertise an absent sampling module or hand-edit canonical __all__.
Explicit profiling.tracing and cProfile imports remain available.
The namespace root intentionally lacks canonical metadata/automatic exports.
This addresses test___all__ failures in the prior Linux, macOS and Windows
CI runs. No test assertions, skips or canonical file contents are changed.
Assisted-by: dot:GPT-6.1
* Match canonical static builtin profiler labels and metadata
Normalize the raw builtin receiver instead of its call-argument accessor.
Snapshot static method module metadata at construction, matching CPython,
so explicit module mutation and deletion remain observable. Avoid a
profiler-time fallback that would obscure deleted metadata.
Correct float.__getformat__ to CPython's classmethod calling convention,
including subclass binding, rather than special-casing its profiler label.
Assisted-by: dot:GPT-6.1
* Support collision-safe frozen package containers
Generate empty frozen packages only for source namespace directories with
real frozen descendants. Preserve genuine modules and packages across
frozen-library registrations while retaining real-real replacement order.
Record the container marker in FrozenModule and its library envelope;
legacy module/package encodings remain unchanged. This is closed-tree
frozen packaging, not full PEP 420 namespace merging.
Add structural codec, mock generator, and registration-priority checks.
* Revert frozen package containers before restoring canonical profiling root
The container implementation changed filesystem namespace precedence.
Restore the existing importer and frozen envelope unchanged; the following
offline sampling closure restores the canonical regular package root.
This reverses d5a8049234.
Assisted-by: dot:GPT-6.1
* Restore canonical profiling package with offline sampling codec
Provide native records, status constants and genuine uncompressed binary
reader/writer APIs for canonical offline tools. Keep the unsupported live
process capability absent and preserve existing canonical test gates.
Restore the complete selected sampling production/test/assets closure via
update_lib, including its asyncio frame-schema dependency. The prior frozen
container extension is reverted separately; canonical packages work with
both filesystem and frozen imports without changing namespace semantics.
Assisted-by: dot:GPT-6.1
* Keep platform-dependent C long normalization explicit
Allow only the c_long-to-i32 conversion lint: Windows uses the same type,
while LP64 requires the existing truncation to match CPython normalization.
No runtime behavior or test assertions change.
Assisted-by: dot:GPT-6.1
* Include profiling notices in library and desktop packages
Bundle a byte-identical copy of the root profiling notice in rustpython-pylib
and include the root notice in configured desktop resources. Avoid a symlink
so Windows checkouts preserve the actual notice text. Terms are unchanged.
Assisted-by: dot:GPT-6.1
* Keep profiling asset license notice in pylib crate
* Serve frozen profiling assets through native resource readers
* Preserve canonical profiling asset line endings on checkout
---------
Co-authored-by: youknowdot <dot@youknowone.org>
221 lines
7.7 KiB
Python
Vendored
221 lines
7.7 KiB
Python
Vendored
"""Tracing profiler for Python.
|
|
|
|
This module provides deterministic profiling of Python programs by tracing
|
|
every function call and return.
|
|
"""
|
|
|
|
__all__ = ("run", "runctx", "Profile")
|
|
|
|
import _lsprof
|
|
import importlib.machinery
|
|
import importlib.util
|
|
import io
|
|
from profiling.tracing._utils import _Utils
|
|
|
|
# ____________________________________________________________
|
|
# Simple interface
|
|
|
|
def run(statement, filename=None, sort=-1):
|
|
"""Run statement under profiler optionally saving results in filename
|
|
|
|
This function takes a single argument that can be passed to the
|
|
"exec" statement, and an optional file name. In all cases this
|
|
routine attempts to "exec" its first argument and gather profiling
|
|
statistics from the execution. If no file name is present, then this
|
|
function automatically prints a simple profiling report, sorted by the
|
|
standard name string (file/line/function-name) that is presented in
|
|
each line.
|
|
"""
|
|
return _Utils(Profile).run(statement, filename, sort)
|
|
|
|
def runctx(statement, globals, locals, filename=None, sort=-1):
|
|
"""Run statement under profiler, supplying your own globals and locals,
|
|
optionally saving results in filename.
|
|
|
|
statement and filename have the same semantics as profile.run
|
|
"""
|
|
return _Utils(Profile).runctx(statement, globals, locals,
|
|
filename, sort)
|
|
|
|
# ____________________________________________________________
|
|
|
|
class Profile(_lsprof.Profiler):
|
|
"""Profile(timer=None, timeunit=None, subcalls=True, builtins=True)
|
|
|
|
Builds a profiler object using the specified timer function.
|
|
The default timer is a fast built-in one based on real time.
|
|
For custom timer functions returning integers, timeunit can
|
|
be a float specifying a scale (i.e. how long each integer unit
|
|
is, in seconds).
|
|
"""
|
|
|
|
# Most of the functionality is in the base class.
|
|
# This subclass only adds convenient and backward-compatible methods.
|
|
|
|
def print_stats(self, sort=-1):
|
|
import pstats
|
|
if not isinstance(sort, tuple):
|
|
sort = (sort,)
|
|
pstats.Stats(self).strip_dirs().sort_stats(*sort).print_stats()
|
|
|
|
def dump_stats(self, file):
|
|
import marshal
|
|
with open(file, 'wb') as f:
|
|
self.create_stats()
|
|
marshal.dump(self.stats, f)
|
|
|
|
def create_stats(self):
|
|
self.disable()
|
|
self.snapshot_stats()
|
|
|
|
def snapshot_stats(self):
|
|
entries = self.getstats()
|
|
self.stats = {}
|
|
callersdicts = {}
|
|
# call information
|
|
for entry in entries:
|
|
func = label(entry.code)
|
|
nc = entry.callcount # ncalls column of pstats (before '/')
|
|
cc = nc - entry.reccallcount # ncalls column of pstats (after '/')
|
|
tt = entry.inlinetime # tottime column of pstats
|
|
ct = entry.totaltime # cumtime column of pstats
|
|
callers = {}
|
|
callersdicts[id(entry.code)] = callers
|
|
self.stats[func] = cc, nc, tt, ct, callers
|
|
# subcall information
|
|
for entry in entries:
|
|
if entry.calls:
|
|
func = label(entry.code)
|
|
for subentry in entry.calls:
|
|
try:
|
|
callers = callersdicts[id(subentry.code)]
|
|
except KeyError:
|
|
continue
|
|
nc = subentry.callcount
|
|
cc = nc - subentry.reccallcount
|
|
tt = subentry.inlinetime
|
|
ct = subentry.totaltime
|
|
if func in callers:
|
|
prev = callers[func]
|
|
nc += prev[0]
|
|
cc += prev[1]
|
|
tt += prev[2]
|
|
ct += prev[3]
|
|
callers[func] = nc, cc, tt, ct
|
|
|
|
# The following two methods can be called by clients to use
|
|
# a profiler to profile a statement, given as a string.
|
|
|
|
def run(self, cmd):
|
|
import __main__
|
|
dict = __main__.__dict__
|
|
return self.runctx(cmd, dict, dict)
|
|
|
|
def runctx(self, cmd, globals, locals):
|
|
self.enable()
|
|
try:
|
|
exec(cmd, globals, locals)
|
|
finally:
|
|
self.disable()
|
|
return self
|
|
|
|
# This method is more useful to profile a single function call.
|
|
def runcall(self, func, /, *args, **kw):
|
|
self.enable()
|
|
try:
|
|
return func(*args, **kw)
|
|
finally:
|
|
self.disable()
|
|
|
|
def __enter__(self):
|
|
self.enable()
|
|
return self
|
|
|
|
def __exit__(self, *exc_info):
|
|
self.disable()
|
|
|
|
# ____________________________________________________________
|
|
|
|
def label(code):
|
|
if isinstance(code, str):
|
|
return ('~', 0, code) # built-in functions ('~' sorts at the end)
|
|
else:
|
|
return (code.co_filename, code.co_firstlineno, code.co_name)
|
|
|
|
# ____________________________________________________________
|
|
|
|
def main():
|
|
import os
|
|
import sys
|
|
import runpy
|
|
import pstats
|
|
from optparse import OptionParser
|
|
usage = "cProfile.py [-o output_file_path] [-s sort] [-m module | scriptfile] [arg] ..."
|
|
parser = OptionParser(usage=usage)
|
|
parser.allow_interspersed_args = False
|
|
parser.add_option('-o', '--outfile', dest="outfile",
|
|
help="Save stats to <outfile>", default=None)
|
|
parser.add_option('-s', '--sort', dest="sort",
|
|
help="Sort order when printing to stdout, based on pstats.Stats class",
|
|
default=2,
|
|
choices=sorted(pstats.Stats.sort_arg_dict_default))
|
|
parser.add_option('-m', dest="module", action="store_true",
|
|
help="Profile a library module", default=False)
|
|
|
|
if not sys.argv[1:]:
|
|
parser.print_usage()
|
|
sys.exit(2)
|
|
|
|
(options, args) = parser.parse_args()
|
|
sys.argv[:] = args
|
|
|
|
# The script that we're profiling may chdir, so capture the absolute path
|
|
# to the output file at startup.
|
|
if options.outfile is not None:
|
|
options.outfile = os.path.abspath(options.outfile)
|
|
|
|
if len(args) > 0:
|
|
if options.module:
|
|
code = "run_module(modname, run_name='__main__')"
|
|
globs = {
|
|
'run_module': runpy.run_module,
|
|
'modname': args[0]
|
|
}
|
|
else:
|
|
progname = args[0]
|
|
sys.path.insert(0, os.path.dirname(progname))
|
|
with io.open_code(progname) as fp:
|
|
code = compile(fp.read(), progname, 'exec', module='__main__')
|
|
spec = importlib.machinery.ModuleSpec(name='__main__', loader=None,
|
|
origin=progname)
|
|
module = importlib.util.module_from_spec(spec)
|
|
# Set __main__ so that importing __main__ in the profiled code will
|
|
# return the same namespace that the code is executing under.
|
|
sys.modules['__main__'] = module
|
|
# Ensure that we're using the same __dict__ instance as the module
|
|
# for the global variables so that updates to globals are reflected
|
|
# in the module's namespace.
|
|
globs = module.__dict__
|
|
globs.update({
|
|
# Set __spec__ to None so the profiled program behaves like a
|
|
# script run directly (gh-140729).
|
|
'__spec__': None,
|
|
'__file__': spec.origin,
|
|
'__name__': spec.name,
|
|
'__package__': None,
|
|
})
|
|
|
|
try:
|
|
runctx(code, globs, None, options.outfile, options.sort)
|
|
except BrokenPipeError as exc:
|
|
# Prevent "Exception ignored" during interpreter shutdown.
|
|
sys.stdout = None
|
|
sys.exit(exc.errno)
|
|
else:
|
|
parser.print_usage()
|
|
return parser
|
|
|
|
# When invoked as main program, invoke the profiler on a script
|
|
if __name__ == '__main__':
|
|
main()
|