From 763d5d48b57ee13bbefe9b9306d6f5db8deb69ee Mon Sep 17 00:00:00 2001 From: Jack O'Connor Date: Thu, 28 Aug 2025 01:58:20 +0100 Subject: [PATCH] Add sorted.py to microbenchmarks (#6086) * Add microbenchmark for `sorted` I chose 5 * Iterations to try better show that RustPython sort implementation scales noticeably worse CPython's with respect to the number of elements. * Mention how to run a specific benchmark * Update python version in bench README 3.13 better reflects the current state of the project vs 3.7. --- benches/README.md | 26 +++++++++++++++++--------- benches/microbenchmarks/sorted.py | 9 +++++++++ 2 files changed, 26 insertions(+), 9 deletions(-) create mode 100644 benches/microbenchmarks/sorted.py diff --git a/benches/README.md b/benches/README.md index f6e512961..4823751fd 100644 --- a/benches/README.md +++ b/benches/README.md @@ -4,19 +4,26 @@ These are some files to determine performance of rustpython. ## Usage -Running `cargo bench` from the root of the repository will start the benchmarks. Once done there will be a graphical +Running `cargo bench` from the root of the repository will start the benchmarks. Once done there will be a graphical report under `target/criterion/report/index.html` that you can use use to view the results. -To view Python tracebacks during benchmarks, run `RUST_BACKTRACE=1 cargo bench`. You can also bench against a +`cargo bench` supports name matching to run a subset of the benchmarks. To +run only the sorted microbenchmark, you can run: + +```shell +cargo bench sorted +``` + +To view Python tracebacks during benchmarks, run `RUST_BACKTRACE=1 cargo bench`. You can also bench against a specific installed Python version by running: ```shell -$ PYTHON_SYS_EXECUTABLE=python3.7 cargo bench +PYTHON_SYS_EXECUTABLE=python3.13 cargo bench ``` ### Adding a benchmark -Simply adding a file to the `benchmarks/` directory will add it to the set of files benchmarked. Each file is tested +Simply adding a file to the `benchmarks/` directory will add it to the set of files benchmarked. Each file is tested in two ways: 1. The time to parse the file to AST @@ -24,8 +31,9 @@ in two ways: ### Adding a micro benchmark -Micro benchmarks are small snippets of code added under the `microbenchmarks/` directory. A microbenchmark file has +Micro benchmarks are small snippets of code added under the `microbenchmarks/` directory. A microbenchmark file has two sections: + 1. Optional setup code 2. The code to be benchmarked @@ -39,8 +47,8 @@ a_list = [1,2,3] len(a_list) ``` -Only `len(a_list)` will be timed. Setup or benchmarked code can optionally reference a variable called `ITERATIONS`. If -present then the benchmark code will be invoked 5 times with `ITERATIONS` set to a value between 100 and 1,000. For +Only `len(a_list)` will be timed. Setup or benchmarked code can optionally reference a variable called `ITERATIONS`. If +present then the benchmark code will be invoked 5 times with `ITERATIONS` set to a value between 100 and 1,000. For example: ```python @@ -49,7 +57,7 @@ obj = [i for i in range(ITERATIONS)] `ITERATIONS` can appear in both the setup code and the benchmark code. -## MacOS setup +## MacOS setup On MacOS you will need to add the following to a `.cargo/config` file: @@ -63,4 +71,4 @@ rustflags = [ ## Benchmark source -- https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/nbody-python3-2.html +- diff --git a/benches/microbenchmarks/sorted.py b/benches/microbenchmarks/sorted.py new file mode 100644 index 000000000..8e5e0beeb --- /dev/null +++ b/benches/microbenchmarks/sorted.py @@ -0,0 +1,9 @@ +from random import random, seed +seed(0) + +unsorted_list = [random() for _ in range(5 * ITERATIONS)] + +# --- + +# Setup code only runs once so do not modify in-place +sorted(unsorted_list)