`keep_count: <n>` (or `auto`) on a create generator keeps its output path
set at a roughly constant size, so the dataset doesn't drift over a long
benchmark: each operation chooses between Create and Remove with a
probability based on the current shard size — 0.5 at the target,
approaching 0/1 as the size deviates (logistic curve, 5% of the target as
the scale). The last path in a shard is never removed, so a set can only
be empty if it started empty. `auto` snapshots the size at the end of
setup as the target. Mutually exclusive with `remove_factor`.
Sizes and targets of dynamic path sets are now printed with each periodic
report in both the generated and storage modes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Path sets that a `create` generator outputs to are now dynamic: created
nodes are added to the set on success and `remove_factor` removes take
nodes out of it (before sending; put back unless the outcome shows the
node is gone). The output set is the explicit `tag` from the create
generator's config, the `children_of` set of its fixed parent, or an
anonymous set when only `remove_factor` needs one; it is only populated
if some generator reads it. This allows pointing e.g. a `set` generator
at nodes produced by a `create` generator, and replaces the old
per-generator `paths_created` bookkeeping (along with the name
uniqueness enforcement, which never worked across threads anyway).
Dynamic sets are sharded by worker thread, so each thread reads and
updates only its own shard (shard mutexes still guard against response
callbacks running on client/commit threads). Since a dynamic set may lag
behind the real state, requests drawing paths from one report `ZNONODE`/
`ZNODEEXISTS` results as a separate `ignored_errors` counter instead of
failing, even without `continue_on_errors`. Generators whose input set
is currently empty decline, and another generator is picked instead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Each `PathGetter` now draws from exactly one `PathSet` — a literal path
list, one `tagged` reference, or one `children_of` reference (mixing
sources in one `path` is no longer supported). Sets are registered in
`NodesSetup` while parsing the setup and generator configs, so generators
referencing the same tag or parent share one in-memory copy of the paths
instead of each generator merging its own copy.
The initialization sequence is reordered accordingly: parse generators
(registering sets), create the setup tree (populating tag sets on the
fly), resolve `children_of` sets by listing each distinct parent once,
then validate. Tag sets nobody reads are not populated at all.
The setup tree now stores `repeat` counts unexpanded, mirroring the
config; instances are expanded during the creation traversal (root names
are recorded for cleanup). For repeated subtrees this removes the
per-instance child pointers from memory.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Generators used to be constructed per worker thread, each re-parsing the
config and building its own copy of every path list (including the results
of `children_of` listings), multiplying memory usage by the concurrency.
Now a single `Generator` is built once on the main thread and shared; the
only per-call mutable state is the rng, passed in via `GenerateContext`
(`{rng, thread_idx}`) by each thread.
This also deletes the `setSeed`/`setSeedImpl` cascade with its per-component
seed offsets: with a fixed `generator.seed`, each thread's rng is simply
seeded with `seed + thread_idx`. `children_of` parents are now listed once
instead of once per thread, and the workload description is printed once.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- `GeneratedRunner` (generated mode) and `LogRunner` (replay and
setup-nodes-collection modes) replace the two-mode `Runner`.
- `--input-request-log` now always selects replay mode, taking precedence
over the config's `generator` section instead of being silently ignored.
- `BenchmarkContext` is renamed to `NodesSetup` and moved to its own file;
the setup tree walk now emits `Create` requests into a callback, which
`StorageRunner` reuses instead of a hand-rolled tree walk.
- Shared helpers are extracted to `RunnerCommon`: `getOption` (command
line > config > default), `ConnectionFactory`, `BenchmarkOutput`.
`StorageRunner` uses `getOption` too.
- Generators are initialized sequentially up front in `GeneratedRunner`,
removing `generator_init_barrier`.
- SIGINT is now blocked process-wide at program entry, so Ctrl+C is always
seen by the `InterruptListener` polls instead of sometimes terminating
the process via a global-thread-pool thread that didn't block it.
`StorageRunner` and `LogRunner` now also shut down gracefully on SIGINT
and print the final report.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Michael Kolupaev