Literal paths, `children_of` and `tagged` entries can be combined freely in
one `path` section, as on master. Sets are weighted by an estimate of their
total size: the calling thread's shard size scaled by the number of shards,
so a per-thread sharded dynamic set is not underweighted against an
unsharded one.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A `PathGetter` can now reference several `tagged` sets; paths are drawn
uniformly from their union by first picking a set with probability
proportional to its size and then a path within it. Sizes are read from a
new per-shard atomic `cached_size` (relaxed loads) instead of locking the
shard mutex, so the weights can lag slightly behind concurrent updates of
dynamic sets, which is acceptable.
`PathGetter::takePath` was unused and is removed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`keep_count: <n>` (or `auto`) on a create generator keeps its output path
set at a roughly constant size, so the dataset doesn't drift over a long
benchmark: each operation chooses between Create and Remove with a
probability based on the current shard size — 0.5 at the target,
approaching 0/1 as the size deviates (logistic curve, 5% of the target as
the scale). The last path in a shard is never removed, so a set can only
be empty if it started empty. `auto` snapshots the size at the end of
setup as the target. Mutually exclusive with `remove_factor`.
Sizes and targets of dynamic path sets are now printed with each periodic
report in both the generated and storage modes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Path sets that a `create` generator outputs to are now dynamic: created
nodes are added to the set on success and `remove_factor` removes take
nodes out of it (before sending; put back unless the outcome shows the
node is gone). The output set is the explicit `tag` from the create
generator's config, the `children_of` set of its fixed parent, or an
anonymous set when only `remove_factor` needs one; it is only populated
if some generator reads it. This allows pointing e.g. a `set` generator
at nodes produced by a `create` generator, and replaces the old
per-generator `paths_created` bookkeeping (along with the name
uniqueness enforcement, which never worked across threads anyway).
Dynamic sets are sharded by worker thread, so each thread reads and
updates only its own shard (shard mutexes still guard against response
callbacks running on client/commit threads). Since a dynamic set may lag
behind the real state, requests drawing paths from one report `ZNONODE`/
`ZNODEEXISTS` results as a separate `ignored_errors` counter instead of
failing, even without `continue_on_errors`. Generators whose input set
is currently empty decline, and another generator is picked instead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Each `PathGetter` now draws from exactly one `PathSet` — a literal path
list, one `tagged` reference, or one `children_of` reference (mixing
sources in one `path` is no longer supported). Sets are registered in
`NodesSetup` while parsing the setup and generator configs, so generators
referencing the same tag or parent share one in-memory copy of the paths
instead of each generator merging its own copy.
The initialization sequence is reordered accordingly: parse generators
(registering sets), create the setup tree (populating tag sets on the
fly), resolve `children_of` sets by listing each distinct parent once,
then validate. Tag sets nobody reads are not populated at all.
The setup tree now stores `repeat` counts unexpanded, mirroring the
config; instances are expanded during the creation traversal (root names
are recorded for cleanup). For repeated subtrees this removes the
per-instance child pointers from memory.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
