mirror of
https://github.com/redis/go-redis.git
synced 2026-10-28 08:24:54 +08:00
master
100
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Nedyalko Dyakov
|
4fe689525a | test: read CLIENT INFO in the SETINFO Exec (#4211) | ||
|
Nedyalko Dyakov
|
470a926a08 |
test(cluster): skip unknown-endpoint spec before 8.4 (#4215)
* test(cluster): skip unknown-endpoint spec before 8.4 On Redis 8.0 and 8.2 the cluster nodes segfault in a module timer shortly after cluster-preferred-endpoint-type is set to unknown-endpoint. The spec restores the setting quickly, so it only fails when the timer fires inside that window, but then every later cluster spec hits a dead cluster. * test(cluster): guard unknown-endpoint teardown on skip |
||
|
Nedyalko Dyakov
|
c9092ec492 |
fix(auth): make empty-password AUTH opt-in (#4194)
* fix(auth): make empty-password AUTH opt-in #4161 made a Username with an empty Password send AUTH user "". Redis OSS accepts it (the nopass default user takes any password), but Redis Enterprise and Redis Cloud databases with unauthenticated access can reject it with WRONGPASS, so configs with a username and no password stopped connecting. Restore the earlier default (no AUTH without a password) and add Options.AuthEmptyPassword (auth_empty_password=true) to opt in, on all options types and credential providers at connect time. Public Hello again sends AUTH only with a password. The client-side cache namespace uses the ACL user the connection runs as. Refs #4161 * fix(auth): keep CSC namespace per username A Username that sends no AUTH runs as the default user, or as a TLS-certificate user. Mapping it to the default namespace let clients with different certificates share cache entries. It now gets its own namespace per Username ("noauth <name>", which no ACL user can match). Test: delete the ACL user before closing the admin client, so the user does not leak into the ACL specs. * fix(auth): split sentinel empty-password opt-in Sentinel and data-node credentials are independent, so one flag for both could force AUTH user "" onto Redis Enterprise data nodes to authenticate a nopass sentinel user. FailoverOptions and UniversalOptions gain SentinelAuthEmptyPassword (sentinel_auth_empty_password=true); AuthEmptyPassword now covers the master/replica connections only. |
||
|
Nedyalko Dyakov
|
454251f54b |
fix(autopipeline): reroute cluster FD recovery (#4111)
* fix(autopipeline): reroute cluster FD recovery A cluster full-duplex node child replayed a dropped session's tail only on a fresh connection to the same node, and failed its carry and queue once lease retries ran out. If topology change had closed or replaced that master, every accepted command on it failed instead of reaching the slot's new owner (codex P2 on #4002, listed as a known limitation). Add an internal owner seam, AutoPipelineOptions.fdReroute: an engine with an owner runs that work through the owner, in order, instead. - connection error: the replayable tail (same budget partition and sent-NoRetry split as the replay) goes through the owner; - lease retries spent: the carry, then the queue, go through the owner instead of failing. The cluster FD router sets it to the ClusterClient pipeline, which routes by slot against live topology. Standalone FD is unchanged. * fix(autopipeline): keep retry budget on FD reroute fdReroute now takes the remaining retry bound. rerouteReqs goes through flushCarryBudgeted, so each group passes its remaining budget and a command never runs more than budget+1 times; the cluster child uses cc.processPipelineRetries with that bound in place of MaxRedirects. - The Close-time flush (flushReqs) also goes through the owner, so a child closed by topology GC does not flush through its dropped node. - runReroute returns a per-command transport error even when the owner returns an earlier Redis error (redis.Nil), so the flush still aborts. - flushReqsVia reports per-command metrics before completing, as the reader does. This also adds them to the standalone Close-time flush. * test(autopipeline): fold reroute tests into FD tests * docs(autopipeline): note nested hooks on FD recovery |
||
|
Nedyalko Dyakov
|
f4461968fc |
test(pool): assert PoolSize soft cap in limit spec (#4180)
"race limit the number of connections" asserted IdleConns == 0 and TotalConns == PoolSize after 1000 concurrent Gets with MinIdleConns 50. A Get that misses while min-idle dials are in flight dials an excess non-pooled conn, and the late min-idle conns stay idle, so total and idle both overshoot by the same amount. PoolSize is a soft cap on pooled conns (see .claude/specs/pool.md), so this is allowed. Assert the contract instead: no Get errors, in-use conns == PoolSize, pooled conns <= PoolSize. Also close the spec's pool and nil-guard the shared AfterEach so the spec can run focused. |
||
|
Nedyalko Dyakov
|
c44dd63cc7 |
test(cmd): relax HPTTL TTL assertion (#4172)
HPTTL right after HEXPIRE asserted 10000ms +/- 2ms, which fails on slow CI runners (seen 9991). TTL only counts down, so assert 9000 < ttl <= 10000 instead. |
||
|
Nedyalko Dyakov
|
2821f8801f |
feat(pool): keep a warm spare for full-duplex (#4101)
* feat(pool): keep a warm spare for full-duplex The full-duplex engine holds one pipeline-pool connection for its whole session, and the pipeline pool never kept idle connections. After a connection error the replacement had to dial first, while every unacked command waited. Add Options.PipelineMinIdleConns (default 0, also on Cluster, Ring, Failover and Universal options and the URL parsers) to keep idle connections in the pipeline pool. A full-duplex engine raises it to at least 1 through a new ConnPool.RaiseMinIdleConns; the pool refills the spare in the background whenever it is leased. A negative value opts out. Clients that never use full-duplex keep a zero idle footprint. The spare saves the dial (TCP/TLS), not the connection init (HELLO/AUTH), which still runs on first lease. * fix(pool): keep the warm spare fresh The pool checks ConnMaxIdleTime only when it hands a connection out. Under LIFO the full-duplex spare is never leased, so it aged past ConnMaxIdleTime (30m by default) and was discarded by the very lease it exists for, which then dialed. In a pool whose min-idle floor was raised, Get/TryGet now replace the oldest idle connection once it is half-way to ConnMaxIdleTime, but only when another idle connection serves the lease. The close goes through the stale path, and the floor refills in the background. One atomic load for all other pools. Also fix the warm-spare test race seen on CI: IdleLen counts a min-idle connection as soon as its dial starts, so the waits returned while the spare was still dialing. Wait on Len as well. * fix(pool): add the FD spare only when it fits Add the warm spare only when the pipeline pool has room for it next to the engines: PipelinePoolSize must exceed the engine count, or the session spills to the main pool, and MaxIdleConns (if set) must too, or every recycle evicts the returned session connection and dials again. Also refresh the spare before its ConnMaxLifetime ends, not only before ConnMaxIdleTime; re-check a raise that races an early-return min-idle check; and fix the pipeline pool docs (permanent raise, the FullDuplexMaxHold bound, cluster pool lifecycle, pipelinePoolOptions). * fix(pool): use fdPipeCount after master merge * fix(pool): refresh the spare after the lease Closing the aged spare before popIdle could leave this Get with no connection (a concurrent Get took the other idle one, or the newest one failed the health check) and force a dial. * test(pool): fold warm-spare tests into existing files * docs(pool): note lifetime jitter for the FD spare |
||
|
Nedyalko Dyakov
|
dc464bf45c |
fix(lint): mark discarded done receive for errcheck (#4115)
golangci-lint v2.15.0 reports the bare receive from the chan error req.done in cscMissCoalescer.fetch. The value is intentionally dropped: the reader's result is in req.servedBy and ctx.Err() is returned. |
||
|
Nedyalko Dyakov
|
16a7904db0 |
feat(autopipeline): add FullDuplexPipes option (#4107)
NumShards meant two different things: the number of half-duplex queue shards (raising it usually hurts and needs Unordered on the async face), and, with FullDuplex, the number of held full-duplex connections (raising it is the only way past one connection's ceiling). FullDuplexPipes now names the full-duplex pipe count. NumShards > 1 is still honored under FullDuplex when FullDuplexPipes is 0, so 9.23.0 configs keep working; Validate rejects the two set to different values. Unlike NumShards, FullDuplexPipes has no effect when full duplex cannot engage, so it never turns into unordered half-duplex shards. Cluster full duplex keeps one pipe per master. |
||
|
Nedyalko Dyakov
|
b57341e5a0 |
feat(otel): MultiDB metrics in redisotel-native (#4007)
The MultiDB client (#3954) reports failovers, active database changes, member circuit breaker transitions and health checks through an optional capability interface on the OTel recorder. The native recorder did not implement it, so applications using the repository's standard instrumentation received none of those events. Add a "multidb" metric group (part of MetricGroupAll) with six instruments in the module's redis.client.* naming: failovers (counter) and failover duration (histogram), active database changes (counter), circuit breaker state changes (counter), health checks (counter, by outcome) and health check duration (histogram). Durations share the operation duration buckets. Databases are identified by the host-only FQDN the client reports; there is one per member, so it is recorded as an attribute without a cardinality switch. The recorder methods match the capability interface's signatures; the tests pin the method set through a local mirror of that interface until it reaches master with #3954, when the mirror can become a direct assertion. |
||
|
Nedyalko Dyakov
|
7a7bf2107f | test(autopipeline): consolidate test files (#4091) | ||
|
Nedyalko Dyakov
|
f79ab09081 |
feat(search): add FILTER and LIMIT aggregate steps (#4076)
* feat(search): add FILTER and LIMIT aggregate steps FTAggregateStep now supports Filter and Limit, so FILTER and LIMIT can be placed anywhere in the FT.AGGREGATE pipeline, e.g. FILTER before and after GROUPBY. AggregateBuilder.Filter now adds a FILTER at its call position instead of setting the tail Filter field, and AggregateBuilder.Limit is added. The tail Filter and LimitOffset/Limit fields are deprecated. Their wire output is unchanged: they are still sent after Steps, LIMIT then FILTER. * docs(search): document one SORTBY per GROUPBY segment The server allows only one SORTBY between two GROUPBY steps and returns "Multiple SORTBY steps are not allowed" for a second one. The builder SortBy doc said a SortBy after another step starts a new SORTBY step, without saying that the server rejects it. State the rule on SortBy, FTAggregateSortByStep and FTAggregateStep. * docs(search): leave the one-SORTBY rule to the server go-redis does not check that only one SORTBY appears between two GROUPBY steps. Say so in the docs, and add a test that a second SORTBY is sent as built, so the client keeps working if the server relaxes the rule. |
||
|
Nedyalko Dyakov
|
6f2b2639e5 |
chore(release): v9.23.0 (#4074)
* chore(release): v9.23.0 * docs(release): scope FD Pipeline, MaxQueuedCommands, LRUClock |
||
|
Nedyalko Dyakov
|
e3d2548a38 |
fix(pool): fix ConnStateMachine waiter races (#4072)
notifyWaiters removed a waiter before its CAS and, on failure, pushed it back as a new list element. AwaitAndTransition still held the old one, so on timeout Remove was a no-op while waiterCount was decremented: a dead waiter stayed queued and a later notify moved the conn to its target state (e.g. UNUSABLE) for nobody. Now the waiter is removed only after a successful CAS. When ctx was canceled while the waiter was being served, the caller got ctx.Err() although the transition happened, and waiterCount was decremented twice. The timeout path now checks w.done under sm.mu and returns the served result. Refs #4028 |
||
|
Nedyalko Dyakov
|
c2002af06c |
docs(example): add full-duplex examples (#4061)
* docs(example): add full-duplex examples Three runnable examples next to example/autopipeline: the blocking face on one held connection with pipelines on it, the async face with submission windows and Submit, and NumShards > 1 with per-key routing, pool sizing and the pooled fallback for pipelines that span engines. * docs(example): fold FD engines, show queue limit The multi-engine example used the async face too, so the two repeated each other. autopipeline-fullduplex-async now runs one engine, then NumShards 4 (pool sizing, routing by first key, what is not ordered, pipelines that span engines), then compares one engine with four at low and high load. The half-duplex tour shows MaxQueuedCommands: a burst over the cap is rejected with ErrAutoPipelineQueueFull and retried after a backoff. The full-duplex READMEs say why it limits only diverted commands there: a full FullDuplexWindow blocks the submitter instead of rejecting. |
||
|
Nedyalko Dyakov
|
b5f8864de6 |
feat(autopipeline): add MaxQueuedCommands hard limit (#4070)
* feat(autopipeline): add MaxQueuedCommands hard limit Half-duplex autopipelining had no bound on accepted commands, so a slow server and fast producers could grow client memory until OOM. MaxQueuedCommands > 0 caps commands accepted but not yet completed (queued, waiting for a batch permit, executing, and diverted commands on the deferred face). A command at the limit fails at once with ErrAutoPipelineQueueFull instead of being queued. Slots are released before the batch completes, on every completion path (batch, solo, permit timeout, shutdown drain, diverted, panic). Default 0 keeps the current unbounded behavior. Rejected with FullDuplex, which already bounds outstanding commands with FullDuplexWindow. * fix(autopipeline): tighten MaxQueuedCommands accounting Count commands run outside the pipeline on the blocking face too: each holds a pooled connection, so leaving them out broke the hard bound. A rejected submit now consumes an announced arrival only while the counter is positive. A caller retrying rejections in a loop drove expectedArrivals far negative, and the next announced wave vanished into the deficit, splitting flushes into shallow batches. * fix(autopipeline): reject full queue before sizing Check MaxQueuedCommands before MaxBatchBytes sizing, so a submit to a full queue fails without running user Args/MarshalBinary code. The atomic admission after sizing still makes the final decision. When a rejection consumes the last expected arrival, wake the shards. A flusher waiting for that wave only re-checks the count on a wake, so it otherwise waited out the silence fallback. * fix(autopipeline): allow MaxQueuedCommands with FD With FullDuplex, Do and blocking or connection-hostile commands still run outside the pipeline, one goroutine each on the deferred face, and FullDuplexWindow does not bound them. Rejecting the combination left FD users no way to cap them. Allow it: the limit then covers the commands run outside the pipeline, and the FD stream keeps its window. Also check the queue before cluster slot routing, not only before sizing: shardFn reads and stringifies key arguments (user code). * fix(autopipeline): release slots after setting errors On the permit-backstop path, release a batch's MaxQueuedCommands slots only after every drained command has its error, and before the closes wake the waiters. Releasing first let producers be admitted while the failing batch was still outstanding. |
||
|
Nedyalko Dyakov
|
f813adcb85 |
feat(autopipeline): run N full-duplex engines on one client (#4022)
* feat(autopipeline): spike N full-duplex engines per client
SPIKE, not for merge. Answers whether the ~2x measured for
"multi-connection full duplex" is a property of N engines or an artifact
of the benchmark harness, which fakes multi-conn by building N separate
*redis.Client instances and round-robining across them. That holds six
sockets per client to do I/O on one -- 48 for eight wires -- and keeps
the routing outside the library, so key routing could not be tested.
NumShards>1 under FullDuplex now means N engines on ONE client: one pool,
one hook chain, one invalidation stream, N held wires. Commands route by
key hash, so same-key commands keep one wire and per-key order holds
without an Unordered opt-in. Keyless commands round-robin. A whole
FDPipelined batch pins to one engine, since splitting it would lose the
internal order a pipeline exists to guarantee.
The change is TWO-SITED, which is the finding. Relaxing the Validate
rejection alone silently turns full duplex OFF: fdOn is separately gated
on nShards == 1, so NumShards>1 fell through to half-duplex round-robin
shards, which are unordered, and a same-key write-then-read returned
redis.Nil. That is exactly what the deleted error warned about ("reject
it rather than silently falling back to half-duplex"). Both the
validation and the construction gate have to move together, with
nShards forced back to 1 so the shard array stays flusherless.
Measured at 1024 callers, 10 commands per Exec, one binary:
proxy, 8 clients 1,686,080 ops/s 48 held sockets
in-client, 8 1,791,369 ops/s 12 held sockets
Known gap before this could ship: engines lease from the pipeline pool
and DefaultPipelinePoolSize is 10, so more than ~10 engines spill to the
main pool. A real feature must size that pool from the engine count.
cmdFirstKey is also deliberately crude (args[1]) rather than using
declared key positions.
* refactor(autopipeline): document FD sharding, guard the pool
Drops the exploratory framing from the multi-connection full-duplex code
and closes two of its three known gaps.
Key extraction no longer assumes args[1]. It calls
cmdFirstKeyPosWithInfo(cmd, nil), the same helper client-side caching
uses, so keyless commands, the static command table, and eval/evalsha
variants whose key position depends on the runtime numkeys argument all
route correctly instead of hashing a non-key argument.
Engine count is now checked against the pipeline pool. Each engine holds
one pooled connection for as long as it runs, and DefaultPipelinePoolSize
is 10, so 11 or more engines left the surplus permanently spilling onto
the main pool and competing with ordinary commands. Construction fails
with a message naming PipelinePoolSize instead.
The two-sited rule is now stated at both sites: relaxing the Validate
rejection without relaxing the fdOn gate leaves full duplex silently OFF
for NumShards>1, so commands fall through to unordered half-duplex shards
and a same-key write-then-read can read a stale value. Tests cover it
from both directions -- reply correlation and per-key ordering across
eight engines, and Config() reporting FullDuplex true rather than having
fallen back.
Still open, and why this stays draft: NumShards:0 means one engine and
sizing is not automatic. A fixed count above 1 costs low-concurrency
callers about 30% (eight engines at eight callers), so the count should
be derived from observed load, roughly one wire per 256-512 callers.
* fix(autopipeline): fix FD engine routing edges
- The NumShards>1 ordering check exempted any requested FullDuplex.
Without a pipeline pool full duplex does not engage and construction
falls back to round-robin shards, which need Unordered. The exemption
now follows the effective state.
- Keys are read with stringArg, as Ring does, so []byte, *string and
numeric keys route like the string form, and an empty key routes by
hash instead of round-robin.
- Engine index uses uint32 modulo; converting to int first goes
negative on 32-bit platforms.
- Document that N engines keep order per first key only, so multi-key
commands are not ordered across engines, and that NumShards>1 under
full duplex is new, so the mode is opt-in.
* docs(autopipeline): FD engine ordering and window
Document that with NumShards > 1 keyless commands, including ones that
change every key (FLUSHDB, SWAPDB), round-robin and are not ordered
against keyed commands; that keys route through stringArg like Ring and
the cluster slot hash; and that FullDuplexWindow applies per engine.
* docs(autopipeline): pool check is per autopipeliner
The NumShards pipeline-pool check counts only the autopipeliner being
built. The blocking and async faces and WithTimeout clones share one
pipeline pool, so it must hold the sum of their engines. Say so; a
client-wide engine count is a follow-up.
* fix(command): route raw FCALL by its first key
FCALL has the EVAL layout (name, function, numkeys, keys...), but the key
resolver only special-cased EVAL. The typed FCall sets its key position;
a raw Do/Process FCALL fell back to position 1 and hashed the function
name. That misrouted it on Ring (wrong shard), on cluster (a MOVED away)
and across full-duplex engines (a following GET of its key could run
first). A zero-key FCALL now resolves as keyless, like a zero-key EVAL.
* docs(autopipeline): document multi-engine NumShards
The exported FullDuplex and NumShards docs still described a single
full-duplex stream and said NumShards > 1 needs Unordered. Describe the
multi-engine mode: engines routed by first key, per-first-key order
only, per-engine window, and the pipeline pool sizing.
* fix(autopipeline): keep FD batches on their keys' engine
With several full-duplex engines, FDPipelined sent the whole batch to
its first command's engine. A later command in the batch whose key
hashes elsewhere could then run ahead of an earlier, unawaited command
for that key on that key's engine.
A batch now rides an engine only when every keyed command hashes to it
(keyless commands ride along). Otherwise FDPipelined returns
ErrFDPipelineSpansEngines and Pipeline() runs an ordinary pipeline.
* docs(autopipeline): note key types in engine routing
With several full-duplex engines a command is routed by a hash of its
first key in string form (fmt for non-string arguments, as Ring does),
so the same Redis key passed as different Go types (true and "1") can
land on different engines. Say so on NumShards.
* test(autopipeline): use the configured test address
The full-duplex shard tests dialed 127.0.0.1:6379 directly and skipped
under the REDIS_PORT override. Use apTestAddr and internalTestRedisAddr
like the other autopipeline tests.
* fix(command): route raw XGROUP and ZUNION by key
The raw-key resolver fell back to args[1] for XGROUP, whose key follows
the subcommand, and for ZUNION, ZINTER and ZDIFF, whose key follows
numkeys. With several full-duplex engines a raw command could then run
on a different engine from earlier writes to the same key, and Ring
picked the wrong shard. The typed constructors already set position 2.
* fix(command): route raw keyed commands by key
The raw-key resolver fell back to args[1] for SDIFFCARD, SUNIONCARD,
TS.NRANGE and TS.NREVRANGE (key after numkeys) and HIMPORT SET (key
after the subcommand), where the typed constructors set position 2.
HSCAN, SSCAN and ZSCAN with a hash tag in MATCH routed by the pattern,
a hint meant for the keyless SCAN. The key decides the node: a cluster
scan took a MOVED hop, a Ring scan read the wrong shard, and a
multi-engine full-duplex scan could pass earlier writes to its key.
* docs(autopipeline): note the cost of FD engines
NumShards is a fixed engine count, and several engines fragment small
batches. Give the measured trade on NumShards: 8 engines were 30%
slower than 1 at 8 callers, broke even at about 128, and were 2.2x
faster at 1024.
|
||
|
Nedyalko DyakovandVladislav Kotsev
|
349bcf47a9 |
feat(autopipeline): run Pipeline on the full-duplex connection (#4021)
* feat(autopipeline): add fdQueue.pushBatch for FD batches
FDPipelined submits a pipeline as one contiguous run; pushBatch enqueues
it under a single lock, all or nothing.
* feat(autopipeline): run Pipeline on the full-duplex connection
ap.Pipeline() delegated to the client's pipeliner, so a pipeline built
from a full-duplex autopipeliner quietly left the held connection, took a
pooled one per Exec, and lost the cross-caller coalescing the engine
exists for. It now submits the batch to the engine as one contiguous run.
FDPipelined does the submission: the batch is admitted all-or-nothing, so
its commands occupy consecutive in-flight slots and their replies come
back in submit order, which keeps a pipeline's internal ordering exactly
as a dedicated connection would. Batches that cannot ride the wire -- a
blocking command, a per-command read timeout, anything the divert
predicate claims, or a batch larger than the queue -- fall back to the
ordinary pipeline path.
Measured on a 14-core client against a local server, 10 commands per
Exec, 3 passes:
callers pooled on the engine active sockets
256 586,557 871,469 149 -> 1
1024 516,042 785,191 150 -> 1
So +48.6% and +52.2%, on one socket instead of 149.
Above ~2048 callers a single engine saturates its submit queue and
admission serializes; spreading pipelines across several auto-pipeliners
avoids that (1.6M ops/s across eight), which is an argument for a
supported multi-connection full-duplex mode rather than for this commit.
TxPipeline is deliberately untouched: MULTI/EXEC needs connection
affinity for the whole transaction, which the engine does not offer.
* fix(autopipeline): keep FD pipeline Exec semantics
Pipeline() on a full-duplex autopipeliner runs the batch on the held
connection. Three things differed from an ordinary pipeline:
- A retryable reply (LOADING and the like) was retried per command off
the reader, so later commands of the pipeline could run before it.
Pipelined commands now settle retryable replies inline, and a
retryable reply on the first command re-runs the whole batch the
ordinary way, as processPipeline does.
- With ContextTimeoutEnabled the caller's deadline did not bound the
wait. Such clients keep the pooled pipeline path.
- Exec dropped redis.Nil. It now returns the first error, Nil included.
FDPipelined is documented as standalone only.
* fix(autopipeline): close FD pipeline Exec gaps
Pipeline() on a full-duplex autopipeliner still differed from an
ordinary pipeline:
- Hooks, a Limiter or an OTel recorder: an ordinary pipeline runs the
ProcessPipelineHook chain, takes one Limiter decision per batch and
records one pipeline duration. The FD path did none of that, so such
clients now keep the pooled pipeline. Running the FD path under these
is a follow-up.
- The whole-batch retry on a retryable first reply re-ran pipelines
holding a NoRetry command. It now stops on cmdsContainNoRetry, as
generalProcessPipeline does.
- That retry also restarted the full MaxRetries budget, so MaxRetries:1
allowed three executions. The FD attempt now counts as the first: the
re-run gets MaxRetries-1 retries, after one backoff.
- A canceled ctx was only checked while waiting for queue room, so a
pre-canceled Exec still ran. submitBatch now refuses a done ctx before
admission.
* fix(autopipeline): count FD replays in the retry budget
The whole-batch retry assumed the FD engine issued the pipeline once.
After a connection-error replay it had issued it twice, so a lost first
write, LOADING on the replay and the pooled re-run made three executions
under MaxRetries:1, where an ordinary pipeline stops at two.
The reader now stamps each pipelined command's attempt count on its
batch before completing it, fdPipelined reports the highest, and the
re-run gets MaxRetries minus that, or none when it is spent.
* fix(autopipeline): keep FD pipelines off cluster children
A cluster node's FD child is the node client's cached async
autopipeliner, so ForEachMaster hands it to users, and its engine is
redirect-aware: it follows MOVED/ASK by re-running one command off the
pipe. A Pipeline() on it would let the rest of the batch run ahead of a
redirected command. FDPipelined now refuses redirect-aware engines, so
such a pipeline keeps the node client's ordinary pipeline, as before.
* fix(autopipeline): keep NoRetry batches off the FD pipe
An ordinary pipeline holding a NoRetry command retries nothing after a
connection error. The FD tail recovery replays per command, up to the
first sent NoRetry one, so a pipeline like SET k; RawWriteTo(GET x)
that lost its connection ran the SET twice. FDPipelined now refuses a
batch with a NoRetry command, and Pipeline() runs it the ordinary way.
The pipeline retry budget read fdAttempts, which only the reader
stamped. A command that was replayed and then failed carried no count,
so a pooled re-run could exceed MaxRetries. The batch is stamped at
submit and on every replay.
* fix(autopipeline): run pipeline hooks and metrics on FD
Pipeline() and FDPipelined kept away from the full-duplex path whenever
hooks, a Limiter or an OTel recorder were set, or bypassed them. They
now run on it:
- The batch runs inside the client's ProcessPipelineHook chain, with
fdPipelineExec as its terminal where processPipeline sits in an
ordinary pipeline. Fallback and the whole-batch retry happen inside
the chain, so hooks run once. Batched commands get no per-command
ProcessHook, as in an ordinary pipeline.
- An FD batch records one pipeline duration and, on failure, one
pipeline-level error; the reader skips its per-command metrics for
batched commands.
- A Limiter stays asked per write chunk, which can partly admit a batch
longer than MaxBatchSize. Documented on FDPipelined.
Only ContextTimeoutEnabled still keeps the pooled pipeline.
* fix(autopipeline): no FD pipeline re-run after a replay
The whole-batch retry re-ran a pipeline whose first command got a
retryable reply even when the engine had already replayed its unread
tail after a connection error, so a command in that tail could run
three times where an ordinary pipeline runs it twice. It now re-runs
only a batch the engine issued once. The Close-time flush stamps the
batch as issued again, so a flushed batch is not re-run either.
The re-run continues the FD attempt's measurement (start and attempts)
through processPipelineRetriesAfter, so one pipeline metric covers the
whole operation, and a ctx that ends in the backoff still records it.
* fix(autopipeline): keep flushed FD pipelines whole
When Close wins against an admitted FD pipeline, shutdownFlush sends
its commands through flushReqs. That split them at MaxBatchSize and at
attempt boundaries, so a retryable first reply retried only the first
chunk. Every batch of one FD pipeline now points at its first batch
(fdGroup), and the flush keeps that run in one chunk, so the pooled
retry re-runs the pipeline as a whole.
The pooled flush already records the pipeline metric. fdPipelined now
reports that the flush ran the batch, and FDPipelined/fdPipelineExec
skip their own metric in that case. failQueue skips the per-command
error for pipelined commands, as failReqs does, since fdPipelineExec
reports the batch once.
* fix(autopipeline): FD batch errors and queue wait
An FD pipeline now returns a transport or protocol failure before any
Redis error, as processPipeline does, so a caller that checks only
Exec's error sees that later commands have no known result.
The whole-batch re-run for a retryable first reply is also ruled out
when any command in the batch failed that way. The engine does not
replay such a command, since it may already have run, so the re-run
could execute it twice; an ordinary pipeline does not retry there.
A batch waiting for more room than one writer take freed put the room
signal back, took it straight back, failed to fit, and spun holding
submitMu.RLock. It now passes a wake it cannot use on once per take,
which keeps the wake chain for smaller waiters without spinning.
* fix(autopipeline): FD pipeline metrics and backoff
The pooled re-run after an FD attempt restarted its retry backoff at the
first step. generalProcessPipelineFrom now continues it from the prior
attempts, so a pipeline sleeps the same steps an ordinary one does.
A pipeline refused before admission (ctx done, or closed) recorded no
pipeline metric; an ordinary pipeline records one when withPipelineConn
fails. It now records one with a single attempt.
The Close flush marked every batch flushed before running any, so a
batch it never ran (a dead endpoint fails the rest) recorded nothing.
It now marks a batch where its chunk runs. And the FD metric is skipped
only when the flush ran the whole pipeline: when it ran only an unread
tail, the FD metric still covers the whole operation.
* fix(autopipeline): skip FD duration with no start
The start time is read only when a pipeline duration callback exists.
A recorder installed while an FD pipeline ran found a zero start, and
fdPipelineMetrics reported time.Since(time.Time{}), a duration of
centuries. It now records no duration for that operation, as an
ordinary pipeline does, which reads the callback once at the start.
Also pin that CSC state commands (SELECT, AUTH, HELLO with a protocol,
RESET, CLIENT TRACKING) never ride the FD pipe: they leave it as
runsOutsidePipeline commands, so Pipeline() falls back to the ordinary
pipeline and its CSC guard, and FDPipelined refuses the batch.
* fix(autopipeline): Close flush budget and retry timing
An FD pipeline flushed on Close can carry commands with different
attempt counts, and runs as one pipeline. It took the retry budget of
its first command, so a later command with more attempts could run past
its MaxRetries+1 executions. It now gets the smallest budget of its
commands, and fails with errFDRetryBudgetExhausted when any has none
left. A command with more budget may get fewer retries than it alone
would, as in an ordinary pipeline, whose budget covers the whole batch.
The pooled re-run of an FD operation that began with no duration
callback (zero start) started its clock at the re-run when a recorder
was installed in between, so the duration covered only the tail while
the count included the FD attempt. generalProcessPipelineFrom now
records no duration for a continued operation with no start.
* fix(autopipeline): keep flush budget groups separate
flushCarryBudgeted grew a group while the next command had the same
attempt count or belonged to the same FD pipeline, so a command in front
of a pipeline joined that pipeline's budget group. The group's smallest
budget then applied to it, and a pipeline command with no budget left
failed the unrelated command too. A group is now one FD pipeline, or a
run of other commands with the same attempt count, never both.
* fix(autopipeline): stamp flush issues where they run
shutdownFlush stamped every queued pipelined batch with one more issue
before either flush ran. When the carry flush hit a dead endpoint the
fresh queue was only failed, never issued, yet its callers saw an issue
count of 2 and reported a retry that never happened. flushReqs now
stamps the issue, with fdFlushed, where it runs a chunk. A batch that
never runs keeps its count; the transport error it is failed with
already rules out a re-run in fdPipelineExec.
* fix(autopipeline): flush a split FD pipeline whole
When the writer took only a prefix of an FD pipeline before the session
failed, Close found the prefix in the carry and the tail still queued,
and flushed them as two pooled pipelines. A retryable first reply then
retried only the prefix, and the tail ran under a full budget of its
own. shutdownFlush now moves the queued tail into the carry, so the
pipeline runs as one at the smallest remaining budget of its commands.
* docs(autopipeline): note lease retries in FD metrics
A full-duplex pipeline's attempts count the times the batch was issued.
Connection-lease retries belong to the engine: a batch queued during
them waits and is issued once, so they are not counted, where an
ordinary pipeline counts each failed connection acquisition. Say so on
fdPipelineMetrics.
* fix(autopipeline): reserve queue room for FD batches (#4057)
* fix(autopipeline): reserve queue room for FD batches
* fix(autopipeline): FD batch head-of-line starvation
* fix(autopipeline): remove unused function
* fix(autopipeline): fail a split FD pipeline whole
Live connection-error recovery partitions the unacked tail per command
(budget spent, after a sent NoRetry, not replayable) and fails part of
it. When the writer had taken only a prefix of an FD pipeline, the rest
of that pipeline, kept for replay or still queued, ran on the next
connection, and Exec reported a transport error for a pipeline whose
later commands executed. A pipeline on a dedicated connection sends
nothing more once it has failed. failPipelines now fails the whole
pipeline: its members kept for replay and its tail at the queue head.
---------
Co-authored-by: Vladislav Kotsev <vladygk8@gmail.com>
|
||
|
Nedyalko Dyakov
|
8546b33468 |
perf(autopipeline)!: batch full-duplex reply reads and use a queue instead of chan (#4018)
* perf(autopipeline): batch full-duplex reply reads The full-duplex reader used one WithReader per reply, which re-armed the socket read deadline on every reply: 5.85% of all CPU in SetReadDeadline plus 1.95% in deadline() at ~400k replies/s. Read one WithReader per already-buffered GROUP of replies instead. Reading the whole in-flight snapshot in a single WithReader was tried before and was slower, because it blocks on commands the writer has pushed but not yet flushed. This cannot do that: after the first reply of a group it continues only while rd.Buffered() > 0, so it never waits on the socket inside a group and writer/reader overlap is preserved. Refs #4014 * perf(autopipeline): replace the FD submit channel with a slice queue The full-duplex submit path was a buffered channel. Every command cost the submitter a channel send and woke the writer once, so the writer re-entered its select roughly once per command: that single select measured 3.25 s, 6.9% of all CPU, at ~450k ops/s, with selectgo at 7.4% and procyield at 4.5%. fdQueue is a mutex plus a slice. The writer takes a whole wave in ONE lock and one bulk copy, and park(minDepth) expresses "wake me when the queue holds N more, or not at all" -- something a channel cannot say, since a channel can only wake on the next arrival. selectgo and procyield leave the profile entirely, and total samples fell 47.0 s -> 45.6 s while throughput rose. Measured against stock 9.23 full-duplex, both WITHOUT a flush delay, so this isolates the submit path and the batched reads from the accumulation feature (2 vCPU client, 64B GET/SET 70/30, inflight 1, one wire, 3 passes, medians): callers stock queue delta 128 199,600 216,000 +8.2% 256 243,454 287,049 +17.9% 512 323,669 387,301 +19.7% 1,024 366,850 443,017 +20.8% 2,048 360,880 432,862 +19.9% Comparing WITH a delay would overstate this commit: stock 9.23 does not reference MaxFlushDelay in the full-duplex path at all, so a delayed stock arm is the same configuration as an undelayed one, and the difference would also contain everything #4014 adds. With the accumulation on top the total against stock is roughly +40% at 256 callers and above, of which about half is this commit and about half is the accumulation. This commit also makes MaxFlushDelay a true CEILING rather than a fixed pause: the writer arms a short grace (fdAccumGrace) and re-arms it only while the batch is still growing, so a wave that keeps arriving rides to the cap while a wave that stops flushes after one grace. That belongs here rather than in #4014 because it only pays WITH the queue -- ported onto the channel submit path it measured neutral to -3.2% (0/3 paired passes won at 1024 callers), since re-arming per arrival costs a select re-entry that the queue's per-wave take amortises. The queue methods all tolerate a nil receiver, reproducing nil-channel semantics, because engines built without one (tests, and any future construction path) previously relied on a nil channel blocking forever. Refs #4014 * perf(autopipeline): lower the FD accumulation gate for the queue path The in-flight threshold that decides whether waiting MaxFlushDelay is worthwhile depends on the SUBMIT MECHANISM, not only on the workload, so the queue path wants a different value from the channel path it replaces. Measured as a one-line diff on each path, 2 vCPU client, 64B GET/SET 70/30, inflight 1, one wire, 3 paired passes per width (gate 128 against gate 512): callers channel submit slice queue 128 -11.1% (0/3) +1.3% (2/3) 256 +5.2% (3/3) +21.8% (3/3) 512 +1.7% (2/3) +15.7% (3/3) 1,024 -0.5% (1/3) +1.7% (2/3) 2,048 +1.5% (3/3) -1.4% (1/3) On the channel path the lower gate costs 11% at 128 callers because waiting there wakes the writer once per arriving command; #4014 therefore keeps window>>7. With the queue the writer takes a whole wave per lock, so the same gate is free at 128 callers and worth +21.8% at 256 and +15.7% at 512, winning every paired pass, with p50 improving too (851 -> 716us at 256, 1215 -> 1049us at 512). Refs #4014 * refactor(autopipeline): drop FullDuplexFastSubmit The submit channel this option existed for is gone. FullDuplexFastSubmit skipped a blocking three-arm select on every submit, trading submit fairness for the selectgo cost. The slice queue replaced that channel with a mutex plus an append, which has no wait to skip, so the option became a no-op that only bumped a counter -- a public performance knob that silently stops meaning anything. Measured on a 2 vCPU client against managed Redis (RTT ~206us), 7 paired passes, 128-2048 concurrent callers. While the channel path is in place the option does fire on essentially every submit (2.9-5.4M fast-path takes per cell, 0 with it off) and it does pay: +3.95% throughput at 512 callers and +4.85% at 1024, both winning 7 of 7 passes, and -2.7% to -5.4% CPU per op winning 6-7 of 7 at every width. Its documented fairness cost did not appear, because the writer keeps the channel shallow enough that the depth gate never closes and no producer is left blocked to starve. The queue supersedes all of it: +12.6% to +37.5% over the same baseline, 7 of 7 at every width, so the option recovered only 1-24% of what replaces it. On the queue path there is nothing left for it to take. Only ever shipped in v9.23.0-beta.1; absent from v9.22.0, so removing it before v9.23.0 is not a break for any stable consumer. TestFullDuplexFastSubmitCloseRace kept its value as a submit/Close race test, so it is renamed TestFullDuplexSubmitCloseRace and keeps racing pushes against the shutdown drain; only the flag and the take-count assertion are gone. The 9.23.0-beta.1 release notes still list the option and stay that way: that beta did ship it, and older entries are left untouched. The removal belongs in the 9.23.0 notes, written when that release is cut. Refs #4018 * perf(autopipeline): shrink fdReq by storing the write time as an offset fdReq is copied at least three times per command -- into the submit queue, out of it with the wave, and into the in-flight ring -- and runtime.duffcopy measured 4.5% of all CPU with 46% of it attributed to those copies. time.Time was the largest field at 24 bytes of 96. Store nanoseconds since an engine-lifetime epoch instead: fdReq drops to 80 bytes, a 16.7% reduction on every copy. The offset is taken with now.Sub(epoch) against an epoch captured once when the engine is built, so it inherits the monotonic reading that time.Time carries and the OTel duration metric stays immune to a wall-clock step mid-command. Storing a UnixNano() would also be 8 bytes but would lose exactly that. The epoch must be a real reading rather than the zero Time: Sub against the zero Time saturates (the wall clock is ~2000 years past it, well beyond a Duration's ~292-year range) and time.Since would fall back to the wall clock. Zero still means "never written", floored at 1ns so a command written within a nanosecond of the epoch is not mistaken for one. writtenTime() reconstructs the absolute time for the retry path, which reports it to the normal pipeline. * perf(autopipeline): bound the FD submit queue by the window The queue kept the old channel's 4096 cap. That cap existed because a buffered chan allocates its whole capacity up front -- several MiB per engine at the default 65536 window, before a command is submitted. The slice queue starts at 64 entries and grows to the live depth, so the cost it avoided is gone. The cap was not free. It bounds ADMISSION, not memory: a submitter whose batch does not fit parks on the queue's cap-1 room signal and is woken one at a time. With 4096 slots, 4096 callers pipelining 10 commands each want 40960 slots, so roughly 3700 of them serialize on that relay. Measured on a 14-core client against a local server, 4096 callers x 10 commands per pipeline, 3 passes: cap 4096 9,232 ops/s p50 2,932 ms window 781,443 ops/s p50 52 ms Resident memory rises with it, 221 -> 855 MB, but that is the cost of actually having 40960 commands outstanding rather than a property of the bound: the commands live in the in-flight deque, which the window already bounds. The capped arm was cheap only because it was stalled. A broadcast wake was tried instead, to keep the 4096 bound and cure the relay. It recovers most of the throughput (632k) but costs 69% more CPU (11.00 vs 5.26 us/op) from every waiter waking on each release, and saves no memory (883 MB). Raising the bound is both simpler and better. Total outstanding is still bounded, now by 2*window. * fix(autopipeline): harden the batched FD reader A fatal reply ended only its read group. The reader then issued another WithReader on the desynced stream, and advance(done) dropped the failed command without completing it, so its caller hung. A fatal reply now ends the whole snapshot, which also keeps done a contiguous prefix. A read group shares one deadline. A reply that is only partly buffered needs another socket read and got whatever earlier replies left. The group now ends once half of ReadTimeout is spent, so the next WithReader re-arms a full deadline. * fix(autopipeline): O(batch) FD queue drain takeInto shifted the whole remaining queue down under the submit lock on every writer wave. At a full default queue (65536) that moved about 6.8 MB per 200-command wave and blocked every submitter while it did. The queue now keeps a head offset: a take costs O(batch), the dead prefix is reset when the queue empties and compacted once it is at least as long as the live part, so a compaction never copies more than was taken since the last one. FullDuplexFastSubmit shipped in v9.23.0-beta.1. Removing it broke code that sets it; it is back as a deprecated no-op field. * refactor(autopipeline): drop unused fdQueue.pushBatch Its only caller is FDPipelined, which arrives with the full-duplex pipeline change, so on its own this change failed lint (unused). It moves there. The queue model test and the deep-drain benchmark use push instead. * fix(autopipeline): contain FD group panics, free burst buffer The grouped reader parses several replies before completing them. A decoder panic on a later reply (a RawWriteTo whose writer panics) escaped to the reader's recover, which never counted the group's read replies, so recovery replayed them: a mutation could run twice, and an already-parsed result was overwritten. The panic is now turned into a fatal error for that command, so the replies before it complete and recovery starts at it, as the per-reply loop did. A drained submit queue kept the backing array a burst had grown (several MiB at the default window) for the engine's lifetime. It now releases an array past 4096 slots when it empties. * refactor(autopipeline)!: drop FullDuplexFastSubmit The field was kept as a deprecated no-op so code written against v9.23.0-beta.1 still compiled. It has had no effect since the submit channel became a queue, and it shipped only in a beta, so remove it. BREAKING CHANGE: AutoPipelineOptions.FullDuplexFastSubmit is removed. Delete the field where it is set; it had no effect. * fix(autopipeline): full deadline per FD socket read A read group runs under the one deadline its WithReader arms. It went on while the buffer was not empty and stopped after half the timeout, so a partly buffered reply reached early in a group still read the socket under the earlier deadline: it could time out, and the tail be torn down and replayed, although it arrived within its own ReadTimeout. The group now goes on only while the next reply is already whole in the buffer (proto.Reader.HasBufferedReply, a non-consuming scan, about 26 ns and no allocation per reply). A reply that needs the socket starts the next group, whose WithReader arms a full deadline. * fix(autopipeline): contain push panics in a read group The grouped reader drains pending pushes before each reply. A panic in a push handler (CSC invalidation, a registered or custom processor) escaped the group's WithReader, so the replies the group had already read were never completed and session recovery replayed them: a write read before the push ran twice. fdPushDrainSafe turns the panic into a fatal drain error at that position, as fdReadReplySafe does for the decoder, so the replies before it complete. HasBufferedReply counted a RESP3 attribute nested in an aggregate as an element, so it could report a reply whose decorated element was only partly buffered as complete, and the group then read the socket under the previous reply's deadline. An attribute now prefixes the element after it. |
||
|
Nedyalko Dyakov
|
f5ef5ed354 |
fix(csc): apply invalidation batches in one shard pass (#4033)
* fix(csc): apply invalidation batches in one shard pass ClientSideCacheInvalidationBatchWindow applied only a fraction of the invalidations it received under load. apply() looped per key calling deleteByRedisKeyCollectingHot, and that walks all 16 shards taking each write lock, so one worker doing 16*N contended acquisitions fell permanently behind the arriving stream. The rest stayed in the spill buffer, and the cache went on serving values the server had invalidated while reporting a ~100% hit rate. Measured on a 20k/sec invalidation stream: 7,729 of 68,846 enqueued invalidations applied (15.5%), with nothing dropped by policy (discarded=0, epochSkip=0). Invert the loops: take each shard's lock once per batch and handle every key of the batch under it, so a batch costs 16 acquisitions instead of 16 per key. Per-key semantics (fetch-order guard, recency horizon, target collection) are unchanged; only the locking granularity differs. Invalidations applied goes to 100%. * docs(csc): note that cached reads are not monotonic The invalidation batch window already documents staleness -- a reader can see the pre-invalidation value for up to the window. It does not document the ordering consequence: a caller reading through a non-cached path can observe a newer value than a concurrent in-flight fetch is about to publish, and a later cached read then returns the older one. This is inherent to asynchronous invalidation, not specific to the window. At publish time the client cannot know its in-flight value is already stale, because the only signal is a push that has not arrived yet; a guard that refuses to publish values superseded by an observed invalidation was tried and never fires. The window widens the exposure, because a published value stays readable until its invalidation is APPLIED rather than observed. Observed roughly once per 44M reads with a 5ms window, and not at all with the window off. * fix(csc): keep the read bit on batched refreshes collectHotAndDeleteBatch built the refresh target without the entry's second-chance bit, which collectHotAndDelete copies for stageRefreshAccess. With the invalidation batch window and refresh on, a refreshed hot key was republished with its bit clear and was the next eviction victim in a warm shard. Copy it, as the single-key path does. |
||
|
Nedyalko Dyakov
|
b32512362f |
perf(autopipeline): honor MaxFlushDelay in FD (#4014)
* perf(autopipeline): honor MaxFlushDelay in FD
The full-duplex writer's drain loop is non-blocking: it flushes whatever
is already queued and never waits. At low concurrency that is correct - a
lone caller pays one round trip and nothing waits on its behalf - but
under load it pays batch and syscall overhead at per-command frequency.
Measured on a 2-vCPU client (64B, GET/SET 70/30, one connection):
1024 callers, full-duplex 4.6 commands/flush, ~75k write syscalls/s,
CPU pinned at 196%
1024 callers, half-duplex 460 commands/flush, ~670 syscalls/s,
130% CPU
MaxFlushDelay was never honored here (no references in
autopipeline_fullduplex.go), so there was no way to coalesce. Honor it,
but gate the wait on in-flight depth: below fdAccumMinFor(window) the
writer never waits, so low-concurrency callers keep their 1xRTT
behaviour. Waiting unconditionally regressed 64 callers from 130k ops/s
at p50 0.48ms to 41k at p50 1.59ms - the same failure documented for the
half-duplex path's old debounce timer.
With MaxFlushDelay=250us:
callers before after
64 130k, p50 0.48ms 134k, p50 0.45ms (unchanged)
256 237k, 160% CPU 228k, 112% CPU (30% less CPU)
1024 345k, p99 4.78ms 420k, p99 3.53ms (+22% ops, -26% p99)
2048 308k, p99 10.1ms 386k, p99 7.19ms (+25% ops, -29% p99)
MaxFlushDelay defaults to 0, so the default path is unchanged and the
coalescing is strictly opt-in.
* fix(autopipeline): correct the FD accumulation wait
Three defects in the MaxFlushDelay wait added by this PR.
The wait returned on the first arriving command, so it waited for an
arrival rather than accumulating to a deadline. Measured 4.6 -> 7.9
commands per flush and +1% throughput, against 178 and +22% once the
writer waits out the full window. Arm the timer once per batch, then
re-enter the select until it fires.
The command arm stopped the timer and then received from its channel
unconditionally. Under Go 1.23+ timer semantics a failed Stop() does not
guarantee a pending value, so that receive can block forever and wedge
the writer and Close. The arm no longer touches the timer at all - it
stays armed for the rest of the window - and re-arming drains
non-blockingly.
The wait observed only the submit channel and its own timer, so
ctx.Done(), readerDone and max-hold could not interrupt it: a writer
parked on MaxFlushDelay held its connection past FullDuplexMaxHold and
delayed Close by up to the configured delay. All three now stop the wait
and flush the prefix already taken off the queue, so the existing
graceful, recycle and connection-error paths end the session. maxC comes
from a one-shot timer, so its tick is carried and acted on after the
flush instead of being consumed and dropped.
Adds tests for the threshold helper, the no-wait contract at low
concurrency, and correctness through the armed wait under load.
* perf(autopipeline): raise the FD accumulation gate to window>>7
fdAccumMinFor returned max(64, window>>9) = 128 at the default window, so the
MaxFlushDelay wait armed from 128 concurrent callers. Measured on the channel
engine it runs at 0.93x the undelayed throughput there, because a 250us window
collects too few commands to pay for itself; window>>7 makes it 0.99x and
keeps the win at higher concurrency (1.02x at 512, 1.09x at 1024, 1.13x at 2048).
The trade is explicit: this gives up a 1.07x gain at 256 callers. It is taken
because the wait only raises throughput when CPU is the binding constraint, and
below this threshold there is CPU headroom, so the writer idles out the window
instead. The margin is far larger on the full-duplex slice-queue engine, where
the same threshold costs 0.39x at 128 callers and 0.82x at 256.
TestFDAccumMinFor pins this constant and its floor boundary, so it moves with it.
Refs #4014
* fix(autopipeline): no FD write after the reader exits
The MaxFlushDelay accumulation wait returns when the reader exits, and
the writer then wrote the batch it had taken off the queue. A read
timeout leaves the socket open, so the server ran those commands, and
recovery replayed them as unacknowledged: one command, two executions.
The writer now keeps such a batch unsent and the connection-error path
recovers it behind the unacked tail as never sent (refunded), so it
runs once. A handoff mark seen at the same point keeps the batch unsent
too and recycles the connection, so nothing more goes to a connection
that is being moved.
Also document that AdaptiveDelay applies to the half-duplex batcher
only; the full-duplex writer has its own in-flight-gated wait.
|
||
|
Nedyalko Dyakov
|
0bc2cb8dc2 |
perf(csc): build the key list only on a miss (#4037)
* perf(csc): build the key list only on a miss extractRedisKeys and the namespaced copy ran before the cache-hit check, but only Reserve uses them and Reserve is on the miss path. At a 94-99% hit rate that allocated a key slice, a namespaced slice and one string per key on nearly every read and discarded all three -- by object count about 18% of everything the client allocated. Object count is what matters here: after cache-key pooling cut allocated bytes by 30%, memclrNoHeapPointers was still 20% of CPU, because malloc and zeroing track the number of objects, not their size. Deferring the "no key list, not cacheable" gate along with it is safe: extractRedisKeys depends only on the command's shape, so a command with no key list was never cacheable and can never be a hit. The gate still runs before anything is cached. * fix(csc): check key types before the cache key Deferring the key list to the miss path also deferred the key-type check, so buildCacheKeyNS encoded the arguments first. For a key the cache cannot render, such as a BinaryMarshaler, MarshalBinary then ran twice: once for the cache key and once for the request. The key positions of every cacheable command are one contiguous span. processCached now reads that span and checks its types before it builds the cache key, without allocating, and builds the key strings only on a miss. extractRedisKeys moves to the tests. * fix(csc): reject negative key positions SetFirstKeyPos is public and takes any int8. isCacheable accepted any nonzero position and cscKeySpan rejected only 0, so a negative one let cscKeysRenderable index args[-1] and panic the process. The keyArg bounds check this replaced had treated such a command as uncacheable. Both now require a position above 0, so the command is sent normally. |
||
|
Nedyalko Dyakov
|
ac839125d6 |
perf(csc): pool the cache-key scratch, fuse the namespace (#4036)
* perf(csc): pool the cache-key scratch, fuse the namespace Building a cache key was 32% of all bytes allocated on the cached read path. Every call heap-allocated a bytes.Buffer, a proto.Writer (which itself allocates two 64-byte scratch slices, 16.2% of all bytes on its own), grew the buffer, and copied it out with String() -- and then the caller did prefix+key for a second full copy of every key. Only the final String() is inherent: the key is retained, the scratch is not. Pool the buffer and writer, and write the namespace into the buffer before the args so one String() yields the namespaced key directly. Five allocations become one. proto.Writer gains Reset so a writer can be re-targeted rather than reallocated. Total bytes allocated 9,858MB -> 6,915MB (-30%); NewWriter, buildCacheKey, cscNamespacedKey and Buffer.grow leave the profile. Throughput +8.7% at 256 callers, +13.6% at 1024, +37.4% at 10k invalidations/s, with p99 down in every cell. * fix(csc): move the un-namespaced cache-key helper to tests buildCacheKey lost its last non-test caller when the cached read path moved to buildCacheKeyNS, which builds the namespaced key in one allocation. golangci-lint runs with run.tests=false, so its remaining callers in csc_test.go do not count and the unused check fails. Move it to csc_cachekey_test.go: production keeps a single entry point and the tests keep the un-namespaced form they assert on. |
||
|
Nedyalko Dyakov
|
2ecf759f46 |
perf(csc): skip the read mark below eviction pressure (#4035)
* perf(csc): skip the read mark below eviction pressure Even after the second-chance bit replaced the per-hit recency stamp, the surviving store was 9.7% of all CPU -- 15.24s of 157.84s, against 50ms for the Load that guards it. It stays hot because entries are invalidated and refetched constantly and fulfill never set the bit, so every fresh entry's first read finds it clear; each such store invalidates, across every core, a line the readers hold SHARED. (Eviction sweeps clearing the bit were ruled out: raising the entry budget from 150k to 400k left the cost unchanged.) The bit is read by nothing except eviction, so do not maintain it on a shard that is nowhere near its cap. len(s.entries) is safe to read on the hit path: writers hold the write lock, readers the read lock. cacheShard.get drops from 10.80% of CPU to 1.44%. Throughput +8.5% to +20.6% and p99 -13% to -44% on an over-provisioned cache; neutral when the cache sits at its cap (+1.1%/+0.1%, hit rate identical), which is the case where the gate cannot help by construction. * docs(csc): note the pressure gate's warm-up trade A read made while a shard is below the gate threshold leaves no mark, so the first evictions after a gradual warm-up go by insertion order among such entries. Recovering those reads would need the per-read store the gate exists to skip. |
||
|
Nedyalko Dyakov
|
58cc03e024 |
perf(csc)!: mark reads with a second-chance bit (#4034)
* feat(csc)!: always refresh invalidated entries ClientSideCacheRefreshRecencyWindow bounded refresh to entries read within a window. It was the only consumer that needed a per-READ recency timestamp on the cache entry, and that timestamp is the dominant cost of a cache hit. Removing the option lets the read path stop stamping entries, which the following commit does. Refresh now covers every invalidated entry. BREAKING CHANGE: Options.ClientSideCacheRefreshRecencyWindow is removed. Refresh-on-invalidate always refreshes every invalidated entry. * perf(csc): mark reads with a second-chance bit A cache hit stamped entry.lastAccessNs. The read already touches the entry's cache line, but a store dirties it, so the line must be held exclusive and costs a writeback when evicted from the CPU cache. With many concurrent readers over a working set larger than L2, that store dominates the cost of a hit. A read now sets a readSinceSweep bit instead. A load that finds it set leaves the line SHARED, so every core can hold it at once and only the first read after an eviction sweep writes. Eviction consumes the bit as a second chance; ordering still uses lastAccessNs, written on insert. Measured at 256 concurrent callers, get=90/set=10, 99.94% hit rate: 263,801 -> 328,842 reads/s (+24.7%), CPU 27.0 -> 18.9 us/op, hit rate unchanged. Removing the recency update entirely measured +40.5%, so this recovers about three fifths of the headroom while keeping eviction honest. * perf(csc): decode simple cached replies without a reader Serving a hit allocated a proto.Reader (and its bufio buffer) per hit and re-parsed the cached bytes. applyCachedFast decodes the simple single-frame replies -- nil, status, bulk string -- straight into the Cmder, and getShared serves the bytes without the defensive copy where the caller provably cannot retain them. * fix(csc): do not evict the entry just fulfilled fulfill publishes an entry with its second-chance bit clear and then evicts. In a warm memory-capped shard, where every resident entry has been read, that entry was the only clear-bit candidate and was evicted on the spot; and since a clear candidate always existed, the sweep that resets the others never ran, so the shard admitted nothing new. The entry just fulfilled is now skipped in its own eviction pass. A refresh republish also restores the invalidated entry's read bit with its access token, so a refreshed hot key is not the first victim. * fix(csc): decline negative bulk lengths other than -1 The fast cached-reply decoder took any negative bulk length as a RESP2 null. The reader rejects lengths below -1, and that rejection is what makes processCached drop a bad entry and refetch; the fast path turned it into a false miss. It now handles only -1 and declines the rest. * fix(csc): reject bare LF in cached status replies The fast cached-reply decoder rejected a status frame with an interior CR but took one with a bare LF, such as "+OK\nMORE\r\n", as a valid string. The reader stops at the first LF and rejects the frame, and that rejection is what makes processCached drop the entry and refetch. The fast path now declines an interior CR or LF. * fix(csc): keep refresh recency in the publish lock A refresh republish published the entry with its second-chance bit clear and restored the bit in a second lock. On a warm shard the republished entry was then the only clear-bit victim, so an insert in that gap evicted the key the refresh had just fetched. The refresher now stages the kept recency on its reservation (stageRefreshAccess), and fulfill applies it in the same lock as the publish. * docs(csc)!: refresh-on-invalidate refreshes all ClientSideCacheRefreshOnInvalidate still said it re-fetches recently read keys. Since the recency window was removed, it re-fetches every cached entry of an invalidated key, so on a write-heavy keyspace the refetch traffic covers the whole resident cache. BREAKING CHANGE: Options.ClientSideCacheRefreshRecencyWindow and UniversalOptions.ClientSideCacheRefreshRecencyWindow are removed. They shipped only in v9.23.0-beta.1; delete the field where it is set. * docs(csc): note reads are not monotonic per caller A read that loses the reservation twice reads the server directly. Another caller's fetch, reserved earlier and still running, can store an older value afterwards, and a later hit returns it until the key's invalidation arrives. This is within the staleness a hit already has, so document it rather than drop the older fetch, which would cost its waiters a miss each under the churn that leads here. |
||
|
Nedyalko Dyakov
|
a8c360cc45 |
fix(command): set key position for 16 commands (#4048)
* fix(command): set key position for 16 commands
The shared key resolver routes Ring shards, cluster slots, client-side
caching and full-duplex engines. For typed commands whose key is not
args[1] it fell back to args[1], or, on a warm cluster cache, to the
COMMAND first key, which is 0 for container and numkeys commands:
BITOP (all BitOp*), MIGRATE op / host instead of the key
OBJECT ENCODING/FREQ/IDLETIME/REFCOUNT, XINFO STREAM/GROUPS/CONSUMERS
subcommand instead of the key
LMPOP, SINTERCARD, ZINTERCARD, ZMPOP numkeys instead of the key
BLMPOP, BZMPOP timeout instead of the key
Ring sent them to the wrong shard; cluster recovered through MOVED.
The constructors now set the key position, as XGroup*, ZUnion and FCall
already do. A raw FCALL/FCALL_RO also resolves its key after numkeys,
like EVAL, instead of hashing the function name.
* fix(command): route raw forms by their key
The typed constructors of the commands whose key is not args[1] set the
key position; a raw Do / NewCmd of the same commands does not, so the
shared resolver fell back to args[1]: raw XREAD hashed "streams" and raw
LMPOP its numkeys, which picks the wrong Ring shard, the wrong
full-duplex engine, or a cold cluster slot.
The resolver now finds the key for the raw forms: XREAD and XREADGROUP
after STREAMS (XREADGROUP scans past the group and consumer names),
OBJECT and XINFO (HELP is keyless), BITOP, LMPOP, ZMPOP, SINTERCARD,
ZINTERCARD, BLMPOP, BZMPOP, and MIGRATE, including its KEYS form.
* fix(command): walk MIGRATE options for KEYS
The raw MIGRATE KEYS form was located by scanning for the first "keys"
token after the timeout, so an AUTH or AUTH2 operand that reads "keys"
was taken for the clause and the resolver routed by the token after it.
It now walks the option grammar: COPY and REPLACE take no operand,
AUTH takes one and AUTH2 two, and the key follows KEYS.
|
||
|
|
d907b985fc |
feat(autopipeline): full duplex flusher (#4002)
* feat(autopipeline): ordered full-duplex dispatch (#3964) * fix(pipeline): PipelinePoolSize alone should create the pipeline pool The dedicated pipeline connection pool was created only when PipelineReadBufferSize or PipelineWriteBufferSize was set. Setting just PipelinePoolSize therefore produced no pipeline pool at all, with no error and no log line, and every pipelined operation silently kept using the main connection pool — competing with ordinary commands for the same connections, which is the opposite of what the option asks for. The gating was inverted. The pool size is the primary intent ("give pipelines their own N connections"); the buffer sizes are tuning. And the buffer sizes already supplied a default for the size (10), so the dependency only ran one way. Any of the three options now creates the pool. Enabling it by size inherits the regular ReadBufferSize/WriteBufferSize, so a caller who only wants to bound pipeline connections no longer has to know that a buffer size is secretly required. Found while measuring client-side caching over a high-latency link: the CSC background batched reads go through withPipelineConn, which falls back to the regular pool when none exists, so they were contending with the very reader misses they exist to prevent. Tests cover all four option combinations and were verified to fail with the gate reverted. * feat(pipeline): give autopipelined clients a dedicated pipeline pool by default Autopipelining is pipeline-heavy by definition, yet the dedicated pipeline pool still had to be enabled separately; without it every batch competes with regular commands for main-pool connections. Two paths now default it in: Config time: Options.init defaults PipelinePoolSize to DefaultPipelinePoolSize (new exported const, 10) when AutoPipelineOptions is set. Cluster node clients do not inherit AutoPipelineOptions (the cluster autopipeliner lives at cluster level and dispatches into the node clients' pipeline hooks), so clientOptions applies the same default there. Sentinel and failover node clients inherit AutoPipelineOptions and are covered by Options.init; Ring does not support autopipelining. A negative PipelinePoolSize opts out explicitly. Runtime: the common pattern configures the autopipeliner after NewClient (AutoPipelineWithOptions on a plain client), where pool creation has already happened. ensurePipelinePool now builds the pool lazily when an autopipeliner is created. baseClient.pipelinePool becomes a pointer to an atomic slot (pool + pool-name ref) so the lazy publication is race-free against concurrent Pipelined callers on the hot path, and WithTimeout clones share the slot — two sharers cannot install two different pools (CAS; the loser closes its candidate). Streaming-credentials and maintnotifications hooks are wired before publication; otel registration happens for the winner. Conn, Tx and SentinelClient never allocate the slot and keep using the main pool. The extraction into buildPipelinePool also surfaced that NewFailoverClient kept a duplicated creation block with the pre-fix gate (buffer sizes only), so PipelinePoolSize alone silently created no pipeline pool on failover clients; the shared helper fixes that, with a regression test. Stale RingOptions/FailoverOptions doc lines still describing the old gate are corrected. * feat(pipeline): 64KiB default buffers and no pre-dial for the pipeline pool The options the dedicated pipeline pool is built with are now resolved by pipelinePoolOptions, a pure function with pinned tests: Buffers default to DefaultPipelineBufferSize (new exported const, 64 KiB) rather than inheriting the regular 32 KiB. Pipeline connections move whole batches per round trip and earn bigger buffers than per-command traffic: measured on the autopipeline engine, throughput plateaus around 64 KiB, gains nothing past ~128 KiB, and very large buffers can regress it. An explicitly larger regular buffer is kept rather than shrunk, and explicit pipeline buffer sizes always win. The RESP3 minimum clamp applies as on the main pool. MinIdleConns is forced to 0 on the pipeline pool. It previously inherited the main pool's value through the options clone, so a client with MinIdleConns set pre-dialed that many pipeline connections at pool creation — a silent doubling of the idle footprint. The pipeline pool is burst capacity: its connections dial on demand, and the pool size (PipelinePoolSize or DefaultPipelinePoolSize) is only a cap, so an unused pipeline pool holds zero connections. * test: make pipeline-conn specs aware of the dedicated pipeline pool The autopipeline-subjects job builds its suite subject by calling AutoPipelineWithOptions on the shared rawClient, which now (correctly) creates the dedicated pipeline pool lazily - so client.Pipelined and client.Pipeline run on pipeline-pool connections, not the main pool's. Two specs assumed the old sharing: should Auth asserted main-pool stats (Hits=2/Misses=1/TotalConns=1) after two Pipelined calls. It now branches on PoolStats.PipelineStats: when the dedicated pool exists, the two pipelines account there (first dials, second reuses) and the main pool holds only the BeforeEach FlushDB dial; in the default subject mode PipelineStats is nil and the original assertions run unchanged. should ClientSetInfo issued CLIENT SETINFO through a pipeline and then read CLIENT INFO through rawClient - with a pipeline pool those are different connections, and SETINFO is per-connection state. It now reads CLIENT INFO through the same pipeline that issued the SETINFOs, which is equivalent on a single-pool client and correct on a split-pool one. Verified against redis:8 in all three GOREDIS_TEST_SUBJECT modes (default, ap-blocking, ap-async): Ran 2 of 1543 Specs, 2 Passed. * test: warm the pipeline-pool conn in AP tests that instrument connections Two autopipeline unit tests assumed batches execute on the main pool's already-initialized connection; with the dedicated pipeline pool now created lazily at autopipeliner creation, a batch's first dispatch dials and initializes a fresh pipeline-pool connection instead. Both tests now warm that connection with a real two-command batch first (a lone command takes the solo fast path through Process on the main pool and would not touch the pipeline pool): TestAutoPipelineRetriesOnNetworkError counted the pipeline pool's first dial as if it were a retry redial (3 dials observed, 2 expected). The warm-up runs before the write failure is armed, so the count again measures exactly the one redial the retry performs. TestAutoPipelineHookPostNextErrorPartialBatch installed a hook that unconditionally injects a post-next error before the pipeline-pool connection existed. Connection init runs its handshake pipeline through the client's hook chain (newConn shares hooksMixin), so the hook failed the pipeline conn's init itself — retries exhausted, the injected error was stamped on every command, and the test measured init poisoning rather than the post-next outcome rule it pins. The warm-up initializes the connection before the hook is installed. * docs: TODO(v10) — ClientTracking* and ClientMaintNotifications are stateful CLIENT TRACKING and CLIENT MAINT_NOTIFICATIONS mutate per-connection state, exactly like CLIENT SETNAME/SETINFO, but live on cmdable: on a pooled client they land on an arbitrary pool connection that is immediately returned to the pool, so the flag applies to a connection the caller cannot address again. They belong on statefulCmdable, next to ClientSetName — but the move removes the methods from Client/UniversalClient/Cmdable (and defining them on both cmdable and statefulCmdable makes every Pipeline selector ambiguous), so it is a breaking change deferred to v10. Documented now; the move happens in a follow-up PR. No behavior change. * docs: drop the TODO(v10) notes on the per-connection commands The ClientTracking*/ClientMaintNotifications move to statefulCmdable ships as its own PR against master (feature/stateful-conn-commands) rather than waiting for v10, so the TODO notes here are obsolete. Docs only. * feat(pipeline): always create the pipeline pool; spill on exhaustion The dedicated pipeline pool is now created unconditionally at NewClient/NewFailoverClient — like the pubsub pool — instead of behind a matrix of triggers (pipeline options set, AutoPipelineOptions declared, or lazily when an autopipeliner was built). It is pure burst capacity: no pre-dialing (MinIdleConns forced to 0), a small cap (DefaultPipelinePoolSize) and 64 KiB buffers, so an unused pipeline pool holds zero connections and costs nothing, while pipelines stop competing with regular commands for main-pool connections. PipelinePoolSize < 0 opts out and restores the old pipelines-on-the-main-pool behavior. To keep this from capping heavy concurrent Pipelined callers — who previously shared the much larger main pool — withPipelineConn spills to the main pool when the pipeline pool is exhausted (ErrPoolTimeout): bursts wider than the cap keep their old capacity, spilled pipelines simply run with regular buffers, and total connections stay bounded by PoolSize + PipelinePoolSize. Pipeline-pool connections are excluded from CLIENT TRACKING during init. Pipelined commands never consult or populate the built-in client-side cache (only the single-command cached path on main-pool connections does), so tracking pipeline reads only grew the server's tracking table and produced invalidation pushes for keys the cache does not hold. With creation unconditional, the lazy machinery is deleted: the atomic slot and CAS publication, ensurePipelinePool, the autopipeliner-factory hooks, the Options.init AutoPipelineOptions defaulting and its cluster clientOptions mirror. pipelinePool is a plain field set before the client is visible to any goroutine; WithTimeout clones share it. Net diff of this commit is negative. Tests: the creation-rule table becomes always-on plus opt-out; new e2e coverage for the spill (a pipeline pool of one, held busy by a pipelined BLPOP, must not queue a second pipeline behind PoolTimeout) and for the tracking exclusion (CLIENT LIST shows the main-pool connection tracked and the pipeline-pool connection untracked). * test: keep HImport mock choreography on the main pool; doc the pool default The HImport mock tests choreograph exact per-connection sequences — a pool of one, booms armed to fire on the Nth replayed PREPARE, session counts pinned per connection. The always-on pipeline pool adds a second connection whose init replays fieldsets (correct production behavior: HIMPORT pipelines work on pipeline-pool connections precisely because of that replay), which fired the armed booms during the pipeline conn's init instead of the reissue under test and shifted the session counts. Opt every client in the file out with PipelinePoolSize: -1: the tests pin HIMPORT reissue/session semantics, not pool routing. Also states the PipelinePoolSize default plainly in the option docs: DefaultPipelinePoolSize (10) connections (review feedback). * test: skip conn-init handshakes in the cluster shard pipeline hooks The shard-level pipeline hooks in the cluster hook-order specs asserted Expect(cmds).To(HaveLen(...)) unconditionally. With the always-on pipeline pool, a node's first pipeline dials a pipeline-pool connection whose init runs its handshake pipeline through the node client's hook chain, so the assertion fired on the handshake and the resulting Ginkgo panic aborted the whole suite. Guard both hooks with the idiom ring_test.go already uses for the same situation: defer GinkgoRecover() and pass connection-initialization pipelines (hello/client) through. The cluster-level hooks are untouched — node connection init never runs through the cluster client's chain. * test: adapt plain-pipeline tests to the always-on pipeline pool TestNoPipelinePoolStats pinned the stats shape of a client without a pipeline pool by creating a client without pipeline options — a premise the always-on pool inverts. Having no pool now requires the explicit opt-out, so the test sets PipelinePoolSize: -1, which is exactly the configuration whose stats shape it verifies. TestPipelineRetriesOnNetworkError is the plain-pipeline twin of the AP dial-count test adapted earlier: pipelines now run on the dedicated pipeline pool, so the first Exec dialed a fresh pipeline-pool connection and the test counted it as a phantom retry redial. Warm the pipeline connection with one Pipelined round trip before arming the write failure, so the count again measures exactly the one redial the retry performs. * feat(autopipeline): ordered full-duplex dispatch (single-connection, WAN-optimized) One held pipeline-pool connection with a writer + reader goroutine pair streams the ordered command stream instead of the half-duplex one-batch-per-round-trip flusher. On a latency-bound link under many concurrent goroutines this gives ~1 RTT latency at one connection: WAN 50ms w=4096 inflight=5 ~389k ops/s @ 52ms p50 vs ~207k @ 116ms for the half-duplex ordered path (errors=0). - Gated by AutoPipelineOptions.FullDuplex; honored only on the async, ordered (Unordered:false, MaxConcurrentBatches<=1), single-shard face of a standalone *Client with a pipeline pool. Cluster stays half-duplex. Validate rejects the contradictory combos; tuning via FullDuplex{Window,IdleTimeout,MaxHold}. - Connection-failure retries re-issue the unacked ordered tail on a fresh conn, honoring shouldRetry/MaxRetries/backoff and the NoRetry flag (parity with the half-duplex pipeline's cmdsContainNoRetry). - Lease/return (idle + max-hold) returns the held conn through the pool so its per-conn hooks (streaming-creds re-auth, maintnotifications) run. - RESP3 push frames demuxed inline (invalidation etc. never misread as a reply). - Per-command process hooks (redisotel spans/metrics, custom ProcessHooks) fire on the FD path; blocking/conn-hostile commands divert to a separate conn. - Bounded in-flight window applies backpressure (Submit blocks when full). 18 FD tests, all -race. Depends on #3959 (getPipelinePool + always-on pipeline pool). See AP_ORDERED_FULLDUPLEX_DESIGN.md. * fix(autopipeline): correct full-duplex teardown, recovery, and shutdown races Fixes four concurrency defects in the ordered full-duplex dispatch engine: - failReqs called cmd.Err(), which awaits batch.done -- the very channel failReqs then closes -- deadlocking the engine goroutine on retry exhaustion / Close. Use rawErr() (the same trap hostHook already documents). - On a connection-error session end the deque was taken (closeRecover) concurrently with the reader, so a command the reader had completed but not yet advanced could be handed to the retry loop and re-executed (and, on the hooked path, double-close its hookDone channel -- a panic). Split into hardClose + takeRemaining and take the tail only after the reader has exited. - A desynced connection was returned to the pool via Put() unless the error was classified bad-conn, poisoning the pool for the next caller (e.g. errFDReaderGone, a write timeout). Remove on any fdConnErr session end. - submit()'s send could win the race against the shutdown drain and enqueue a request that was never completed, hanging the caller forever. Gate the send with an RWMutex that drainQueue takes before draining. Also closes the readerDone/clean-result race: a reader protocol error during an idle/recycle/graceful return now removes the conn instead of Put-ing it back poisoned, and fails any stranded in-flight tail so no caller hangs. Adds pure unit tests for the failReqs deadlock and the deque ownership invariant. Existing full-duplex suite (conn-kill, close-while-backpressured, ctx-cancel, goroutine-leak) passes under -race. * style(autopipeline): fix lint in full-duplex dispatch Clears the three pre-existing staticcheck failures in the ordered full-duplex file (present before the teardown/recovery fix; that commit added none): - ST1008: reorder attempt() and session() return values so the error is the last argument -- (unacked, result, aerr) instead of (unacked, aerr, result). Pure reordering; the two call sites are updated to match. error and fdResult are distinct types, so any misorder fails to compile. - unused: //nolint:unused on fdInflight.peakLen, which is exercised only by the full-duplex backpressure tests (the repo lints with run.tests=false, so test-only usage is invisible to the unused check). * fix(autopipeline): full-duplex hardening (retry, recover, ctx, drain, bytes) Full-duplex review follow-ups (all in autopipeline_fullduplex.go): - retry read/write timeouts: shouldRetry(aerr, true), matching the cluster pipeline retry paths, so a single deadline miss no longer fails the whole unacked tail (cursor High); fdReqsNoRetry still excludes non-idempotent cmds. - recover a panicking user ProcessHook in hostHook (fail cmd + close batch), so it can neither crash the process nor leave the caller blocked. - close the conn on the connection-error path so a reader blocked in WithReader unblocks at once instead of stalling up to the read deadline (cursor High). - honor a disabled ReadTimeout (<0) as no-deadline instead of clamping to 30s. - log (not propagate) a push-drain error, matching the workers path, so a custom push processor error no longer fails an unrelated in-flight command. - return a clean conn via releaseConnToPool so pending push notifications are drained before Put (no delayed handoff). - acquire under ap.ctx so Close aborts a blocked pool.Get (init and I/O stay on Background so accepted commands still complete); backpressured submit honors the caller ctx. - apply a soft MaxBatchBytes cap in the writer drain loop. * fix(autopipeline): full-duplex hardening (retry, recover, ctx, drain, bytes) Full-duplex review follow-ups (all in autopipeline_fullduplex.go): - retry read/write timeouts: shouldRetry(aerr, true), matching the cluster pipeline retry paths, so a single deadline miss no longer fails the whole unacked tail (cursor High); fdReqsNoRetry still excludes non-idempotent cmds. - recover a panicking user ProcessHook in hostHook (fail cmd + close batch), so it can neither crash the process nor leave the caller blocked. - close the conn on the connection-error path so a reader blocked in WithReader unblocks at once instead of stalling up to the read deadline (cursor High). - honor a disabled ReadTimeout (<0) as no-deadline instead of clamping to 30s. - log (not propagate) a push-drain error, matching the workers path, so a custom push processor error no longer fails an unrelated in-flight command. - return a clean conn via releaseConnToPool so pending push notifications are drained before Put (no delayed handoff). - acquire under ap.ctx so Close aborts a blocked pool.Get (init and I/O stay on Background so accepted commands still complete); backpressured submit honors the caller ctx. - apply a soft MaxBatchBytes cap in the writer drain loop. - honor opt.Limiter per session: Allow() before the session pool.Get, ReportResult() before release (report-before-release), 1:1 per session (the Limiter gates conn acquisition; FD acquires one conn per session). - on a retryable Redis error (LOADING/READONLY/CLUSTERDOWN/…) or a redirect (MOVED/ASK), re-run the command on the client normal path (pipeliner.process) off the reader goroutine (tracked by retryWg so Close waits) and settle the caller with that result; ordering across the divert is not promised (same as the blocking-command divert), NoRetry commands keep their error. - record the native per-command OTel metric (RecordOperationDuration, write->reply) from the reader, mirroring process; skipped for retry-diverted commands (they emit via process). fdReq carries its submit ctx + write timestamp. - divert managed HIMPORT (PREPARE/SET/DISCARD/DISCARDALL) off the FD pipe to the normal Process path: the FD writer streams raw commands and never replays the registered PREPARE (connection-session state), so an HIMPORT SET on the pipe could fail "no such fieldset". Process injects the PREPARE from the registry and keeps it current on PREPARE; the half-duplex sharded path injects inline (himportInjectedCmds) and is left on the pipeline. FD-scoped: gated on ap.fd. - wait for per-command hook-host goroutines during Close: a ProcessHook doing work after next() returns runs on a host goroutine that is the only code closing that command's batch. Track it in a hostWg that run() waits (so Close, which waits ap.wg, waits it too), restoring the drain-before-return contract. The hostWg.Add is gated behind submitMu+closed exactly like the ch send, so run()'s hostWg.Wait never races a live Add (every run() return is preceded by drainQueue). - stamp the FD hook host as the batch executor (dispGid) so a ProcessHook that reads its own command result after next() (cmd.Err()/cmd.String()) gets the just-executed view instead of self-deadlocking on the batch only that goroutine closes (cursor P1). Mirrors runOutsidePipeline's async dispatch guard. * test(autopipeline): consolidate full-duplex tests into one file Merge the per-topic full-duplex test files (limiter, retry, otel, himport, close, hookdeadlock) into autopipeline_fullduplex_test.go and remove them. No test logic changes; the imports are unioned (adds internal/otel). * fix(search): read the full RESP3 map reply in FTHybridCmd (#3956) FTHybridCmd.readReply used ReadSlice, which reads a RESP3 map header's length as an element count and consumes only half its frames, leaving the connection desynced. Peek the type and read a map with ReadReply, like the other FT.* parsers. * fix(autopipeline): full-duplex review round 3 (read-timeout, panic, retry split) - Honor a disabled ReadTimeout in the FD reader. Options.init maps a user's ReadTimeout:-1 to 0; the reader clamped 0 -> 30s, forcing a deadline on a caller who disabled it (regression). It now passes opt.ReadTimeout through: WithReader/deadline() treat <= 0 as "no deadline", while a default client keeps its positive per-reply timeout. A stuck read is still unblocked by the conn Close on the error path. - On a ProcessHook panic BEFORE next(), the hook host now waits for the reader to finish writing the command (<-hookDone) before closing the batch. The command was already streamed, so closing immediately let the reader write cmd.err (an unsynchronized field) after the caller observed the recovered failure — a data race, and a non-idempotent command left half-settled. Mirrors the short-circuit path, which already waited. - Split the unacked tail at the first NoRetry command on a connection failure (fdFirstNoRetry): replay the retryable prefix, fail that command and everything ordered after it, instead of failing the whole tail. Ordinary commands before a NoRetry keep their network retries; the NoRetry command is never re-sent — matching the half-duplex split of contiguous retry-policy runs. - Recover a panic in reply decoding on the FD reader goroutine (e.g. a RawWriteToCmd whose user io.Writer panics mid-stream). Without recovery it would crash the process; now the session is marked failed (fdConnErr), the unacked tail is recovered and the desynced conn removed — mirroring recoverDispatchPanic on the half-duplex dispatchers. Tests (autopipeline_fullduplex_test.go): pre-next-panic settles with an error and does not hang; fdFirstNoRetry split index across leading/middle/trailing NoRetry; reader-decoder panic is recovered and the engine keeps serving. * fix(autopipeline): advance completed FD commands before reader-panic recovery The reader-panic recover (round 3) marked the session failed but did not advance the commands already completed in the panicking frontBatch snapshot. Recovery then re-owned and re-completed them, overwriting good results and double-closing hookDone (a second panic) when process hooks are installed. The reader now hoists `done` (completed-but-not-yet-advanced in the current snapshot) to goroutine scope, reset at the top of each iteration, and the recover calls inflight.advance(done) before failing the session — so recovery only re-owns genuinely-unacked entries. A frontBatch panic advances nothing (done 0). Test: TestFullDuplexReaderPanicMidBatchNoDoubleComplete — several completed hooked GETs precede a decoder-panicking command in one batch; the engine must survive without a double-close. Also sharpen the FullDuplex doc: a hook that short-circuits (returns without next) does NOT block an already-streamed command on the FD path, so a policy/blocking hook must run on a plain client or the half-duplex autopipeline; observability hooks are unaffected. Adds a TODO for opt-in write-gating. * fix(autopipeline): full-duplex review fixes - Limiter denial (fdDenied) and lease/init failure (fdLeaseErr) fail-fast carry + the fd.ch backlog after retries via a non-closing failQueue, instead of hanging accepted commands; a mid-session drop stays fdConnErr so buffered work survives a reconnect. - submit starts the hook host only after admission; submit-reject paths return the completedBatch sentinel (no host leak; raw Process reports a canceled submit). - graceful Close flushes the accepted fd.ch backlog (flushBacklogForClose). - recovered replay tail re-issued in MaxBatchSize/MaxBatchBytes chunks (writeCarryChunked/fdBatchEnd). - reader-panic recover advances completed commands before recovery (no double complete); honor a disabled ReadTimeout; split the unacked tail at the first NoRetry; use the shared initPooledConn; max-hold returns fdIdle when drained; Len() counts fd.ch; retry-divert / short-circuit-hook ordering documented. * fix(autopipeline): recover full-duplex carry suffix; lease lazily - writeCarryChunked pushed each chunk into the in-flight deque before writing it, so a multi-chunk carry/backlog whose write failed partway left the un-written suffix in neither fd.ch nor the deque — those accepted commands never completed and their callers hung. Push the suffix on a write error so the whole set is recoverable (replayed on the conn-error path, failed on Close). - On a Close-flush write error, close the broken connection and fail the tail (like the connection-error path) instead of closeGraceful, which left the reader waiting on replies that never arrive (hanging until ReadTimeout). - Lease the FD connection lazily, on the first command, instead of eagerly at startup: an unused FD autopipeliner no longer dials / hits the Limiter in the background or spins on a lease failure against an empty queue. * ci(govulncheck): use stable Go to pick up security patches setup-go's "1.26.x" resolved to go1.26.5, which govulncheck flags for two standard-library vulnerabilities fixed in go1.26.6: GO-2026-6090 (crypto/tls) and GO-2026-5972 (encoding/asn1). Track the latest stable toolchain so future security patches are picked up automatically instead of pinning a patch. * fix(autopipeline): flush Close backlog between sessions; lease only with work - run()'s terminal shutdown drains now call shutdownFlush: carry plus the fd.ch backlog are dispatched as one batch on the client's normal pipeline path and completed, honoring the "accepted => completes" Close contract on the between-sessions paths exactly like the in-session flushBacklogForClose, instead of failing accepted work ErrClosed. takeQueue replaces drainQueue as the gate-closing shutdown drain. Adds TestFDShutdownFlushCompletesBetweenSessionsBacklog. - The wait-for-work is hoisted to the top of run()'s loop: with an empty carry (initial entry, fdIdle, and the fail-fast exits after fdDenied / exhausted fdLeaseErr / a failed fdConnErr tail) the engine blocks for the next command before touching the Limiter or dialing, so a failure against an empty queue cannot spin the lease loop. Non-blocking when work is queued, so fdRecycle with pending work still re-leases immediately. * fix(autopipeline): FD shutdown, panic, retention and retry-divert fixes - The fdLeaseErr and fdDenied branches check ap.ctx first: when Close is what interrupted the acquisition, run() flushes carry + the fd.ch backlog via shutdownFlush instead of failing accepted work with the cancellation error. - writeBatch recovers a command-encoding panic (user BinaryMarshaler) and converts it to a connection error, mirroring recoverDispatchPanic on the half-duplex dispatchers; the conn-error path replays or fails the batch. - fdInflight.advance zeroes the consumed prefix before reslicing, so a drained burst does not retain a window of completed fdReq values (args, contexts) while the engine idles. - The reader's divert distinguishes redirects from retries: MOVED/ASK always divert, retryable Redis errors divert only when MaxRetries > 0 — with retries disabled the error surfaces instead of process() re-sending attempt zero. * fix(autopipeline): bound off-pipe FD retries at the window Under a sustained retryable stream (LOADING/READONLY on every reply) each diverted reply spawned an independent normal-path retry goroutine while the reader kept advancing the window and admitting more writes — unbounded goroutines parked in backoff/pool acquisition. retryOnNormalConn now acquires a window-sized semaphore slot before spawning; a full semaphore blocks the READER, which stops advancing the deque, fills the window, and blocks the writer and submitters — end-to-end backpressure, worst case window on-pipe + window off-pipe. No cycle: retries drain on the main pool, independent of the reader. * fix(autopipeline): split FD retry budgets; metric parity; writer priority - run() keeps two budgets, each counting only consecutive failures of its own kind: leaseAttempts (fdLeaseErr/fdDenied) and retryAttempts (fdConnErr tail replays). leaseAttempts resets whenever a session actually runs and both reset on a clean end, so transient lease failures can no longer consume the reconnect budget and leave a genuine drop with zero replay attempts. - The reader's inline completion emits the native error metric (classifyCommandError + pool.GetMetricErrorCallback) for non-retryable Redis errors, matching the normal command path. - The writer runs a non-blocking readerDone priority check before its main select, so randomized select can no longer write another batch to a connection the engine already knows has no reader. * feat(autopipeline): enable full-duplex on the blocking face Drop the !blocking term from the fdOn gate: FullDuplex is now honored on the blocking AutoPipeline face too. No new machinery was needed — submit()'s fd branch already skips setReady on the blocking face (its contract) and processBlocking waits on the returned batch, the same shape as a half-duplex enqueue; the dispGid self-deadlock guard and everything downstream are face-agnostic. A single blocking caller gains nothing (one command in flight — nothing to overlap; only held-conn overhead); the benefit appears with many concurrent blocking callers, whose commands overlap on the shared pipe like the async face (~1 RTT each instead of batch phase-locking). The FullDuplex GoDoc states this and notes the blocking face's per-goroutine ordering is unaffected by the retry divert (each caller waits per command by construction). Adds TestFullDuplexBlockingFace. * fix(autopipeline): FD retry-panic recovery, window cap, carry stop - retryOnNormalConn recovers a panic from process (hooks/encoders are user code): the command gets the panic as its error and always completes, so the caller and a hooked command's host cannot hang — matching the writer, reader and hostHook recovery discipline. - The writer's drain is capped by the remaining window room (min(MaxBatchSize, window - inflight)), so FullDuplexWindow smaller than MaxBatchSize can no longer be blown through by a single batch. - writeCarryChunked observes readerDone between chunks (carry replay and the Close flush both pass it): if the reader exits mid-replay the un-written remainder is pushed to the deque for recovery instead of being written to a reader-less connection. * fix(autopipeline): emit error metrics for failReqs-terminated FD commands Commands terminated through failReqs (lease failure, retry exhaustion, a NoRetry tail, Close) never reach the reader's inline completion, so they were invisible to the native error callback. failReqs now classifies the shared failure once and emits the callback per command. No duration metric: these commands have no meaningful write-to-reply span (many were never written). * fix(autopipeline): FD metric parity, Validate gap, chunked shutdown flush - failQueue emits the native error callback per drained request, mirroring failReqs: on fdLeaseErr/fdDenied both halves of the failure (carry and the accepted channel backlog) are now visible to extra/redisotel-native. - Validate rejects FullDuplex with a user-set NumShards > 1 (one held FIFO connection is a single stream) instead of silently falling back to half-duplex; the cluster wiring's internal contentSharded flag is exempt, preserving the documented cluster fallback. - shutdownFlush re-issues the between-sessions Close backlog in the same MaxBatchSize/MaxBatchBytes chunks as normal FD writes (fdBatchEnd) instead of one unchunked pipeline that ignored MaxBatchBytes. * fix(autopipeline): FD metric parity, Validate gap, chunked shutdown flush - failQueue emits the native error callback per drained request, mirroring failReqs: on fdLeaseErr/fdDenied both halves of the failure (carry and the accepted channel backlog) are visible to extra/redisotel-native. - Validate rejects FullDuplex with a user-set NumShards > 1 (one held FIFO connection is a single stream) instead of silently falling back to half-duplex; the cluster wiring's internal contentSharded flag is exempt, preserving the documented cluster fallback. - shutdownFlush re-issues the between-sessions Close backlog in the same MaxBatchSize/MaxBatchBytes chunks as normal FD writes instead of one unchunked pipeline that ignored MaxBatchBytes; it stops (failing the remainder) after a transport-class chunk failure instead of re-running the retry cycle per chunk against a dead endpoint, and recovers an encoder panic. - The reconnect retry budget resets when a session completed work (including a successful carry replay): a drop after real progress is a fresh failure, not a consecutive one. - The submit queue is capped at min(window, 4096): a buffered channel allocates its capacity eagerly (~5 MiB per engine at the default window); backpressure is enforced by the in-flight deque, so the window keeps its meaning as the in-flight bound. Also compacts the review-round comments in the touched files (invariants and contracts preserved; narration removed). * fix(autopipeline): replay the FD tail after recovered panics Recovered panics (reply decode, batch encode) surfaced as plain errors, so shouldRetry rejected them and the connection-error path permanently failed the whole unacked tail — mostly commands the panic never touched. Wrap them with an errFDPanicRecovered sentinel and treat it (and errFDReaderGone) as replayable: the conn is desynced exactly like a transport error, so the tail replays on a fresh connection with the NoRetry split and MaxRetries budget unchanged. * fix(autopipeline): make the FD retrySem acquire interruptible The reader acquired a retry slot with a bare channel send, so with the semaphore full it parked where it could no longer observe hardClose or the client context, and session teardown waited on readerDone for as long as a slot took to free. The acquire now selects on the semaphore and the autopipeliner context: on Close the request fails with ErrClosed instead of parking the reader past teardown. Outside of Close the wait stays bounded — every slot holder is a retry running through process(), whose timeouts guarantee the slot frees. * fix(autopipeline): address FD review findings Four fixes to the ordered full-duplex engine from PR review: - Recycle the session promptly on a maintenance handoff. The writer now observes cn.ShouldHandoff() (set when the reader drains a MOVING/FAILING_OVER push) and ends the session with a clean recycle, so the pool queues the handoff at Put instead of the connection taking writes to a moving node until idle/max-hold and missing the handoff deadline. - Initialize the held connection with the session-initiating caller's context (values only, via WithoutCancel) instead of context.Background(), so a CredentialsProviderContext can resolve context-derived credentials. FD holds one connection for many callers, so credentials are session-scoped (the first caller's), not per-call — the same limitation any shared/pooled connection has. - Enforce FullDuplexWindow while flushing the backlog on graceful Close: writeCarryChunked now applies the same between-chunk window backpressure as the serve loop, so Close cannot exceed the documented in-flight bound (~2x) when replies are slow. - Preserve accepted retries during Close: an accepted command that came back with a retryable/redirect reply now completes its normal-path retry (tracked by retryWg, sem-free during teardown) even when Close races a full retry-sem, instead of being failed with ErrClosed. Adds TestFullDuplexHandoffRecyclesPromptly and a curConn test hook. * fix(autopipeline): FD review round-2 fixes Follow-up review fixes on the ordered full-duplex engine: - Close no longer hangs on the backlog-flush window wait. writeCarryChunked bounds its wait for reader drain (a stuck reader with ReadTimeout disabled no longer blocks Close forever) and caps each chunk to the remaining window, so in-flight cannot exceed FullDuplexWindow during graceful Close. - The writer polls cn.ShouldHandoff() inside the window gate too, so a MOVING mark under sustained backpressure recycles promptly instead of waiting for max-hold. - Off-pipe retries (retryOnNormalConn) run on the caller's context with cancellation removed, so a CredentialsProviderContext resolves the same identity as the FD session init, instead of context.Background(). - hostHook tracks whether hookDone was awaited, so the panic-recover path never re-awaits it (a closed channel makes the double-await harmless today, but the flag removes the ambiguity). * fix(maintnotifications): skip endpoint DNS detect when mode is disabled (#3969) * Update options.go * Update options_test.go * Update options.go * fix flaky test * fix(maintnotifications): skip endpoint DNS detect when mode is disabled * perf(proto): add zero-copy semantics to scan, remove redundant data c… (#3972) * perf(proto): add zero-copy semantics to scan, remove redundant data conversions in reader * chore: unpin govulncheck version * chore: require govulncheck to use latest go version * chore: pin govulncheck and resolve go-version to stable --------- Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com> * fix(autopipeline): FD review round-3 fixes Follow-up review fixes on the ordered full-duplex engine: - The FD reader's inline per-command error metric now attributes to the caller's request context. The error callback used the background context while the duration callback already used req.ctx; both now share req.ctx (falling back to bg only when nil) so exemplars and context-scoped attributes correlate, matching process(). - The streaming re-auth worker semaphore is sized for the COMBINED ceiling of the main pool plus the dedicated pipeline pool. The same hook drives both, so sizing to only the main PoolSize serialized pipeline re-auths behind PoolSize workers and left pipeline capacity unavailable past reAuthTimeout after a credential rotation. Adds NewManagerWithWorkers for explicit sizing. - PipelinePoolSize doc clarifies the pipeline buffers default to the larger of the regular buffer size and DefaultPipelineBufferSize (64 KiB), so each pipeline connection allocates at least 64 KiB per direction. * fix(autopipeline): FD review round-4 fixes Follow-up review fixes on the ordered full-duplex engine: - Carry replay now stops when the held connection is marked for handoff. A MOVING/FAILING_OVER push drained by the reader marks the conn, but carry replay runs BEFORE the serve loop and never polled ShouldHandoff, so a large replay kept streaming to a moving node and could miss the handoff deadline. writeCarryChunked polls cn.ShouldHandoff() between chunks and in the window gate; on a mark it pushes the unwritten remainder and returns the new errFDConnMoving sentinel, which run() treats as replayable (like reader-gone and panic-recovered) so the whole accepted tail replays on a fresh connection instead of being failed. Pinned by TestWriteCarryChunkedStopsOnHandoff. - Carry replay's window gate is now a predicate loop, matching the serve loop, instead of a one-shot if. inflight.room is a cap-1 channel that can hold a stale signal (the reader popped, the writer refilled without consuming it); a one-shot wait could consume that stale signal with the deque still full, compute zero remaining room, leave the chunk cap at MaxBatchSize, and write past FullDuplexWindow. Rechecking inflight.len() after every wake fixes it. - The between-sessions shutdown flush initializes each chunk with a backlog request's own context (cancellation removed) instead of the engine background context, so a CredentialsProviderContext resolves the right tenant when the flush has to initialize a fresh pooled connection. A chunk can mix callers; the first request in the chunk is the representative, matching how the diverted-retry and session-init paths pick one context for a batch. - The dedicated pipeline pool spills to the main pool on ErrPoolExhausted too, not only ErrPoolTimeout. pipelinePoolOptions clones the client options, so a positive MaxActiveConns smaller than the pipeline PoolSize makes Get return the hard-ceiling ErrPoolExhausted immediately; failing there broke the advertised spill behavior even while the main pool was idle. * fix(autopipeline): FD review round-5 fixes Follow-up review fixes on the ordered full-duplex / pipeline-pool paths: - The pipeline-pool spill no longer charges the Limiter twice. withPipelineConn already calls Limiter.Allow and reports the result in its defer, but the spill routed through withConn, which repeats getConn.Allow + releaseConn.ReportResult. A token-bucket limiter then consumed two tokens per spilled pipeline (or rejected the second admission despite idle main-pool capacity) and circuit breakers counted the operation twice. Spill now goes through a new spillToMainPool helper that uses _getConn/releaseConnToPool, so the operation is accounted exactly once. - Push-notification handler context now reports the pool that owns the connection. With a dedicated pipeline pool, a RESP3 notification received while running an ordinary pipeline arrives on a pipeline-owned connection, but pushNotificationHandlerContext hard-coded ConnPool to the main pool, so a handler that operates on the source pool modified the wrong one. It now resolves ConnPool via poolForConn(cn) (guarding nil cn). - FullDuplex GoDoc corrected: a ProcessHook that calls next but MUTATES the command (e.g. cmd.Args) or sets its result BEFORE calling next races the writer's serialization and the reader's completion, because the write is already queued when the host starts. Only observing hooks (read after next) are unaffected. The prior "hooks that always call next are unaffected" was too broad; gating publication on next would make wire order depend on per-hook pre-next latency and break the ordered guarantee, so the constraint is documented instead. * fix(autopipeline): FD review round-6 fixes - Handoff during carry replay no longer re-executes already-sent commands. When the held connection is marked for handoff (MOVING/FAILING_OVER) while a recovered carry is still being written, writeCarryChunked now stops and returns the UNWRITTEN suffix out-of-band instead of pushing the whole tail for replay. The session drains the already-written prefix to completion (those callers get their real replies), Puts the connection so the maintnotifications OnPut hook performs the seamless handoff (queueHandoff + MarkQueuedForHandoff), and replays ONLY the never-sent suffix on the next lease. Previously the whole tail — the written prefix included — was replayed at-least-once on a still-alive node, re-running non-idempotent commands and missing the seamless OnPut handoff. Pinned by TestWriteCarryChunkedStopsOnHandoff (suffix returned out-of-band, not pushed) and TestFullDuplexHandoffRecyclesPromptly. - The dedicated pipeline pool re-resolves MaxConcurrentDials against its own size. Options.init caps it to the main pool size, so a pipeline pool larger than the main pool (e.g. PoolSize:1 with the default 10-slot pipeline pool) dialed one connection at a time, serializing an initial burst into PoolTimeout or a spill despite idle pipeline slots. An explicitly lower limit is preserved. - FullDuplex GoDoc documents that Options.Limiter is admitted once per held session, not per command — inherent to the held-connection model. Callers that need strict per-command admission/circuit-breaking should not enable FullDuplex. - An FD command diverted to the normal path after a RETRYABLE reply (LOADING/READONLY/TRYAGAIN) no longer exceeds the retry budget by one. The FD socket attempt was already spent, so the divert now starts the normal-path retry loop at attempt 1 (processStartingAt / processWithRetry startAttempt), keeping the total at MaxRetries+1 instead of MaxRetries+2. MOVED/ASK redirects still get the full budget (they did not execute on the FD socket). Pinned by TestRetryStartAttempt. * chore(deps): bump rojopolis/spellcheck-github-actions (#3967) Bumps [rojopolis/spellcheck-github-actions](https://github.com/rojopolis/spellcheck-github-actions) from 0.63.0 to 0.66.0. - [Release notes](https://github.com/rojopolis/spellcheck-github-actions/releases) - [Changelog](https://github.com/rojopolis/spellcheck-github-actions/blob/master/CHANGELOG.md) - [Commits](https://github.com/rojopolis/spellcheck-github-actions/compare/0.63.0...0.66.0) --- updated-dependencies: - dependency-name: rojopolis/spellcheck-github-actions dependency-version: 0.66.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix(autopipeline): FD review round-7 fixes Follow-up review fixes on the ordered full-duplex / pipeline-pool paths: - The pipeline pool preserves an EXPLICIT MaxConcurrentDials that equals PoolSize. Options.init rewrites a 0 to PoolSize, which made an explicit dial cap (e.g. PoolSize:1, MaxConcurrentDials:1) indistinguishable from the default, so pipelinePoolOptions widened it to the pipeline pool size — permitting the dial burst the option was meant to prevent. init() now records whether the value was set explicitly (maxConcurrentDialsSet) and pipelinePoolOptions honors it. - Successful full-duplex connection-error replays now report real retry_attempts. A carried tail that succeeds on a replacement connection reported attempts=1 (and terminal failReqs reported zero), unlike the normal command path. fdReq carries an attempt count (1 at submit, incremented per replay) fed to the OTel duration and error callbacks; the never-sent clean-recycle suffix stays at 1 (first attempt). - The pipeline-pool spill no longer waits out PoolTimeout. pipelinePool.Get blocked up to PoolTimeout (6s default) before returning ErrPoolTimeout, so an eleventh concurrent pipeline could stall for seconds even while the main pool was idle. Added ConnPool.TryGet (non-waiting turn acquisition, still dials when there is room) and the spill uses it, falling back to the main pool immediately at capacity. - Close no longer fails accepted work on a mid-flush handoff. When a MOVING push marks the held conn while Close flushes the backlog, flushBacklogForClose returns errFDConnMoving on a LIVE conn; the graceful path treated it as a hard write failure (hardClose + failReqs + Remove), skipping the seamless OnPut handoff. It now clean-recycles like the serve-loop/carry-replay paths: drain the written prefix, Put the conn (OnPut handoff), and complete the never-sent suffix on another connection via shutdownFlush. * fix(autopipeline): correct cached-divert retry budget and spill count Round-8 review fixes (cursor r3809014600 / codex r3809017107; codex r3809017101). F -- retry budget on the cached full-duplex divert: A cacheable command (for example GET) can ride the FD pipe uncached. On a retryable non-redirect reply (LOADING, READONLY, TRYAGAIN) it diverts with startAttempt=1. processCommand routed it to processCached. processCached dropped startAttempt and ran processWithRetry from 0. So a diverted cache miss ran one attempt more than MaxRetries+1. Now processCommand gives startAttempt to processCached, and processCached gives it to all five processWithRetry calls (four fallbacks and the fetch). A cache hit returns before the retry loop, so startAttempt does not apply to a hit. A -- distinct non-wait pool sentinel (ErrPoolTryFull): TryGet returned ErrPoolTimeout when the pipeline pool had no free turn. So getConn counted every deliberate spill as a pool timeout. It changed Stats.Timeouts and fired the POOL_TIMEOUT and connection-timeout callbacks. But nothing waited, and the operation then succeeded on the main pool. Now TryGet returns ErrPoolTryFull for the non-wait case. A real wait still returns ErrPoolTimeout. The spill path matches the new sentinel. getConn keys its timeout count on ErrPoolTimeout, so it no longer counts a spill. The client also prints a throttled hint on the spill. The hint prints at most one time per 30 seconds per pool-set. It tells the operator to raise PipelinePoolSize. A small PipelinePoolSize that spills by design prints this hint at that rate. Test: TestTryGetReturnsTryFullWithoutCountingTimeout checks the sentinel and the zero timeout count. * fix(autopipeline): count carried FD attempts in the divert budget Round-9 review fixes (codex r3811864747, r3811864754). Divert budget: retryStartAttempt ignored req.attempts. A command carried after a connection error (req.attempts grows on each fdConnErr replay) and then diverted on a retryable reply started the normal retry loop from the base, so its total could exceed MaxRetries+1. The divert now adds req.attempts-1 to the retryable base. A redirect (MOVED/ASK) keeps the base 0, as in r7: the FD socket did not execute the command, and a redirect is not charged to the retry budget. processWithRetry clamps the start to maxRetries, so the command still runs at least once. The common case (req.attempts == 1) is unchanged. Cluster pipeline dials: ClusterOptions.init pre-normalized MaxConcurrentDials to PoolSize. Each node's init then treated the value as explicit, so the pipeline pool used the node PoolSize as its dial cap instead of widening to PipelinePoolSize. A cluster whose pipeline pool is larger than the node pool serialized its pipeline dials. The cluster init no longer normalizes MaxConcurrentDials; the per-node init still normalizes it for the node's main pool. Test: TestClusterPipelinePoolWidensDials. * fix(options): latch MaxConcurrentDials explicitness across reuse Round-10 review fix (codex r3812523456). A caller may reuse one *Options across more than one NewClient call. The first init normalizes an unset MaxConcurrentDials (0) to PoolSize. A second init then recomputed maxConcurrentDialsSet from the normalized value and marked it explicit, so the second client's pipeline pool used the node PoolSize as its dial cap instead of widening to PipelinePoolSize. init now latches maxConcurrentDialsSet on the first call only; clones copy the latched flags. Test: TestReusedOptionsPreserveUnsetDials. * fix(autopipeline): complete Close-handoff suffix on a freed permit Round-11 review fixes (cursor r3813809211, codex r3813782196). D -- Close-handoff suffix: on Close during a mid-flush handoff, session() flushed the never-sent suffix while attempt() still held the pipeline conn and its per-session Limiter permit. A concurrency-capping Limiter at its cap (or both pools saturated) then rejected the second Allow and deterministically failed the accepted suffix. session() now RETURNS the suffix (like the recycle path); run() completes it after attempt() has Put the conn and reported the Limiter, so the flush gets a fresh permit and connection. E -- divert duration metric: a diverted command stamped its operation-duration start on the retry goroutine, after the FD reply arrived, so the metric excluded the initial FD attempt while the attempt count included it. The divert now passes req.writtenAt as the operation start, so the duration spans the FD write to final completion. Retryable replies only; a redirect keeps start 0. * fix(csc): retire conn on pre-command drain desync; cap divert budget Two review fixes. Pre-command push-drain desync (codex r3813782196 follow-up / round-13 audit): _process drains pending pushes before executing a command, then reads the reply on the same conn (and, on the CSC-miss capture path, caches it). The drain runs under a short hard cap separate from the reply read; a fragmented RESP3 frame straddling that cap leaves the reader mid-frame. The error was logged and ignored, so the reply read consumed the residue -- a wrong result, or a wrong-key cache. peekAndProcessPushNotifications now PROPAGATES a mid-frame error (built-in ProcessPendingNotificationsBuffered; custom PeekReplyType-then-propagate) instead of swallowing it, and _process now closes the conn and retries on a fresh one instead of ignoring the error. Divert retry budget (codex r3815644334): when a command's retry budget is already spent on the FD socket (req.attempts > MaxRetries) and it then gets a retryable reply, the reader diverted it and the startAttempt clamp forced one over-budget send. The reader now diverts a retryable reply only while the budget is not spent; otherwise it inline-settles with the true attempt count. * fix(autopipeline): settle FD redirects inline on the standalone path Round-14 review fix (codex r3817201953). The full-duplex engine runs only for a standalone *Client, whose normal path cannot follow a MOVED/ASK -- it neither re-routes to the target node nor sends ASKING. On a redirect the FD reader diverted the command through processStartingAt anyway, which re-hit the same endpoint, wasted a round trip, and returned the redirect regardless. The reader now settles a MOVED/ASK inline (surfaces it to the caller), exactly as a plain standalone command does; only retryable replies (LOADING/READONLY/TRYAGAIN) still divert. A cluster-capable FD/CSC would route the redirect instead -- a follow-up. * fix(autopipeline): do not report Close-cancel to the limiter When Close cancels the full-duplex engine context mid-session, the in-flight attempt fails with context.Canceled. The per-session Limiter report treated that as an upstream failure, so a circuit breaker backed off (or opened) on every graceful Close. Map a Close-driven cancel -- context.Canceled with fd.ap.ctx already cancelled -- to a nil result before ReportResult. Any other error, including a caller-context cancel while the engine is still running, is reported as-is. --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: sadd amr3e <saddamr3e@gmail.com> Co-authored-by: Albert Huynh <huynhalb@gmail.com> Co-authored-by: Vladislav Kotsev <vladygk8@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * perf(autopipeline): halve full-duplex allocations (#3970) * perf(autopipeline): ring-buffer FD in-flight queue The full-duplex unacked in-flight FIFO (fdInflight) was a grow-only slice: the writer appended at the back and the reader dropped the front by re-slicing, so the popped-off prefix was never reused and the backing array reallocated + copied on every window churn — the single largest allocation site on the FD hot path (31% of per-op bytes at 2048 sync callers). Replace it with a ring buffer that reuses one backing array for the life of the session and grows only when live count exceeds capacity (presized to MaxBatchSize). All observable semantics are unchanged: frontBatch returns the same wrap-aware copy into the caller's buffer, advance keeps the load-bearing slot-zeroing (so drained entries don't pin cmd/ctx/batch), takeRemaining unwraps into the same contiguous ordered tail. Result (standalone bench, 1ms, 90% GET / 10% SET, sync callers): 770 -> 530 B/op at 2048 callers. Throughput and latency unchanged. Adds a randomized ring model test (push/advance/frontBatch/takeRemaining vs a reference slice, driving snapshots and growth across the wrap seam). * perf(autopipeline): pool blocking-face batch The blocking full-duplex face allocated a fresh apBatch (struct + done channel) per command — 23% of per-op bytes on the FD hot path. That face is the one path where a batch is a pure single-waiter completion signal: it never setReady()s the command (so the batch is invisible to await/readyBatch/resultReady) and discards it right after Wait returns. Pool those batches. Pooled batches use a buffered(1) done signalled by a non-blocking send (reusable, unlike a closed channel); close() branches on an immutable pooled flag so every completer (reader, failReqs, shutdownFlush, flushBacklogForClose) funnels through the one entry point; the batch is recycled in processBlocking after Wait, gated on pooled && dispGid==0. The async face and every shared flush batch keep the close()-once channel untouched. Result (standalone bench, 1ms, 90% GET / 10% SET, sync callers): 530 -> 353 B/op at 2048 callers (770 -> 353 with the preceding ring change, -54%); CPU/op 7.9 -> 6.6 us at 256 callers. Throughput and latency unchanged. Adds a concurrent batch-pool reuse test (get/signal/wait/recycle under -race). * chore(autopipeline): silence unused lint on newFDInflight The ring change moved production construction to newFDInflightCap; the no-arg newFDInflight wrapper is now called only from tests, and the repo runs the unused linter with tests:false, so CI flagged it. Mark it //nolint:unused with the same rationale as peakLen, which is unused for the identical reason. * test(autopipeline): sparse-traffic pooled-batch coverage Adds TestFullDuplexSparseTrafficPooledBatch: drives the blocking full-duplex face with traffic spaced past the idle timeout, so the engine returns its connection to the pool and re-leases on every command — each command runs in its own single-entry batch, repeatedly. This is the path where the pooled completion batch and the idle-return/re-lease machinery meet, guarding against a mis-delivered wakeup from a recycled batch under intermittent execution. Checks ECHO round-trips its own token (no misattribution) and per-caller INCR is strictly sequential (no lost or duplicated completion), with a watchdog so a dropped completion fails as a timeout rather than hanging. Complements the existing idle-return and max-hold-recycle tests, which cover the connection lifecycle but not per-command result correctness under repeated re-lease. * fix(autopipeline): guard advance on empty ring fdInflight.advance clamped n down to the live count but then still computed head % len(buf), which panics on a never-grown ring (nil backing buffer) when the clamp leaves n==0. Return right after the clamp, restoring the slice implementation's tolerance of advance on an empty queue. Adds TestFDInflightRingAdvanceEmpty (advance on a nil-buf ring, and over-advance on a drained one). * fix(autopipeline): bound FD inflight ring and recycle rejected batches Two allocation fixes on the full-duplex path (codex review): - The session presized the in-flight ring to MaxBatchSize, an uncapped soft per-flush threshold. A large MaxBatchSize with a small window then allocated that many entries up front -- tens of MB, or an OOM -- and again after every idle/recycle. Live in-flight is bounded by the window (the writer blocks once in-flight reaches it and caps each drain by the remaining window room), so the ring is now presized to min(MaxBatchSize, window): capped by the window (the peak) to kill the OOM, and by MaxBatchSize so the small-batch default is unchanged. grow() remains the backstop up to the window as load ramps. - The hook-free blocking face drew a completion batch from the pool before admission, but the fd.closed, ctx.Done(), and engine-close reject paths returned without recycling it. Under backpressure, repeated context timeouts discarded one pooled batch and channel per rejected command, defeating the pool. Each post-acquisition reject path now returns the batch via putFDBlockingBatch (a no-op for the non-pooled shape). * perf(autopipeline): gated fast-submit for the full-duplex path (#3980) Adds AutoPipelineOptions.FullDuplexFastSubmit (opt-in): the submit hot path tries a non-blocking channel send before the blocking three-arm select, cutting the selectgo cost (~40% of submit CPU) that dominates at high producer counts on low-RTT links. The fast path is queue-depth gated -- it fires only while the submit channel is under 10% full -- so once the queue bursts deep the fair blocking select takes over and the p99 tail is preserved. The non-blocking send runs under the SAME submit RLock as the blocking send, so it cannot land in the channel after the shutdown drain, and the pooled blocking batch is recycled on the reject paths. Measured (FD blocking face, latproxy, >=1k concurrent callers): +15-33% throughput at 1ms RTT and +6-18% at 5ms, tail equal-or-better; a no-op on high-RTT (WAN) links and at low concurrency. Off by default. Pinned by TestFullDuplexFastSubmitCloseRace. * fix(autopipeline): fail FD session on push-drain desync On the full-duplex reply path the reader drained RESP3 push frames ahead of each reply and only LOGGED a drain error before calling readReply. A custom PushNotificationProcessor can return after consuming part of a frame, leaving the reader desynced, so the next readReply would read shifted bytes and misalign every later reply in the FIFO — a silent wrong-result/wrong-key-cache cascade. The shared pre-command drainer already treats a custom-processor error as connection-fatal. Propagate the drain error instead: wrap it as errFDPushDrainFailed and fail the session. The error is added to the run() replay predicate (alongside errFDReaderGone / errFDPanicRecovered), so the unacked tail is REPLAYED on a fresh connection rather than failed — the tail is mostly commands the drain never touched, and NoRetry commands are still protected by the existing split. Addresses the codex P1 push-drain finding on #4002. * fix(pool): TryGet pipeline spill, retry drain desync Two request-path fixes from the #4002 review: withPipelineConn acquired the dedicated pipeline pool with Get(ctx), so a saturated pipeline waited up to DefaultPipelinePoolTimeout (100ms) and recorded a spurious Stats.Timeouts before spilling to the main pool. Use TryGet: it returns ErrPoolTryFull at once (no wait, not counted as a timeout), so the pipeline spills immediately. The spill trigger now matches on ErrPoolTryFull / ErrPoolExhausted. The pre-command push-drain failure closed the conn and set retryTimeout, but retryTimeout only makes shouldRetry retry TIMEOUT errors — a custom-processor sentinel returned shouldRetry=false and the unrelated command failed instead of retrying on a fresh conn. Add a forceRetry flag, set only when the conn was already closed (so replay is safe), and OR it into the retry decision. Addresses the codex TryGet and force-retry findings on #4002. The cluster/ring PoolStats finding was verified already-fixed on this head (both fold the pipeline pool), and the FD lease main-pool spill is deferred (replied on-thread). * fix(autopipeline): bound FD tail retries per-command The full-duplex reconnect loop reset the session-level retryAttempts to 0 on any session progress (advancedTotal > 0), then gated tail replay on that counter. Under a flaky peer that acknowledges some replies and then drops the connection, the unacked tail therefore got a fresh MaxRetries budget after every partial success, so a mutating command whose reply is repeatedly lost could re-execute far beyond MaxRetries+1. Bound the replay by the head command's own attempt count instead. unacked[0] is the oldest unacked command (carried longest, highest attempts) and each replay bumps every carried command's attempts together, so gating on unacked[0].attempts caps the whole tail at MaxRetries+1 total attempts regardless of interleaved progress. retryAttempts still drives the backoff escalation only. Also keep the push-processor error in the chain: the errFDPushDrainFailed wrapper now uses %w for both causes (was %v for the processor error), so callers can still errors.Is / errors.As their processor error after retries are exhausted. Addresses the codex retry-budget and error-chain findings on #4002. * docs(autopipeline): clarify FullDuplexFastSubmit tradeoffs Restructure the FullDuplexFastSubmit doc comment into What / Benefit / Drawback so the fairness cost is explicit: the non-blocking send lets a producer that finds room jump ahead of producers already blocked on a full channel, which under a deep queue would starve them and inflate p99. State that the fdFastSubmitGatePct depth gate is what bounds it. Also document that ordering is unaffected: the enqueue is synchronous even on the async (Submit) face (only the reply is deferred), so a caller's command N is on the ordered channel before submit returns and before submit(N+1) starts — a goroutine's own commands keep program order regardless of path. Fast-submit changes only enqueue wait time and inter-producer interleaving (fairness), never intra-caller order. No code change. * fix(autopipeline): treat FD push-drain as fatal reply Three review findings on the full-duplex reply/shutdown paths. 1) Push-drain classification (regression from the earlier %v->%w wrap). The reader wraps a push-drain failure as errFDPushDrainFailed with %w so callers keep the processor cause in the chain. But the reply classifier used !isRedisError, which walks the chain via errors.As, so a drain error wrapping a Redis-typed or retryable cause was NOT treated as fatal: it fell through to the retryable-reply divert and was handled as that command's reply, leaving the unread frame in the stream and shifting every later reply (FIFO desync) — the exact failure the drain propagation prevents. Add fdReplyIsFatal: errFDPushDrainFailed is always fatal (abort the session, replay the tail) regardless of the wrapped cause, while a plain Redis error stays a normal inline reply. This restores the pre-command/in-session symmetry (pre-command drain -> forceRetry; in-session drain -> fatal). Unit test TestFDReplyIsFatalPushDrain pins it, including the %w chain staying intact. 2) Shutdown retry budget. shutdownFlush ran the carried tail through processPipeline, whose retry loop restarts at attempt zero and grants another MaxRetries+1 executions; a carried command whose budget was already spent (attempts can reach MaxRetries+1 on the run() replay path before a Close/ctx-cancel routes it here) could then far exceed its budget. Fail budget-exhausted carried commands directly instead of re-sending; freshly queued work is untouched, and NoRetry chunks are already not retried by generalProcessPipeline. 3) In-flight ring retention. curInflight kept the last session's grow-only deque (up to fd.window, ~MiB at the default) resident through the idle gap between sessions and after the engine exited. Release it before parking for the next command and on run() exit; both are safe against the fdConnErr progress read. Addresses the cursor push-drain finding and the codex shutdown-budget and in-flight-ring findings on #4002. * fix(autopipeline): bound FD replay per-command, not by head Last round bounded the full-duplex replay tail on the head command's attempt count (unacked[0].attempts <= MaxRetries) to stop a flaky peer from re-executing the tail past its budget. That over-corrected: when the oldest command has spent its budget, the head-only gate also denies retries to NEWER commands written behind it in the same session. With MaxRetries: 3, command A can reach its 4th attempt, command B is then written for the first time behind it, and a drop before either reply fails BOTH — B gets none of its configured retries. Partition the tail by each command's OWN attempt count instead: fail the commands that have spent their budget (attempts > MaxRetries), keep the rest eligible to replay, then apply the existing NoRetry split to the eligible suffix. Carried commands are the oldest (written first, attempts bumped together), so the exhausted set is the leading run and the eligible suffix keeps FIFO order. Exhausted commands are failed with the real connection error (aerr), preserving the MaxRetries: 0 behavior. Each command is failed exactly once. The pure fdPartitionByBudget helper (shared with shutdownFlush) is unit-tested for this case. Addresses the codex per-command-retry finding on #4002. * refactor(cmd): thread explicit retry bound into pipeline flush generalProcessPipeline hard-coded its retry loop to c.opt.MaxRetries. Add a maxRetries parameter (both existing callers, processPipeline and processTxPipeline, pass c.opt.MaxRetries, so their behavior is unchanged) and a processPipelineRetries entrypoint. The autopipeliner's shutdown flush passes each carried command's REMAINING budget so replaying it on Close honors its configured MaxRetries+1 total. * fix(autopipeline): recover Close backlog, cap shutdown retries Two Close-path findings. 1) Never-sent backlog on a dead conn (cursor eAn1r). On the graceful-Close flush, when the connection is already dead the fd.ch backlog had been written into the in-flight deque by writeCarryChunked and then failed wholesale with errFDReaderGone, even though those commands never touched the broken socket. Return the unacked tail from that path as fdConnErr instead of failing it inline: it flows through run()'s standard recovery (per-command budget partition + NoRetry split) and, because ap.ctx is cancelled, the eligible prefix is executed by shutdownFlush on a fresh connection. The never-sent backlog is completed by execution; a written-but-unacked NoRetry command is still failed by the split and never re-executed. takeRemaining holds only UNACKED commands, so nothing already settled is re-run. 2) Shutdown retry budget (cursor). shutdownFlush ran carried commands through a full processPipeline retry loop, so a partially-spent command could get another MaxRetries+1 executions and exceed its budget. Flush each carried command with its OWN remaining budget instead: carry is post-bump (a command carried at attempts=A has completed A-1 executions), so it may run MaxRetries+1-A more times (fdCarryRemainingRetries); a spent command (attempts >= MaxRetries+2) is dropped, and a never-run command (attempts==1, initial or lease/denied Close race) still gets the full budget. flushCarryBudgeted groups equal-attempt runs and flushes each with its bound; fresh queue work keeps the full budget and flushes after carry (FIFO). This also fixes an off-by-one in the earlier drop (attempts > MaxRetries) that discarded a command with exactly one execution left. The pre-lease attempt bump can only cause an early DROP on Close, never an extra execution — within the terminating-Close best-effort contract. fdCarryRemainingRetries is unit-tested (TestFDCarryRemainingRetries), including the MaxRetries==0 rows. Asserting end-to-end execution counts needs a counting client harness the suite lacks; the shutdownFlush integration tests cover completion. * style(autopipeline): check flushReqs error on shutdown flush errcheck flagged the final fd.flushReqs call in shutdownFlush. Its error is already handled inside flushReqs (it fails the remainder on transport failure) and there is nothing after it, so explicit-ignore the return value with _ =. * fix(autopipeline): writer alloc cap, reply-deadline, retry split Three pre-existing full-duplex findings from the review. Cap writer scratch (eEHir): the serve-loop drain scratch was sized to MaxBatchSize, but the writer can never have more than fd.window commands in flight, so a large MaxBatchSize with a small window over-allocated (up to OOM). Size it min(fd.maxBatch, fd.window), matching the in-flight ring. Arm reply deadline post-write (eEHiv): writeBatch pushed the batch into the in-flight deque BEFORE WithWriter, so the reader could arm its per-reply ReadTimeout while a slow user encoder (BinaryMarshaler) was still serializing — timing out a healthy connection and triggering a replay that duplicates a mutating command. Publish only after a clean serialize+flush; a deferred push still adds the batch on an encoder panic or partial write so conn-error recovery settles every caller. writeBatch is called synchronously from the serve loop, and closeGraceful runs only after it returns, so there is no window where the reader exits before the batch is published; a reader error during the write still recovers the pushed batch via the tail path. Split retry policy on shutdown flush (eEHim): flushReqs chunked purely by size, so a NoRetry command (e.g. RawWriteToCmd) sharing a chunk with retryable commands made generalProcessPipeline disable retries for the whole chunk (cmdsContainNoRetry), stripping the retryable ones of their budget. Break each chunk at the first retry-policy change, isolating NoRetry commands like the half-duplex dispatcher. The divert-vs-session-limiter finding (cursor eEUTx) is replied as a pre-existing follow-up: honoring it would thread a limiter-skip through getConn and the shared process path, disproportionate to a Medium edge that needs a concurrency-capping limiter at capacity. * fix(autopipeline): cancel cached engine when shared pools close A WithTimeout/WithReadTimeout clone shares the base client's pools and onCloseHooks registry but is a distinct wrapper with its own autopipeliner slots. Closing the clone set the shared apClosed flag and ran the shared onClose hooks, but never touched the ORIGINAL wrapper's cached autopipeliners — so their full-duplex run() loop, and the half-duplex shard flushers (same root cause), parked on ap.ctx forever. New submits were rejected via isClosed(), so nothing could wake them: a leaked engine per pool-sharing clone close. Register a shared onClose hook at ap creation that cancels the cached ap's engine context. Any sharer's Close now cancels it, so run()/the flushers exit. Cancel (not Close) so the clone's Close does not block on the original's drain; idempotent with the wrapper's own Close, which still closes its aps directly. onClose.run() fires before the pools close, so run()'s shutdown flush still completes accepted work. Two hook ids (blocking + async) cover the at-most-two cached instances per client; cluster/ring pass a nil shared flag and are unaffected. TestFullDuplexEngineReapedOnClonePoolClose asserts the engine goroutines are reaped after a WithTimeout-clone Close (the existing rejection test does not, since a parked engine still refuses submits). * fix(pool): keep TryGet non-blocking on occupied dial slots TryGet acquires a pool turn without waiting, but when PoolSize > MaxConcurrentDials and every dial permit is held, it then blocked on the dialsInProgress send in queuedNewConn (and, the same class, on the drainer wait) until an in-flight dial finished its whole retry sequence or the ctx expired — violating the non-blocking contract a pipeline spill relies on. Thread the existing wait flag into queuedNewConn and the drainer-wait branch: when wait is false, give up immediately with ErrPoolTryFull (freeing the turn) instead of blocking. Get (wait=true) is unchanged. The straggler-hold gate (HasFreeCapacity) already guarded this exact saturation; TryGet now matches. * fix(conn): retire desynced conn on pipeline drain or panic Two connection-disposition gaps on the shared Pipeline/TxPipeline path. Push-drain desync: generalProcessPipeline and pipeline/txPipelineProcessCmds only LOGGED a pre-write processPushNotifications error and kept writing. A custom processor can consume part of a fragmented RESP3 push frame, so continuing decodes leftover bytes as replies and shifts results across commands. Propagate the error instead, wrapped as errConnUnusable. Panic mid-write: a user encoder panicking inside writeCmd unwound through withConn/withPipelineConn with fnErr still nil, so their deferred release Put the possibly partially-written (desynced) conn back into the pool. Recover, mark the conn errConnUnusable, then re-panic to preserve the caller's panic. errConnUnusable is a new marker releaseConnToPool removes on (alongside isBadConn), so both cases retire the conn instead of poisoning the pool. The batch fails with the error rather than retrying — shouldRetry is deliberately left untouched, so the caller sees an error instead of shifted results. This changes what Pipeline.Exec can return for all users; no MOVED/ASK/ErrClosed matcher mis-routes it (verified in the cluster and maintnotifications/CSC paths — they inspect typed per-command errors). * fix(autopipeline): refund attempt for never-sent carry suffix run() bumps the whole carry's attempts before re-issuing it, charging every command for a send. When the reader dies or the write fails mid-replay, the never-written suffix was pushed into the in-flight deque still carrying that bump, so run()'s per-command budget partition could declare it exhausted a replay early (e.g. with MaxRetries: 1 a suffix command failed after only its original execution). Refund the bump (fdRefundUnsentAttempt, floored at 0) for the never-sent suffix at each dead-conn / write-error push site; the next replay re-bumps it when it is actually sent, so accounting is net-flat. The handoff (errFDConnMoving) path is untouched: it returns the suffix out-of-band and fdRecycle does not re-bump, so no double-refund. TestFDRefundUnsentAttempt pins the helper. * fix(autopipeline): unique per-engine close-hook id The round-7 shared-pool close hook (which cancels a cached engine when any pool-sharing wrapper closes) used a per-slot CONSTANT id and registered per call outside the getOrCreateAutoPipeliner mutex. Two problems: - A client and a WithTimeout clone that each cache the same face share the onClose registry, so the second registration overwrote the first; closing one sharer then cancelled only one engine and leaked the other's run loop / shard flushers. - Registering outside the mutex raced concurrent first-callers (and registered a hook per caller, only one of which ap.Close could later unregister). Register the hook ONCE, inside getOrCreateAutoPipeliner under its mutex, on the FRESH build, with an id made unique per engine (apCloseHookSeq). ap.Close unregisters by that id so the registry stays bounded and a closed engine leaves no stale callback. Cluster/ring pass a nil onClose (they use a nil shared flag and have no clone-close leak). TestFullDuplexEngineReapedForClientAndClone asserts closing a clone reaps BOTH the client's and the clone's cached engines. * fix(autopipeline): cancel raced engine, skip FD shard buffers Two autopipeline construction findings. Close race (eJm7t): a WithTimeout clone creating an engine while another pool-sharing wrapper closes could pass getOrCreateAutoPipeliner's entry sharedClosed check, build the engine, and register its close hook only AFTER the concurrent onClose.run snapshotted its callbacks — so the hook never fired and the engine's goroutines parked on already-closed pools. baseClient.Close sets sharedClosed before running onClose, so re-check it after registering (still under the slot mutex): if set, cancel the just-built engine, detach the possibly-missed hook, and return ErrClosed instead of caching a doomed instance. Skip FD shard buffers (eJm71): full-duplex submissions go straight to the FD engine and no shard flusher runs, yet the constructor preallocated each stripe queue to MaxBatchSize (times apEnqueueStripes on the blocking face). MaxBatchSize: 1_000_000 therefore allocated ~8M interface slots up front regardless of FullDuplexWindow — tens of MB or an OOM before any command. Skip the prealloc when fdOn; the queues are never enqueued to and Len reads an atomic, so a nil queue is safe. TestFullDuplexSkipsShardQueuePrealloc pins it. * fix(autopipeline): wait for engine flush on shared-pool close The shared-pool close hook (round-9) cancelled the cached engine and returned, so a pool-sharing WithTimeout clone's Close ran baseClient.closeResources — closing the shared pools — while the engine's shutdown flush was still running against them, failing accepted work instead of completing it. Extract cancelAndDrain: cancel the engine and wait (bounded by the drainAll backstop) for its flushers / shutdown flush / shard sweep, WITHOUT flipping ap.closed or detaching the hook. The hook now calls it, so a sharer's Close waits for the flush before the pools close (pool still open during onClose.run). ap.closed stays false on that path — the engine is rejected through the shared-closed flag, and a later owner Close still runs the full teardown — so TestSharedClosedRejectsEveryEntryPoint's contract holds. ap.Close layers the closed-CAS + hook unregister on top of cancelAndDrain. The wait is bounded, so a wedged flush cannot hang the closing wrapper. Addresses the codex shared-autopipeliner-shutdown finding on #4002. * fix(sentinel): stop failover from rebuilding sentinel after close The shared-pool autopipeliner close hook now drains the engine (cancelAndDrain), and that drain can dial. On a NewFailoverClient the failover Close hook runs before the autopipeliner hook (onClose registration order: failover registers at construction, the engine hook lazily), so a drain-time dial through masterReplicaDialer -> MasterAddr rebuilt the SentinelClient + pubsub after the only failover cleanup had already run, leaking them past the pool teardown. Add a closed flag to sentinelFailover, set under mu by Close. MasterAddr and replicaAddrs return pool.ErrClosed instead of running the sentinel discovery fan-out once closed, so a late dial cannot resurrect sentinel resources. Close stays idempotent. TestSentinelFailover_ClosedDoesNotRebuildSentinel pins it using the returned error as the oracle (ErrClosed vs the unreachable-fan-out error), needs no sentinel infrastructure. Addresses the codex drain-autopipeliners-before-the-failover-hook finding on #4002. * fix(autopipeline): sent-aware NoRetry gate, tail recovery, retry cap Five review fixes on the full-duplex replay core, from the #4002 deep review. Sent-aware NoRetry gate: fdReq gains a sent marker, stamped in writeBatch before the flush is attempted (a partial write or encoder panic still counts as possibly-on-the-wire). fdFirstNoRetry now splits the tail at the first SENT NoRetry only — a NoRetry command recovered from a dead connection's never-written backlog is replayed (its first send) instead of failed, so the caller no longer gets an error for a command the server never saw. A sent NoRetry still fails, never re-sends. Pinned by TestFDFirstNoRetrySentGate. Stranded-tail recovery: the graceful-Close final-drain branch failed the whole unacked tail on a reader error; it now returns the tail (plus the never-sent handoff suffix) as the fdConnErr recovery set, so run()'s budget partition and the sent-NoRetry split decide, and shutdownFlush executes the eligible prefix on a fresh connection — same contract as the backlog-flush failure branch. Recycle-suffix refund: the recycle-then-reader-fail branch re-entered fdConnErr recovery without refunding the never-sent suffix's optimistic attempt bump, so with MaxRetries=1 a command could be declared budget-exhausted after one real execution. Both recovery branches now build their set via fdRecoverTail (append + refund the suffix only). Pinned by TestFDRecoverTailRefundsSuffixOnly. Retry-goroutine cap: retrySem was sized to the FD window (default 65536) — a retryable-reply storm (LOADING during a restart diverts every reply) could park tens of thousands of goroutines on the main pool. Clamp to 2x PoolSize (the divert path serializes on the pool anyway); the reader blocking on a full sem is the designed backpressure, engaging earlier. Live-replay window gate: writeCarryChunked's bounded stuck-wait escape assumed ap.ctx was always cancelled, but the function also runs the ordinary session- start carry replay; a slow first reply (>1s) there dropped the FullDuplexWindow bound for the rest of the carry. The live path now waits like the serve loop (reader death bounds it via ReadTimeout), keyed on ap.ctx so a concurrent Close switches to the bounded escape. Addresses the never-sent-NoRetry, stranded-tail, refund, retrySem and window-gate findings on #4002. * docs(cluster): document MaxConcurrentDials normalization ClusterOptions.init deliberately leaves MaxConcurrentDials un-normalized (the per-node Options resolve it so the pipeline pool can widen its own dial cap), but the field doc claimed '<= 0 defaults to PoolSize', which reads as an init-time guarantee. State where resolution happens and that introspecting the field after init still reports the zero value. Addresses the codex dial-limit-normalization note on #4002. * feat(autopipeline)!: per-batch Limiter admission in full-duplex Move Options.Limiter in the full-duplex engine from per-SESSION to per-BATCH. Before: attempt() paid Allow once at conn lease and ReportResult once at release, so one permit spanned the whole session. Now: writeBatch pays Allow before each written chunk and reports the write outcome (nil on success) right after the flush. Why: a lease-long permit pinned a circuit breaker's half-open probe budget for the session lifetime (probe starvation) and produced near zero failure-signal density (one report per session, however long it ran). The Limiter's unit everywhere else in the client is one connection-acquiring wire operation — a single-command attempt, a pipeline exec, a half-duplex flush — and the FD equivalent of that unit is one written chunk. Deny contract: a denied Allow fails ONLY that chunk's commands, with the Limiter's own error verbatim (errors.Is-able); the connection is untouched and the session survives, so the next chunk pays Allow again — fast-fail while a breaker is open, automatic resume when it closes. The fdDenied session result and its run() arm are gone (unreachable). Strict pairing: every successful Allow gets exactly one ReportResult (including on an encoder panic converted to a write error); a denied Allow reports nothing. Reply errors do not feed ReportResult — a server that answers is healthy; transport failures surface through the next chunk's write and through diverted retries, which pair via getConn. Pinned by TestFDLimiterPerBatch; TestFullDuplexLimiterPerSession is now TestFullDuplexLimiterPairing (per-batch balance). BREAKING CHANGE: Options.Limiter is now consulted once per written batch in the full-duplex autopipeline, instead of once per FD session. * fix(autopipeline): per-command sent-stamp, hook snapshot, drain-error stamping Three review fixes on the full-duplex path plus one documented non-fix. eTWPc — writeBatch stamped sent/writtenAt for the whole chunk BEFORE the WithWriter serialize loop, so an early encoder panic left the never-serialized suffix marked sent; fdPartitionByBudget (keys on attempts) then failed a never-sent NoRetry without its first execution, and at MaxRetries=0 failed the whole suffix. Stamp per-command inside the loop (only commands the serializer reached are sent), and refund the optimistic attempt for every !sent command before the recovery pushBatch. eR1Nh — a push-drain error that wraps a redis.Error (a custom PushNotificationProcessor) was classified by isRedisError, so generalProcessPipeline skipped setCmdsErr on an unwritten pipeline: direct Pipeline/TxPipeline returned an exec error while every command reported nil, and the FD shutdown flush could complete them as successes. errConnUnusable now takes precedence (pipelineErrShouldStamp), so the commands are stamped. eTWPh — with hooks installed, submit only recorded that hooks exist and hostHook loaded the live chain later, so an AddHook between accept and execution retroactively wrapped an in-flight command. submit now snapshots the composed process-hook chain (one atomic load) and hostHook uses that snapshot, honoring "hooks are observed only by commands submitted afterward." eTWPa — a ProcessHook that calls Close from its own hook-host goroutine stalls until the 30s close backstop. Not code-fixed: a reentrancy guard is unsafe (cancelAndDrain is not once-only on the shared-pool path, and an early hostWg.Done races submit's hostWg.Add — a WaitGroup panic). Documented as an unsupported anti-pattern on the FullDuplex GoDoc and at the hostWg.Wait site; the backstop bounds it. Tests in autopipeline_fd_reviewfixes2_test.go, each with a confirmed red-check. * fix(autopipeline): report reply-side outcome to the limiter The per-batch full-duplex Limiter reported each chunk's WRITE outcome, so a peer that accepts writes but closes/times out before returning replies (and whose replay writes also succeed) fed the Limiter only successes — a circuit breaker never opened during a read-side failure, contrary to the contract that ReportResult reflects the allowed operation's result. Report the chunk's REPLY-SIDE outcome instead. Allow() stays at write admission; a clean write mints one fdLimiterReport obligation (CAS-guarded, fires ReportResult at most once) attached to the chunk's LAST in-flight req. The reader settles it nil once the reply lands and is not transport-fatal (redis.Nil, WRONGTYPE, MOVED are reply-level, still nil). A write failure/panic reports the error immediately and pushes no obligation. Every fdConnErr recovery return (settleTail at the four takeRemaining sites) and the graceful-Close backlog flush settle any carried obligation with the transport error and clear it; carry replay re-runs through writeBatch and mints a fresh Allow + obligation, so exactly-once 1:1 Allow/ReportResult pairing holds across success, read failure, write failure, panic, Close, and connection-move replay. Diverted retries keep their own getConn pairing. Addresses the report-read-side-transport-failures-to-the-limiter finding (ed53z) on #4002. * fix(autopipeline): refund replay suffix, recover limiter panic, serialize drains Four review findings, rebuilt on the master-merged base (has #3998 WaitClosed). F1 — writeBatch refunds a replay suffix by an ATTEMPT-LOCAL marker (the index serialization reached this call), not the sticky lifetime sent flag. On a second-session replay where an earlier encoder panics, the never-reached suffix is now refunded, so at MaxRetries=1 it is not declared exhausted after only its first execution. First-session behavior is unchanged. F2 — the FullDuplex GoDoc now describes the reply-side limiter lifecycle (permit released when the chunk's replies drain, reply-level errors report nil, transport failures reported via settleTail) so a custom Limiter author sees the real contract. F3 — cancelAndDrain runs its body exactly once via drainOnce, memoizing the result into closeErr (single writer, inside the Once). Two closers can reach it for one engine — an explicit Close racing the shared-pool close hook, which leaves ap.closed false — and two concurrent drains raced on closeErr and could trip a WaitGroup Add-concurrent-with-Wait panic in drainAll's staged sweep. WaitClosed reads closeErr after closeDone, now race-free. F4 — Limiter.Allow is user code on the engine writer goroutine; a panic escaped fd.run and crashed the process. fdAllow wraps it in a recover boundary that fails the still-unwritten chunk (errFDPanicRecovered) instead — no permit granted on panic, so no ReportResult (strict pairing). Tests + red-checks in autopipeline_fd_reviewfixes2_test.go. Addresses the attempt-local-refund, reply-side-limiter-doc, serialize-concurrent-drains, and recover-limiter-panic findings on #4002. * fix(autopipeline): recover panics from Limiter.ReportResult A custom Limiter.ReportResult that panics ran on the FD engine's background goroutines with no recover boundary. On a write-failure report it escaped fd.run and crashed the process; on the reply-success path the reader's broad recovery caught it and treated it as a session failure BEFORE advancing the request, so a command whose reply was already consumed got replayed — an INCR could execute twice. (Limiter.Allow panics were already recovered via fdAllow; ReportResult was left unguarded, and the reply-side reporting from ed53z made it fire on these paths.) fdLimiterReport.settle is the single choke point for every FD ReportResult (the reader success path, writeBatch's write-failure defer, and settleTail all route through it). Wrap its ReportResult call in a recover that logs and swallows the panic; the CAS already fired, so strict Allow/ReportResult pairing holds and the command/reply proceeds exactly as if ReportResult had returned. A faulty limiter can no longer crash the engine or turn a completed reply into a replay. Tests + red-checks in autopipeline_fd_reviewfixes2_test.go (the reply-success no-replay case is the INCR-twice guard). Addresses the limiter-report-panic finding (codex + cursor, High) on #4002. * fix(autopipeline): recover metric-callback panics; fix two doc contracts The FD reader invokes the user-settable OTel duration + native error callbacks after applying a reply but BEFORE advancing the req out of the in-flight deque. A panic in either escaped to the reader's broad recovery, which treats an unadvanced req as an unacked tail and replays it — a mutating command runs twice. Move those callbacks into reportReplyMetrics, which recovers (logs + swallows) a callback panic; the reply outcome is already decided, so the caller completes normally and the reply is never replayed. Mirrors the fdLimiterReport.settle recover. Pinned by TestFDReportReplyMetricsRecoversCallbackPanic (red-checked). Docs (behavior already correct; the GoDoc was stale): - FullDuplex: a standalone full-duplex client surfaces a MOVED/ASK redirect to the caller as the command's error (a standalone Client cannot route it) — it is not re-run through Process. Only retryable replies (LOADING/READONLY) divert. - PipelinePoolSize: a saturated pipeline pool spills to the main pool IMMEDIATELY via a non-blocking TryGet; there is no DefaultPipelinePoolTimeout grace wait before the spill. Addresses the callback-panic-replay (Medium), redirect-replay-doc, and pipeline-spill-timing-doc findings on #4002. * fix(autopipeline): guard all FD metric callbacks; unblock self-reading duration cb Two engine-safety fixes on the full-duplex metric callbacks. A panicking user MetricError callback in failReqs/failQueue ran on the engine goroutine with no recover — a limiter denial, lease failure, or retry exhaustion could crash the process (or deadlock hostWg.Wait while callers block on hookDone), leaving accepted commands unsettled. Route every FD metric-callback emit (reportReplyMetrics duration+error, failReqs per-req, failQueue per-req) through one recover-guarded emitMetricsGuarded; SetErr precedes it and complete() follows, so a panic still settles each command and wakes its caller. Per-req placement keeps a panic on one req from stranding the rest. A custom RecordOperationDuration that reads its OWN command (cmd.Err()/String()) blocked on batch.done because the FD reader completes the req only after the callback returns, and the reader was not the batch executor — wedging the reader and the FD window. reportReplyMetrics now registers the reader as the batch executor around the callback, so the accessor guard returns the just-set view without blocking (same escape a dispatch hook uses). No post-complete reorder — that would race a hook host that may free the command. Tests + red-checks in autopipeline_fd_reviewfixes3_test.go. Addresses the metric-callback-panic (High), error-recorder-panic, and blocking-duration-callback findings on #4002. * fix(autopipeline): abort ordered shutdown flush on panic/unusable-conn; harden release drain Four findings on the Close/shutdown and release paths. F1 — flushReqs recovered an encoder panic during the Close-time carry flush but had an UNNAMED return, so it returned nil; flushCarryBudgeted/shutdownFlush then treated the failed group as success and ran later ordered groups and the fresh queue after an earlier command never completed. Named return set in the recover defer aborts the ordered flush like a transport failure. F2 — flushReqs's per-chunk abort keyed on !isRedisError, so an errConnUnusable wrapper (a custom push processor returning a redis.Error during the close drain) was treated as an ordinary reply and later chunks ran though the earlier chunk was never written. Use pipelineErrShouldStamp so errConnUnusable takes precedence and aborts. F3 — a release-time push-drain error from a custom processor removed the conn only when isBadConn recognized it; a partially-drained (desynced) conn could otherwise return to the pool. Remove unconditionally on a non-nil release-drain error. The built-in processor returns nil on a benign nothing-consumed timeout, so this only removes on genuine desync (defense-in-depth over putConn's buffered-data guard). F4 (cosmetic) — newFDEngine resolved a zero FullDuplexWindow into a local only, so Config() reported 0 instead of the enforced 65536; write the resolved value back. Tests + red-checks in autopipeline_fd_reviewfixes4_test.go (a nil-by-default runPipeline seam gives deterministic redis-free F1/F2 tests; production unchanged). Addresses the shutdown-flush-panic, shutdown-flush-errConnUnusable, release-drain conn-remove, and Config-window findings on #4002. * test(autopipeline): consolidate full-duplex tests into one file Merge the seven mergeable full-duplex test files into autopipeline_fullduplex_test.go and delete the sources: the four autopipeline_fd_reviewfixes*_test.go files plus autopipeline_fdpool/fdring/fdsparse. Pure move; 70 Test funcs preserved, no duplicate declarations. The two //go:build fuzz files stay separate (they cannot fold into an untagged file). Also swaps the obsolete hook-snapshot test for TestFDLiveHookChain, matching the snapshot removal in the next commit. * refactor(autopipeline): drop submit-time hook snapshot The full-duplex hook host captured the hook chain at submit time (hookSnapshot / withProcessHookSnapshot) so a hook added between submit and host execution could not retroactively wrap an in-flight command. Process hooks are added at client construction, before the client serves traffic (user AddHook, or redisotel at setup), so the deferred host can load live hook state like the synchronous path. Remove hookSnapshot and withProcessHookSnapshot (hooksMixin + the cmdableClient interface) and gate the FD host on hookCount() with withProcessHook. Also simplifies the comments in autopipeline_fullduplex.go to shorter Simplified Technical English, no behavior change. * fix(autopipeline): spill full-duplex lease to the main pool; bound close-time retries Two full-duplex engine resource fixes (same file). Spill (e7Z0d): fdEngine.attempt leased with a BLOCKING Get on the dedicated pipeline pool and never fell back, so a saturated pipeline pool (e.g. PipelinePoolSize:1 shared with regular pipelines) failed already-accepted commands — unlike withPipelineConn, which TryGets and spills to the main pool. Mirror it: TryGet, then spill to the main pool on ErrPoolTryFull/ErrPoolExhausted or a pipeline-conn init failure; track the source pool so remove/release returns the conn to the pool that owns it. With no dedicated pipeline pool, fd.pool is the main pool and the lease acquires directly. Cost: a spilled FD session holds a main-pool conn until idle/maxHold. Close-time retry bound (e6AA8, cursor e7q9z): retryOnNormalConn bypassed retrySem on ap.ctx cancellation and ran slot-less, so a Close mid retryable-reply storm could spawn one goroutine per in-flight reply (up to FullDuplexWindow) and OOM. Blocking on the sem instead can DEADLOCK with the spill above (a spilled FD session pins a main conn until the reader drains). So on Close: take a slot if free, else fail the command ErrClosed — never block, never run slot-less. retryWg still covers granted slots. Pinned by TestFullDuplexSpillsToMainPoolWhenPipelinePoolFull (server-gated) and TestFDRetryOnNormalConnBoundedDuringClose. Known narrow residual (PoolSize:1, OFF Close): an off-pipe retry that blocks the reader can deadlock while an FD session pins the sole main-pool conn until the reader drains. The spill EXTENDS the reachable set — before, this needed PipelinePoolSize<0 (fd.pool IS the main pool); now a saturated dedicated pipeline pool reaches it too via the spill. The Close path above is non-blocking, so it is unaffected. Called out on the review thread; not fixed here. * fix(redis): replay NoRetry commands after a pre-write drain desync When a pre-command push-drain fails, _process closes the conn before the command reaches the wire and sets forceRetry, so replay is the command's FIRST execution and safe. But processWithRetry still short-circuited on cmd.NoRetry(), so a RawWriteToCmd or ZeroCopyStringCmd returned the drain error without ever executing, contradicting the clean-replay the forceRetry comment promised. _process now surfaces forced (the pre-write desync) distinctly from retry; the gate becomes cmd.NoRetry() && !forced, so a never-written NoRetry command replays while NoRetry still blocks ambiguous write/read failures (a possibly-written command). Mirrors the full-duplex sent-marker rule. A pre-write-drain replay counts as an attempt in the retry/OTel accounting (each loop pass counts), consistent with the existing counter. Addresses the retry-NoRetry-after-pre-write-drain finding (e6AAz). * fix(autopipeline): don't block the reader on retrySem while the session is spilled When the FD session holds a MAIN-pool connection — a spilled lease (PipelinePoolSize saturated) or the no-dedicated-pool case — off-pipe retries on a retryable reply wait for that same connection. Once retrySem filled, retryOnNormalConn's blocking send parked the reader; the writer's idle/max-hold recycle then waited for the parked reader, so the session could not release the connection until a retry's pool acquisition timed out. A burst of retryable replies could stall every in-flight command for PoolTimeout. Track the session's pool in curConnSpilled (set in attempt before the reader is spawned; biased true when a lease is undecided so an unknown state never blocks). In retryOnNormalConn: take a free slot if available; on Close fail ErrClosed (unchanged); while spilled, run the retry SLOT-LESS rather than block, so the reader keeps draining and the session can release the pinned conn; otherwise block for backpressure as before. Residual: up to a window of transient retry goroutines while spilled, reachable only with a saturated pipeline pool at a small PoolSize — documented, and the lesser evil versus a hang. Pinned by TestFDRetryOnNormalConnSpilledDoesNotBlockReader. Addresses the avoid-blocking-retries-behind-their-own-spilled-lease finding (fCCni). * fix(internal): drain pooled semaphore timer unconditionally putSemTimer gated its stale-fire drain on `!t.Stop()` and skipped it when Stop returned true. Stop's return cannot decide whether a fire is pending: it reports false for an already-expired timer, but a fire can settle into the channel while Stop runs, and Stop can report true in that window. A timer holding an undelivered fire then went back to the pool dirty, and the next Acquire's Reset-and-select read the stale value as an instant timeout. Drain unconditionally with a non-blocking select/default after Stop. It is safe under both timer-channel semantics and never hangs: after Stop the timer is disarmed, so at most one fire can be pending; the default branch runs when none is. Observed in CI on Go 1.24 (oldstable, GODEBUG unset) as a spurious timeout in TestFIFOSemaphoreNoStaleTimerFire. Not reproduced locally on go1.24.13 or go1.25.3 across tens of thousands of iterations; the fix holds by construction, since Stop's return cannot detect a pending fire. * fix(autopipeline): harden full-duplex serve loop and shutdown Bot-review batch for the full-duplex autopipeline engine. Recover custom-Cmder user code that runs on run()'s serve loop, which has no top-level recover — a panic there killed the engine and stranded every in-flight and future command. cmdApproxBytesSafe wraps the batch-sizing Args() call: on a panic it fails and DROPS just that command before any bytes are written, instead of letting it reach writeBatch, whose write-time recover would tear the session down and replay the poisoned command's innocent batch-mates at-least-once. Applied at both serve-loop sizing sites. The carry-replay sizing is deliberately left unguarded: its commands already passed serve-loop sizing, so a deterministic panicker never reaches it, and guarding there could only tear down a HEALTHY connection (nothing written yet) to cover a non-deterministic Args() — a deeper contract violation. fdFirstNoRetrySafe wraps the NoRetry() classification scan: on a panic it declines to replay and fails the tail conservatively (a command that cannot be classified, and may be a sent NoRetry, must not be re-sent). Make the non-spill retrySem acquire in retryOnNormalConn cancellable: a Close arriving while the reader is parked on a full sem would deadlock cancelAndDrain, which waits on the reader to advance the deque. On ctx cancel it now fails ErrClosed, matching the documented Close rule. Drain clone engines before detaching the shared-pool close hook: Close now unregisters via defer AFTER cancelAndDrain, so a pool-sharing wrapper closing the shared pool concurrently still blocks on this engine's shutdown flush before the pools are torn down. The defer also survives a drain panic. Publish the resolved FullDuplexIdleTimeout and FullDuplexMaxHold so Config() reports what the engine enforces (a zero field -> its default), matching the window write-back already present. Anchor the OTel duration on the command's FIRST write, not the last replay: stamp writtenAt only when zero so the duration spans the whole retry sequence. Document the AutoPipeliner hook contract (call next; no Close/control call from a hook; no panic; no state mutation). Tests pin the recover wrappers (cmdApproxBytesSafe / fdFirstNoRetrySafe) and the resolved-config reporting. Not covered by a server-driven harness in this batch: the panicking Args()/NoRetry() paths end-to-end through the live engine, the non-spill Close race, and the concurrent shared-pool-close teardown timing. Addresses the serve-loop-sizing-panic (1489), NoRetry-scan-panic (1997), non-spill-close-deadlock (875), clone-drain-on-shared-close (2023), unreported- FD-config (555), replay-duration-anchor (1797) and hook-contract (943) findings. * fix(csc): report the FD attempt on a diverted cache hit On the full-duplex divert (retryOnNormalConn) a cacheable command spends its first attempt on the FD socket, then re-enters processCached with startAttempt>0. If it is served as a cache HIT it returns before processWithRetry's loop — the only place state.attempts is set — so the OTel duration callback reported zero attempts, hiding the FD attempt. Seed state.attempts = startAttempt at entry so a diverted hit reports the attempt it already spent. startAttempt governs the reported COUNT here, not the retry BUDGET (a hit does no network attempt); a miss's processWithRetry overwrites the seed. Addresses the diverted-hit-attempt-count (811) finding. * docs(client): correct WithTimeout autopipeliner note The old note said a clone's autopipeliner is NOT stopped by the parent's Close. That is misleading: the clone's engine registers a close hook on the shared pool, so closing any pool-sharing wrapper drains and stops every registered engine's flusher before the shared pools are torn down — no flusher outlives the pool. The drain does not mark the clone's autopipeliner closed, so an explicit clone Close still fully tears it down (idempotent). Addresses the WithTimeout-lifecycle-doc (1052) finding. * Revert "fix(internal): drain pooled semaphore timer unconditionally" This reverts commit |
||
|
|
f6ffe09a91 |
feat(csc): Refresh and miss coalescing (#3989)
* feat(csc): refresh-on-invalidate and reader-miss coalescing Two additions over the shared-tracking client-side cache, both inert unless CSC is enabled. Refresh-on-invalidate re-fetches recently-read keys as soon as their invalidation push arrives, instead of leaving them to be reloaded by the next reader that misses. Invalidated cache keys that were valid and read within the recency horizon are collected under the shard lock as they are deleted, batched over a fixed window from the first collect, and flushed when the window expires, when a size cap is hit, or on demand the moment a reader misses a key still sitting in the window. Refetches reserve each cache key before touching the network and release any reservation they do not publish, so a reader never blocks on an orphaned in-progress entry. Reads run on a background pipelined connection. Enabled via Options.ClientSideCacheRefreshOnInvalidate; window and demand behaviour tunable through GOREDIS_CSC_REFRESH_WINDOW_MS and GOREDIS_CSC_REFRESH_DEMAND. Reader-miss coalescing pipelines concurrent cache-miss reads onto a single CSC-tracked main-pool connection. Each miss is sent as the caller's own per-key command (not rewritten to MGET, so it is cluster-safe); every reply is applied to its caller's command and published to the cache under the connection's captured tracking generation, so entries stay invalidatable. initConn issues CLIENT TRACKING ON for every pooled connection while CSC is active, so the main pool is tracked; using it explicitly (rather than the pipeline pool) keeps the path correct even if a later change makes a dedicated pipeline pool deliberately untracked, since publishing on an untracked connection would serve stale until TTL. Enabled via GOREDIS_CSC_COALESCE_MISSES; worker count via GOREDIS_CSC_COALESCE_WORKERS (default 8). Adds LocalCache.deleteByRedisKeyCollectingHot / LRUClock / InvalidationStats and per-cache invalidation counters to support the above, with unit coverage of the recency filter and an integration test asserting a refreshed entry is itself invalidatable. * feat(csc): full-duplex reader-miss coalescing + invalidation batching (#3965) * docs: point security reports at Redis VDP (#3949) * fix(pool): log buffered push data at debug level (#3948) The buffered-push-data notice in isHealthyConn fired unconditionally. With client-side caching every tracked write parks an invalidate frame on idle pooled conns, so at default log level the line floods the log on nearly every pool Get under write-heavy workloads. It documents the normal healthy path, not a problem - gate it behind LogLevelDebug like the equivalent notices in maintnotifications. * feat(csc): reader-miss coalescing modes (workers + full-duplex) and invalidation batching Builds on the CSC refresh-on-invalidate + base miss-coalescing PR. Adds: - Coalescing MODES for the reader-miss path, selected by config: * "workers" (default): a small pinned worker pool fetches reserved misses. * "fullduplex": one held connection with a writer + reader goroutine pair pipelines the reserved misses (concurrent, replies streamed back). Ordering is preserved: the miss-coalescer's reservation dedups concurrent misses of the same key to a single fetch, and each caller blocks on its own request's completion — the caching client is blocking per goroutine, so no goroutine ever sees its own reads reordered; full-duplex only overlaps independent goroutines' fetches on the wire. - Invalidation batching: coalesce invalidation-driven cache deletes within a configurable window instead of one delete per push. Config (no environment variables): new AutoPipeline-free Options fields — ClientSideCacheCoalesceMisses (enable), ClientSideCacheCoalesceMode ("workers"/"fullduplex"), ClientSideCacheCoalesceWorkers, and ClientSideCacheInvalidationBatchWindow. Prototype-only read-path telemetry (READPATH_LOG + LocalCache stat counters) removed; no stats surface added. Measured on a 50ms-RTT WAN proxy under invalidation churn, the coalescing fixes turn published v9.22.0's miss-stampede collapse (p99 up to ~1.1s, throughput floored) into ~5x throughput at ~1 RTT p99, matching rueidis; the full-duplex mode gives the tighter mid-range tail at fewer connections. See AP_CSC_TWOCLIENT_VS_RUEIDIS.md. Depends on: feature/csc-refresh-and-miss-coalescing. * fix(csc): address #3965 review and lint in coalescing modes - lint (the only red CI check): drop the redundant `chan struct{}` type from the recycle declaration (ST1023) and //nolint:unused the deliberately-off, measured cscRefreshCooldown knob. - Reject the "pinned" PROTOTYPE engine from the public ClientSideCacheCoalesceMode option: it holds a connection with no idle invalidation drain and can serve stale values (cursor HIGH). It now falls back to "workers" and is reachable only via an internal benchmark hook (cscForcePinned). - Pass the new CSC miss-coalescing / invalidation-batching knobs through UniversalOptions.Simple() (ClientSideCacheRefreshOnInvalidate, CoalesceMisses, CoalesceMode, CoalesceWorkers, InvalidationBatchWindow) so UniversalClient users can enable them (codex P2). - Give the invalidation batcher a stop path: it is stopped and cleared when the last user releases the binding (releaseLocked), and ensureBatcher refuses to start one for an already-released binding, so its goroutine no longer lives past the binding re-arming its timer forever; a later re-acquire starts fresh. - Full-duplex session: acquire the pool connection only after the first miss arrives, and release it if CSC serving was disabled meanwhile, so an idle session no longer starves a small pool (PoolSize:1) until PoolTimeout. Adds unit tests for the pinned-mode rejection and the UniversalOptions passthrough. * fix(csc): FD apply-and-gate on id/gen mismatch; drain buffered pushes; doc - Full-duplex readOne now applies the read reply and lets fulfillCached gate only the cache publish on the captured conn id/generation -- matching the workers/pinned engines -- instead of failing the caller with ErrClosed and losing a good reply on a mid-flight id/gen change. - peekAndProcessPushNotifications also drains when the reader has buffered bytes (HasBufferedData), not only when the socket is readable (MaybeHasData), so a buffered invalidation is processed on the idle tick. - Workers-mode doc corrected: a tracked connection is acquired/released per batch, not held across batches. * fix(search): read the full RESP3 map reply in FTHybridCmd (#3956) FTHybridCmd.readReply used ReadSlice, which reads a RESP3 map header's length as an element count and consumes only half its frames, leaving the connection desynced. Peek the type and read a map with ReadReply, like the other FT.* parsers. * fix(csc): retry uncached on mid-miss disable; guard cancelled fetches - A coalesced miss re-runs uncached when CSC serving is disabled after the miss was reserved (RESP3 downgrade / CLIENT TRACKING loss during a conn re-init), via an internal retry-uncached sentinel that processCached catches, instead of surfacing a spurious pool.ErrClosed for a valid cacheable read. - A caller whose context cancels mid-fetch no longer races the coalescer: a CAS interlock (claimAbandon/claimApply) hands the Cmder to exactly one of the caller or the applying worker, so the worker never writes a Cmder the caller has taken back. The reply is still classified and published to the shared cache, and a cancelled Get returns the context error deterministically (matching the non-coalesced path). - A fetch after coalescer shutdown returns pool.ErrClosed instead of hanging on a post-drain enqueue race. Adds TestClassifyCachedReply, TestCSCMissReqClaimInterlock, TestCSCMissCoalesceAbandonedFetchNoRace, TestFullDuplexDisabledMidMissRetriesUncached. * fix(csc): drop batched deletes on flush; rebuild batcher on window change - A full cache flush (FLUSHDB/FLUSHALL) cleared the cache but left the invalidation batcher's queued per-key deletes, which then fired and evicted entries repopulated after the flush (an extra miss within the window). Drop the batcher's pending queue on flush. - A running batcher's window is fixed at creation, so a second client binding to the same shared handler with a stricter window kept the old cadence and its staleness bound did not hold. setInvalBatchWindow now drops the running batcher on a window change so the next invalidation starts a fresh one with the new window. * ci(govulncheck): use stable Go to pick up security patches setup-go's "1.26.x" resolved to go1.26.5, which govulncheck flags for two standard-library vulnerabilities fixed in go1.26.6: GO-2026-6090 (crypto/tls) and GO-2026-5972 (encoding/asn1). Track the latest stable toolchain so future security patches are picked up automatically instead of pinning a patch. * fix(csc): preserve queued deletes on rebuild; honor configured timeouts - The invalidation batcher's stop path drains keys still buffered in its channel into the final flush, so a window-change rebuild loses no queued deletes (they would otherwise serve pre-invalidation values until TTL/MaxStaleness). Adds TestInvalBatchStopAppliesQueuedDeletes. - ensureBatcher reads the window under the handler lock (not the caller's pre-lock snapshot) and returns nil at window 0, so a concurrent tighten or disable cannot be undone by a batcher rebuilt with the stale cadence. - setInvalBatchWindow folds windows strictest-wins across attached clients (explicit 0/inline strictest, then smaller nonzero): a later, looser attach can no longer lengthen batching past an earlier client's staleness bound. The effective window resets when the last user releases. - The miss-coalescer flush budget derives from the configured WriteTimeout + ReadTimeout (plus a 5s pool-Get floor) instead of a fixed 5s, so clients with deliberately long timeouts do not see only coalesced misses clipped early. * fix(csc): batcher stop/release safety; FD idle release; flush budget - enqueue/stop interlock: enqueue sends under stopMu.RLock and stop flips a stopped flag under stopMu.Lock before closing stopCh, so a key sent by a handler holding a stale batcher pointer lands before the close (applied by the stop-drain) and a post-stop enqueue applies inline — no delete is parked in a channel nothing drains. Adds TestInvalBatchEnqueueAfterStopAppliesInline. - The batcher snapshots cache/refresh at creation and apply uses the snapshot, so the release-time stop-drain still lands queued deletes on a shared cache after releaseLocked has cleared the binding (a successor reusing that cache would otherwise serve pre-invalidation values until TTL/MaxStaleness). - Full-duplex coalescer sessions release their held connection after an idle grace (cscFullDuplexSessionIdle, 1s) instead of holding it toward the 30s recycle age: at PoolSize:1 a non-cacheable command otherwise blocks until PoolTimeout. The next miss re-acquires; an idle coalescer holds zero connections. Adds TestFullDuplexSessionReleasesIdleConn. - batchBudget includes the configured PoolTimeout alongside Write/ReadTimeout (5s floor for scheduling only), so a coalesced flush's pool Get waits the same saturation budget as the normal path, including ReadTimeout: -1 setups. * fix(csc): close coalescer lifecycle races; honor pool budget on acquire - cscMissCoalescer is an atomic.Pointer: processCached loads it once (a cache miss racing Client.Close could observe non-nil then call fetch on nil — a panic and a data race); stop Swaps to nil so exactly one closer wins. - Connection acquisition in all three engines goes through acquireCtx: max(5s, PoolTimeout) so a configured pool budget above 5s is honored under temporary saturation (matching the normal command path), and cancelled when the coalescer stops so Close is not stalled behind a blocked pool Get. The workers engine's per-batch Get is split out of the batch write/read budget (batchBudget covers write+read only again). - The runtime cleanup for a client dropped without Close also stops the coalescer workers (idempotent stopWorkers), so a forgotten client no longer leaks the worker goroutines and everything they retain (cache, pools). - Document that ClientSideCacheCoalesceMisses requires the built-in LocalCache; with a custom Cache implementation the option is ignored. * fix(csc): coalescer latency, shutdown and clone fixes - fullDuplexLoop backs off only after an error end; a clean idle/recycle end starts the next session immediately instead of adding a flat 5ms to a miss already queued. - flushBatch no longer routes the engine's bounded batch ctx through c.context(): with the default ContextTimeoutEnabled=false that stripped the deadline, and with ReadTimeout disabled a non-replying server wedged the worker in ReadRawReply and Close in wg.Wait. - baseClient.clone carries the miss coalescer (atomic.Pointer Store of the owner's Load), so WithTimeout clones keep coalescing instead of silently falling back to per-caller fetches; lifecycle stays owner-only. - Document that ClientSideCacheInvalidationBatchWindow requires the built-in LocalCache (matching the miss-coalescing option). * fix(csc): make FD session I/O interruptible; refresh rebuild; stat CAS - The full-duplex session writer/reader pass the engine's cancellable session ctx directly instead of through c.context(), which stripped it under the default ContextTimeoutEnabled=false; and since no context can interrupt a deadline-less socket read, the supervisor's stop path now bounds the graceful drain — after mc.stop it waits up to the batch budget and then closes the held connection. Close always unblocks: promptly on a healthy drain, within the budget against a server that never replies. - setRefreshQueue drops a running batcher when the refresh binding changes (like setInvalBatchWindow), so a client attaching refresh-on-invalidate later does not leave batched deletes permanently feeding a nil refresher. - countBatch's max-batch statistic uses a CAS loop, so concurrent workers cannot commit a smaller maximum over a larger one. * fix(csc): join FD supervisor before release; drain queue in GC cleanup - The FD session joins its supervisor goroutine (supDead) after closing superDone and BEFORE releasing the connection: the deferred scancel makes sctx.Done() ready at return, and a still-parked supervisor could pick it over the equally-ready superDone and close a healthy connection already returned to the pool — possibly under an unrelated command. - The GC cleanup for a client dropped without Close drain-cancels the miss queue after stopping the workers (non-blocking), so a ctx-cancelled caller's queued request cannot strand an IN_PROGRESS reservation that would block another client sharing the injected cache until StaleTimeout. * fix(csc): clear the push-probe deadline for negative ReadTimeout A negative ReadTimeout means every WithReader skips SetReadDeadline, so the 1ms deadline peekAndProcessPushNotifications installs for its probe stayed on the socket and poisoned the next reply read (a spurious i/o timeout on a held full-duplex connection, or on a pooled conn reused right after a drain). Clear the deadline after the probe for negative-ReadTimeout clients; timeout>=0 callers re-arm their own on the next read. Fixed at the source, covering the FD coalescer tick, the autopipeline paths and the CSC hit-drain. * refactor(csc): full-duplex is the only miss-coalescer engine Misses are caller-blocking — a real request waits on every fetch — so the latency-first engine is the right one, always: a lone miss is written immediately, batching stays opportunistic (packs only what is already queued), and new misses stream out while earlier replies are in flight (~1 RTT per miss, no batch phase-lock, no pool Get on the hot path). Remove the alternatives and their configuration surface: - the "workers" engine (N pooled conns, half-duplex batches): its per-batch round trip added a full RTT of tail latency to misses arriving mid-flight, and its batching advantage is preserved by the FD writer's opportunistic packing; - the "pinned" benchmark prototype (no idle invalidation drain; could serve stale after an invalidation while idle; never publicly selectable); - Options.ClientSideCacheCoalesceMode and ClientSideCacheCoalesceWorkers (both unreleased experimental fields) and their UniversalOptions mirrors: ClientSideCacheCoalesceMisses now selects the one engine, no tuning needed. CSCMissCoalesceStats.Mode becomes Active. Background invalidation traffic is unaffected: it keeps the batching-first windowed batcher and refresh queue. * fix(csc): held-conn probe safety and lifetime bounds - checkForData no longer resets the connection deadline: the probe is a raw MSG_PEEK|MSG_DONTWAIT syscall that never touches net.Conn deadlines, and on a held full-duplex connection the reader's idle tick ran it CONCURRENTLY with the writer inside WithWriter — stripping the armed write deadline and letting a blocked write hang past its timeout. connCheck's own reset stays (it runs on idle pool-owned conns). - peekAndProcessPushNotifications falls back to a throttled timed drain (TakeCscPeriodicReadPending, the CSC drainer's opaque-transport discipline) when socket readiness cannot be inspected (Windows; dialer wrappers exposing neither syscall.Conn nor NetConn), so a push addressed to the held connection is consumed within the fallback interval instead of sitting until the next reply. - The full-duplex session's hold is bounded by the connection's REMAINING absolute lifetime via a new pool.Conn.ExpiresAt() accessor (jitter included), floored at 10ms — a near-expiry pooled connection is no longer held for a fresh full recycle age past the expiry the pool's reaper enforces. * fix(csc): scope idle-drain fallback to held conns; drainer deadline hygiene - The opaque-transport timed-drain fallback moves out of the shared peekAndProcessPushNotifications and into the full-duplex session tick, the one place a connection has no alternative drain coverage. The blanket fallback made every push path run throttled timed drains on uninspectable transports (including mock connections), perturbing sequenced push handling — the maintnotifications downgrade test caught it in CI. Pooled push paths keep their drainer/health-check coverage. The timed drain itself is factored into timedPushDrain, shared by both callers. - drainPushNotifications explicitly clears the socket deadline after its hard-deadline probe/drain for clients with a negative ReadTimeout (which never re-arm): the SetDeadline reset removed from checkForData — because it clobbered a concurrent writer's deadline on held full-duplex connections — had been clearing this residue incidentally on the next drainer tick. This is the explicit, correctly-scoped replacement. * refactor(csc): drop the public coalescer stats API Observability rides the client's normal telemetry — coalesced misses fire the otel operation-duration and error callbacks like any other command path — so CSCMissCoalesceStats (an unreleased experimental type) duplicated that surface for no consumer. The engine's internal counters stay for the in-package tests, which read them directly off the coalescer. * refactor(csc): drop the public coalescer stats API Observability rides the client's normal telemetry — coalesced misses fire the otel operation-duration and error callbacks like any other command path — so CSCMissCoalesceStats (an unreleased experimental type) duplicated that surface for no consumer. The engine's internal counters stay for the in-package tests, which read them directly off the coalescer. Also: acquireCtx folds in the pool's full dial budget (DialerRetries x (DialTimeout + DialerRetryTimeout) + DialTimeout) so a slow endpoint gets its configured dial-retry sequence instead of being cancelled mid-sequence; and the ClientSideCacheCoalesceMisses GoDoc documents pool sizing for the held connection (PoolSize 1 + sustained misses starves non-cacheable commands). Also compacts the review-round comments in the touched files (invariants and contracts preserved; narration removed). * fix(csc): bound the recycle-path drain like the stop path Reaching the recycle age signaled the recycle and exited the supervisor while a reader could stay blocked in a deadline-less read on a non-replying server — with the supervisor gone nothing would ever close the conn, and a later Close hung in wg.Wait. The recycle path now waits up to the batch budget for the session to drain and then closes the connection, mirroring the stop path. Also documents that a custom DialerRetryBackoff exceeding DialerRetryTimeout should raise PoolTimeout (an opaque callback's total delay is not observable from options). * fix(csc): progress-based drain backstop; departial test flake - The stop/recycle drain backstop force-closed the conn after one fixed batch budget, which could cut a HEALTHY drain of a deep in-flight pipeline (up to cscFullDuplexDepth replies, ~in-flight x RTT, possibly under maintenance-relaxed timeouts). The supervisor now samples the in-flight queue per budget interval: any consumed reply extends the wait; only a zero-progress interval closes the conn. - TestReleaseConnRemovesConnectionAfterPartialPushRead skips the benign case where the drain probe times out before consuming any byte: the frame is intact and re-pooling is safe; only a Put after bytes moved into the reader is the desync bug the test pins. * fix(pool): clear residual read deadline in checkForData Removing the SetDeadline reset left a prior WithReader's read deadline armed; once expired, rawConn.Read fails fast with i/o timeout before the non-blocking peek runs, the CSC drainer treats the error as fatal and removes an idle-but-healthy conn, evicting its cache coverage. The full-duplex coalescer concentrates coverage on one held conn, so one spurious removal wiped the whole cache (WAN: hit rate 25%, throughput cut to a third). Clear only the READ deadline: the write deadline stays untouched because checkForData runs concurrently with command writes on a held full-duplex connection - which is what the old full SetDeadline reset clobbered. Regression test pins the expired-deadline case. * fix(csc): claim FD writes; progress-aware drain backstop - The abandon interlock gated only the reply side, so the session writer could serialize an abandoned caller's cmd args - a use-after-return on mutable args (e.g. a []byte key), with the reply publishable under the original cache key. The writer now claims each request (PENDING->WRITING) around arg serialization, releases before the flush, and drops abandoned requests with their reservation cancelled; abandonOrWait yields through a WRITING claim, which spans one in-memory batch encode. - The drain backstop measures progress as completed reads (readsDone) instead of len(inflight), which was blind to the reader's active read, and its interval honors the connection's effective (maintenance-relaxed) read timeout via pool.Conn.EffectiveReadTimeout so a legitimately deep read is not cut short. - The partial-push-read test completes the frame and requires a whole- frame parse before skipping: an empty reader buffer alone does not prove the probe consumed nothing. * fix(csc): snapshot miss wire form at enqueue Replace the WRITING ownership claim with a wire snapshot taken in fetch, while the caller still owns cmd: the session writer writes only engine-owned bytes and never reads cmd, so no ownership window spans arg serialization (which could implicitly flush and block on the socket), abandoning callers return immediately, and a post-abandon arg mutation can neither reach the wire nor publish under the original cache key. Abandoned fetches complete again, so their reservations settle instead of stranding IN_PROGRESS behind the miss backlog. Also: fetch's Close-race branch drains the queue after cancelling (a request landing after the shutdown drain is not retained via a live WithTimeout clone), and the recycle backstop honors explicitly disabled read deadlines - it waits for the session instead of force- closing on a zero-progress interval, escalating only once stop fires so Close still terminates. * fix(csc): harden FD session recycle and probe paths - timedPushDrain reads under a hard deadline: WithReader let an active maintenance-relaxed timeout replace the 1ms probe cap, parking the FD session reader for the relaxed duration on a no-data probe while it held the connection. WithReaderHardDeadline bypasses relaxation and clears the deadline on exit (the negative-ReadTimeout cleanup block is now redundant and removed). - The supervisor also watches the recycle channel, so a reader-triggered recycle (handoff/close-on-put) gets the same bounded progress-based drain backstop as the age-triggered path - a stalled in-flight reply can no longer postpone a requested handoff until Close. - The deadline-less recycle carve-out tests rt <= 0: Options.init normalizes ReadTimeout -1 (indefinite) to 0, so the indefinite mode was still force-closed after one budget interval. * fix(csc): epoch inval drop; refresh ownership; close order - Queued invalidations carry the batcher epoch they were enqueued under; drop() bumps the epoch instead of draining, and apply skips stale-epoch items (queue and in-progress batch alike, so the dropCh signal is gone). The flush handler bumps before cache.Flush(), so a per-key invalidation racing in from another tracked connection after the flush keeps its delete - the drain-based drop could discard it and leave a repopulated entry stale. Dedup is per key+epoch so a post-flush re-arrival is not swallowed by the duplicate check. - stopCSCRefresher clears the shared handler's refresh binding only while it still points at the closing client's own queue: a sibling client sharing the cache/processor keeps its refresher fed. - Teardown deactivates cscActive before stopping the coalescer, and fetch's stop paths settle with errCSCRetryUncached so a clone's miss racing the teardown window re-runs uncached instead of failing with ErrClosed on an open pool; the mid-apply branch returns the settle result itself rather than discarding a successfully applied reply. * fix(csc): serialize inval apply with drop; FD handler adapter - apply/drop are serialized by a batcher mutex: apply snapshotting the epoch once let a concurrent FLUSH land mid-batch and stale-epoch deletes still run post-flush, evicting repopulations - the case the epoch exists to prevent. drop() holding the mutex means an in-flight batch finishes before the flush (harmless: the flush wipes it) and every later apply sees the new epoch. - Both FD-session push paths (the reader's per-reply drain and timedPushDrain) hand handlers the nonblocking cscHandlerClient adapter like the background drainer does: a custom push handler calling Close() signaled-and-deferred instead of deadlocking on mc.wg.Wait while the session reader is parked inside that handler. * fix(csc): retry-uncached drains; refresh stack; weak close - The shutdown drain and the GC cleanup settle queued misses with errCSCRetryUncached, matching fetch's stop paths: a caller woken during the teardown window re-runs its read on the still-open pool instead of surfacing ErrClosed. - The shared handler tracks refresh bindings in a stack: clearing the ACTIVE binding restores the next-newest live sibling instead of nil, so closing the newest owner no longer severs an older client's still-running refresher (the identity check alone only protected the reverse order). - The push-handler adapter closes through the canonical *Client via a weak back-pointer: Client.Close also stops the cached autopipeliners, which baseClient.Close bypassed, leaving flush goroutines running against closed pools. Weak so the wrapper stays collectible and the drop-without-Close cleanup still fires. * fix(csc): write-side backstop; probe gate; clone bypass - The drain backstop honors the write side too: the interval covers the connection's effective write timeout (pool.Conn.EffectiveWriteTimeout) and the deadline-less carve-out fires when either read or write deadlines are disabled - a deadline-free write blocked flushing a large request shows no read progress by construction and must not be cut by an age/handoff recycle. - The opaque-transport speculative probe runs only with the built-in push processor: a custom processor may surface the empty-probe timeout per its contract, which would remove and redial a healthy session on every idle probe. - withTimeout clones whose timeouts diverge from the owner's bypass miss coalescing: the shared engine reads the owner's options, so a coalesced miss would honor the owner's deadlines, defeating WithTimeout. Hits and uncached fetch caching are unchanged. - releaseLocked clears refresh/refreshStack with the binding: a client dropped without Close must not leave a dead queue for a successor to inherit or restore. - The drop-without-Close cleanup drain settles with errCSCRetryUncached like every other stop path (the round-22 hunk for this block was a silent patch no-op; applied for real). * fix(csc): honor lifetime jitter in the FD recycle age Capping the recycle age at the raw ConnMaxLifetime before consulting ExpiresAt collapsed every positive-jitter connection back to the unjittered lifetime, re-synchronizing session recycles across clients started together - the herd ConnMaxLifetimeJitter exists to prevent. Bound the age by the connection's actual absolute expiry only (jitter included), subject to the independent 30s session cap. * fix(csc): fail the FD session on push-processor errors Log-and-continue past a surfaced push-processor error let the reader proceed into ReadRawReply on a possibly desynchronized stream: a processor that consumed part of a push before failing leaves the next bytes mid-frame, and a fragment could be applied as a caller's reply and published to the cache. Treat the error as session-fatal (close and remove the connection), matching drainPushNotifications - the built-in processor surfaces only mid-frame failures, and a custom processor's contract cannot prove no bytes were consumed. * feat(csc): native OTel attribution for coalesced misses A coalesced miss returned from processCached without touching processState, so the native recorder saw zero attempts and a nil connection (empty server.address), and engine-failed requests bypassed the error callback that processWithRetry emits. fetch now returns the serving session connection (recorded by the reader before the done settle, which is the happens-before edge) and processCached stamps it into processState as one attempt; applyAndSettle and settleErr emit the native error callback with parity to processWithRetry and the FD autopipeline engine's failReqs - the retry-uncached sentinel is excluded, since its command re-runs on the fully instrumented path. Refs the deferred-observability thread on #3965. * fix(csc): FD miss-coalescer review fixes - Attribute write-side failures to the held session connection: the miss coalescer sets req.servedBy for the whole batch before the write, so a WithWriter error (or ctx cancel during the inflight handoff) settles with the connection attribution the reply path records, not a nil conn / zero attempts. - Tolerate fragmented push frames on held sessions: once readiness is established, the drain uses the longer cscDrainHardReadCap budget (as the background drainer does) instead of the 1ms probe cap, which could time out mid-frame over TLS and fatally close a healthy session. The 1ms cap is reserved for speculative no-data probes (opaque transports). - Route a WithTimeout clone's CSC push-handler Close through the owner wrapper: the clone's cscClientWeak now points at the canonical owner, so closeCanonical calls owner.Close() (full CSC + autopipeliner teardown) instead of baseClient.Close closing shared pools while the owner's drainer runs. * fix(csc): harden refresh-on-invalidate push drain Two fixes to the refresh-on-invalidate reply reader, matching the miss-coalescer / background-drainer policy: - Propagate push-processor errors instead of logging and continuing. A surfaced error means bytes may have been consumed mid-frame; reading the next reply on a desynchronized stream could publish a push fragment under the wrong cache key. Returning the error aborts the batch so withConn retires the connection. - Use the nonblocking cscHandlerClient adapter for the drain. This reader is part of the refresher's waitgroup, so a custom push handler calling Close() on the raw client would deadlock shutdown (Close waits on the refresher goroutine parked in the handler). * fix(maintnotifications): skip endpoint DNS detect when mode is disabled (#3969) * Update options.go * Update options_test.go * Update options.go * fix flaky test * fix(maintnotifications): skip endpoint DNS detect when mode is disabled * perf(proto): add zero-copy semantics to scan, remove redundant data c… (#3972) * perf(proto): add zero-copy semantics to scan, remove redundant data conversions in reader * chore: unpin govulncheck version * chore: require govulncheck to use latest go version * chore: pin govulncheck and resolve go-version to stable --------- Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com> * fix(csc): record cancel metric on coalesced miss A caller that cancels its context while waiting on a coalesced miss returned early without recording an error metric: the serving session still completes the background fetch (applyAndSettle success or settleErr), so neither records THIS caller's cancellation, undercounting the cancellation rate whenever miss coalescing is enabled. Report it here, attributed to the caller's ctx, matching processWithRetry. The conn is passed as nil deliberately: req.servedBy is written by the session and only safe to read after the req.done receive (the happens-before edge), which has not happened on the cancellation branch, and is nil anyway when the cancel beats the serve. The metric recorder treats a nil conn as "no peer attributes". * fix(csc): retry-uncached on coalescer stop-acquire When getConn fails because acquireCtx was cancelled by mc.stop during coalescer teardown, the session settled the pulled miss and drained the queue with the raw context.Canceled. processCached only retries uncached on the errCSCRetryUncached sentinel, so in-flight coalesced reads surfaced a spurious cancellation even though the caller's context was still valid, instead of re-running on the still-open pool like the other stop paths. acquireCtx derives from context.Background() and is cancelled only by mc.stop (its sole non-deadline cancel source), so a context.Canceled here uniquely means teardown: settle first and the queue drain with errCSCRetryUncached and report stopped so the loop exits without backing off. Genuine acquire failures (dial error, deadline) keep the raw error and the backoff path. * fix(csc): detect coalescer stop by signal, not error The full-duplex miss coalescer mapped a getConn cancellation to errCSCRetryUncached by testing errors.Is(err, context.Canceled). But getConn runs the dialer, credentials callbacks, and init hooks, any of which can return context.Canceled from their OWN context while the coalescer is not stopping. Treating the error value as proof that mc.stop fired permanently exited the sole full-duplex worker; the current miss retried uncached, but later misses still enqueued through the still-live coalescer with no worker to settle them, hanging deadline-free callers and leaking IN_PROGRESS reservations. Detect teardown by checking mc.stop directly instead. mc.stop is closed only by stopWorkers, and close(mc.stop) is ordered before acquireCtx cancels its context (before getConn returns), so a real teardown is always visible. On stop, settle retry-uncached and exit; on any other acquire failure — including an unrelated context.Canceled — fail the batch and back off so the worker stays alive. This supersedes the errors.Is check added in the previous round. * docs(csc): note per-session Limiter for miss coalescing ClientSideCacheCoalesceMisses holds one connection per coalescer session and serves many misses on it, so Options.Limiter is admitted (Allow/ReportResult) once per session — per held connection — not per coalesced miss, unlike the plain per-command path. This is inherent to the held-connection model; a per-miss Allow would defeat the coalescing and re-admit a connection already held. Document it on the option so callers that need strict per-command admission or circuit-breaking know not to enable miss coalescing. * chore(deps): bump rojopolis/spellcheck-github-actions (#3967) Bumps [rojopolis/spellcheck-github-actions](https://github.com/rojopolis/spellcheck-github-actions) from 0.63.0 to 0.66.0. - [Release notes](https://github.com/rojopolis/spellcheck-github-actions/releases) - [Changelog](https://github.com/rojopolis/spellcheck-github-actions/blob/master/CHANGELOG.md) - [Commits](https://github.com/rojopolis/spellcheck-github-actions/compare/0.63.0...0.66.0) --- updated-dependencies: - dependency-name: rojopolis/spellcheck-github-actions dependency-version: 0.66.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix(csc): propagate drain errors on fragmented attributes The held-session push drain (pushDrainWithin) dispatched the built-in processor through ProcessPendingNotifications, which swallows an error that occurs AFTER partially consuming a frame — e.g. a RESP3 attribute whose remainder arrives slower than the hard read cap. A swallowed mid-frame error left the connection desynchronized, so later fragments could be misread as command replies and cached under the wrong key, or fail unrelated misses. Use ProcessPendingNotificationsBuffered for the built-in processor, exactly as drainPushNotifications already does, so a post-consumption error is propagated and the caller retires the desynced session. Custom processors keep the unbuffered call (their contract is "invoked only when notifications are known present"). * fix(csc): limit bare-socket push drain to built-in processor Round-8 review fix (codex r3809020181). peekAndProcessPushNotifications fired on MaybeHasData (raw socket readiness). Over TLS, maybeHasData unwraps the transport. It then returns true for a post-handshake control record that has zero RESP bytes. (conn_check.go's connCheck refuses to unwrap TLS for this reason. maybeHasData does unwrap it.) A custom NotificationProcessor then ran ProcessPendingNotifications under the 50ms hard cap. The empty read timed out. The FD session reader treats that timeout as fatal. It closed a healthy connection, and its in-flight misses failed. The built-in buffered processor accepts the empty read. Now the bare-socket-readiness path runs only for the built-in processor. A custom processor drains only when the reader already holds decoded bytes (HasBufferedData), which means any TLS is already unwrapped. Tradeoff: a push notification that arrives as bare socket readiness on a custom and idle session waits for the next reply read, the session recycle, or the MaxStaleness backstop. The connection stays open. This mirrors the built-in-only guard on the opaque-transport probe in the FD session tick. * fix(csc): peek frame type before a custom push drain Round-9 review fix (cursor r3811858390). The round-8 D fix let a custom push processor drain on HasBufferedData. But the buffered bytes can be a coalesced reply, not a push. A custom processor invoked then could consume the reply, cache it under the wrong key, and desync the stream. The built-in processor peeks the frame type and consumes only push frames, but the NotificationProcessor interface does not promise that. pushDrainWithin now peeks the frame type for a custom processor and returns without a read when the frame is not a push. The peek is bounded by the closure's hard read deadline. * fix(csc): propagate custom push-drain peek errors Round-10 review fix (codex r3812520847). The round-9 fix peeked the frame type before it handed a buffered frame to a custom push processor, but it returned nil on a peek error. PeekReplyType can partially consume a fragmented RESP3 attribute (DiscardNext) before it errors, which desyncs the stream. Swallowing the error kept the desynced connection, so later fragments could be read as a command reply and cached under the wrong key. The peek error now propagates, so the session fails and drops the connection, as the built-in buffered path does. A cleanly-peeked non-push frame still returns nil (a real reply left for the caller's read). * fix(csc): drain miss-reader pushes through the shared safe helper Round-11 review fix (cursor r3813795126). The full-duplex miss reader (readOne) drained pushes with the unbuffered ProcessPendingNotifications for both processor kinds -- a third drain site the earlier desync fix did not reach. The built-in processor there swallowed a mid-frame DiscardNext error on a fragmented attribute, and a custom processor got no frame-type peek, so either could leave the stream misaligned; readOne then read the shifted bytes as the reply and cached them under the wrong key. Both drain paths now share drainPushFrames: the built-in uses the Buffered variant (mid-frame errors propagate) and a custom processor is handed only a confirmed push frame (peek first, propagate the peek error). The helper sets no read deadline, so readOne's following ReadRawReply is unaffected. * fix(csc): retire conn on pre-command drain desync; refresh drain helper Two review fixes. Pre-command push-drain desync (round-13 audit): _process drains pending pushes before executing a command, then reads the reply on the same conn and, on the CSC-miss capture path, caches it. A fragmented RESP3 frame straddling the drain's short hard cap left the reader mid-frame; the error was logged and ignored, so the reply read consumed the residue and could cache it under the wrong key. _process now closes the conn and retries on a fresh one instead of ignoring the error (peekAndProcessPushNotifications already propagated mid-frame errors via drainPushFrames). Refresh reader drain (cursor r3813795126 / r3815669446): the refresh reply reader still drained pushes with the unbuffered ProcessPendingNotifications, a fourth drain site that could swallow a mid-frame desync before ReadRawReply and publish a push fragment under the wrong cache key. It now routes through the shared drainPushFrames helper, like the idle-tick and miss-reader paths. * fix(csc): stop the refresh worker on GC cleanup Round-14 review fix (codex r3817196217). When refresh-on-invalidate is enabled and the caller drops the client without Close, the runtime cleanup stopped the miss coalescer and the invalidation drainer but never signaled the refresh worker. runCSCRefresher then stayed parked on its ticker/queue, holding the baseClient, cache, and pools -- a goroutine and memory leak that defeated the drop-without-Close safety net. cscRegisterCleanups now captures the refresh handle and signals it, idempotent and non-blocking (a new cscRevalidateHandle.signalStop guarded by sync.Once, so Close and the cleanup cannot double-close). stopCSCRefresher uses the same signal. * feat(csc): add refresh-failure metric Adds RefreshFailed to CSCRefreshStats -- the refresh-failure counter the HLD lists (go-redis section 4.1). A refresh round trip that errors increments it; those keys stay evicted and a later read repopulates them, so a rising count is the signal that refresh-on-invalidate is degrading to plain eviction. Counted per errored batch, not per key. * fix(csc): drain pending pushes before reading the reply The CSC miss and refresh readers drained push notifications with the Buffered variant, which stops the instant the reader buffer empties. If a second invalidation was still on the socket ahead of the command reply, ReadRawReply read that push as the reply and cached it under the wrong key -- a one-frame shift that then cascaded to later replies. The readers now drain in BLOCKING mode: they block on the socket and skip push frames until a non-push frame (the reply) is next, the same non-buffered discipline the full-duplex reader already uses. PeekReplyType is attribute-aware, so a fragmented RESP3 attribute needs no separate buffered scan. A swallowed boundary-peek TIMEOUT is caught by the reader's shared read deadline (ReadRawReply hits the same expired deadline and fails the session); a swallowed non-timeout peek error would need a malformed attribute mid-push (a server protocol bug), so this path is no weaker than the full-duplex reader. Probe and idle paths keep the Buffered variant so they never block. A socket-pair regression test pins the frame order. Also in this change: - GC cleanup unbinds the dropped client's refresh queue from a shared invalidate handler (clearRefreshQueue), so a surviving sibling's refresh-on-invalidate keeps working instead of feeding a stopped queue. A refresh queue is only created inside attachSharedTrackingCSC, which also builds the drain handle, so the handle is always present when the queue is; a Conn() clone bails before the refresher starts. - Record usedAt on deadline-free (negative-timeout) reads and writes, so a long full-duplex session under ReadTimeout/WriteTimeout=-2 is not misjudged as idle-expired by the pool and needlessly reconnected. * fix(csc): harden miss-coalescing per review - spill invalidation overflow off the full-duplex reply reader; the worker drains it through the same seen/pending dedup, so a burst collapses to one delete per key and no cache work runs on the reader - drop the drain deadline cleanup that re-armed a relaxed timeout (WithReaderHardDeadline already restores the deadline) - cap a coalesced write batch by bytes, not just count - warn when ClientSideCacheInvalidationBatchWindow exceeds MaxStaleness Refs #3965 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: sadd amr3e <saddamr3e@gmail.com> Co-authored-by: Albert Huynh <huynhalb@gmail.com> Co-authored-by: Vladislav Kotsev <vladygk8@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix(csc): address review findings on refresh, batcher, and spill Refresh-on-invalidate: - pass the refresher's internal 5s ctx directly to WithReader/WithWriter instead of c.context(ctx); c.context strips it to Background when ContextTimeoutEnabled is false, so with read/write timeouts disabled a stalled Redis hung the refresher and wedged Close (cursor/codex). - recover() the flush batch and defer its cancel(), so a panic can't kill the refresher (silent permanent degradation) or leak the 5s timer. - skip Reserve's (0,true) 'fetch uncached' targets: they wasted an MGET slot and registered server-side tracking for a never-cached key. Invalidation batcher: - hold the handler lock across drop()+Flush() and read the batcher/cache fresh under it, so a concurrent batcher rebuild can't leave stale-epoch deletes applying after the flush and evicting post-flush repopulations (cursor). - hard-cap the overflow spill and schedule one full Flush past the cap: the invalidatable keyset is NOT bounded by cache size (the server tracks keys past local LRU eviction), so an unbounded spill could OOM (codex). Adds a test. Other: - FTHybridCmd: populate rawVal on the RESP2 path for RawResult parity (Copilot). - docs: mark ClientSideCacheRefreshOnInvalidate experimental + LocalCache-only (Copilot); correct the HasBufferedData 'complete frame' comment; note the Effective*Timeout post-expiry side effect; reconcile the refresh header's stale-read claim; drop a stale symbol reference. * fix(csc): follow-up review fixes (spill race, stats, refresh, acquire) - inval batcher: set flushReq BEFORE releasing stopMu at the spill cap, so a concurrent stop() cannot close stopCh and let the worker exit without seeing the request (lost flush, stale entries). - inval batcher: count a dedup-dropped duplicate invalidation as invalidation + noop, so InvalidationStats reflects incoming pushes (the duplicate ratio) with the window batcher on, matching the inline path. - csc integration: unbind the refresh queue in the drainer's exit defer, so a SELF-DISABLE exit (custom-processor damping / RESP3 downgrade) restores a sibling's binding instead of silently breaking the survivor's refresh. - miss coalescer: derive the connection-acquire budget from a custom DialerRetryBackoff (sum the per-attempt delays), so a backoff longer than DialerRetryTimeout no longer cancels Get mid-retry-sequence. Addresses codex re-review findings on #3989. * fix(csc): address codex re-review (relaxed-timeout race, acquire, spill stats) - pool relaxed timeout: the counter is a relax/unrelax NOTIFICATION balance, not a per-I/O count. getEffective{Read,Write}Timeout no longer decrements it per call; a passed deadline triggers a single CAS-coordinated reset (expireRelaxedTimeout), so a full-duplex connection's concurrent reader/writer/drain-backstop can't drive it negative and wedge later relaxations, and a lost unrelax notification is recovered at the deadline. ClearRelaxedTimeout is clamped at zero. Adds a concurrent-expiry regression test. - miss coalescer acquire budget: revert the DialerRetryBackoff summing — calling a randomized/stateful backoff out-of-band consumes different jitter samples than the pool's actual retries, so the estimate can still be wrong. Use the deterministic flat estimate; mc.stop + PoolTimeout bound it for custom backoffs. - inval batcher: count invalidations superseded by the spill-cap full-Flush (enqueue discard + worker pending reset) so CSCRefreshStats doesn't undercount an invalidation flood. Addresses the codex re-review on #3989. * fix(pool): retire one relaxed holder on expiry The relaxed-timeout deadline safety net full-cleared the whole window on expiry. On a full-duplex conn the reader, writer, and drain backstop all observe the expired deadline concurrently; a CAS on the shared deadline already gates that to one action, but a full clear also wiped a DIFFERENT relaxation sharing the conn -- a notification SetRelaxedTimeout outstanding alongside an expired handoff deadline lost its window (the :566 race). Retire exactly one holder instead: expireRelaxedTimeout decrements the counter by one (clamped, clears the timeout values only when the last holder leaves). To keep overlapping handoffs from leaking a permanently relaxed conn, SetRelaxedTimeoutWithDeadline now takes a holder slot only for the FIRST deadline-scoped window and re-arms by replacing the deadline in place (Swap), so the eventual expiry retires the single holder and the window clears. Add deterministic regression tests: a notification window survives an expired handoff deadline, and overlapping handoffs clear without a leak. Addresses the codex conn.go:566 finding on #3989. * fix(csc): split invalidation and deletion stats InvalidationStats conflated two quantities and undercounted whenever the window batcher deduped, spilled, or fell back to a full Flush, which drove a recurring "another uncounted path" review cycle. Split them: - Invalidations counts keys named in INCOMING pushes, tallied once at the handler choke point before dedup/batching. - Deletions / DeletionsNoop count APPLIED deletes at the two cache choke points (DeleteByRedisKey and deleteByRedisKeyCollectingHot), so the count is accurate whether refresh-on-invalidate is on or off. Invalidations - Deletions now measures dedup plus the flood full-Flush fallback directly, so the batcher no longer needs ad-hoc per-drop counting (removed addInvalidationStats and its three call sites). Both are exposed on CSCRefreshStats. Also snapshot the refresh recency horizon at ENQUEUE into cscInvalItem.sinceToken and use it in apply, so a batch-window delay cannot advance the live horizon and chill a key that was hot when the invalidation arrived (it would degrade to plain eviction instead of a proactive refresh). Value-passed, no added allocation. Addresses the cursor csc_inval_batch.go batch-chill finding and the codex csc_inval_batch.go stats-undercount finding on #3989. * fix(pool): serialize relaxed-timeout mutations SetRelaxedTimeoutWithDeadline stored its new timeout values before swapping the deadline, so an expiry observed concurrently (a full-duplex reader/writer/drain backstop) could CAS the old deadline to zero and clearRelaxedTimeout the fresh values between the store and the swap -- leaving a window with a future deadline but zeroed timeouts (active but ineffective; commands fall back to the shorter normal timeout during a migration and can fail spuriously). Reordering alone only moves the race. Add relaxedMu to serialize all four mutators (Set, SetWithDeadline, Clear, expire) so the timeout/deadline/counter atomics are always updated or cleared as one unit. expireRelaxedTimeout re-validates the deadline under the lock (a generation check): an expiry the caller observed lock-free is dropped if a newer window replaced it. The read path (getEffective*/HasRelaxedTimeout) stays lock-free and takes the lock only via expire on the rare observed expiry, so the per-I/O hot path is unchanged. The former CAS retry loops collapse to plain load/store under the lock. Addresses the codex relaxed-timeout rearming-vs-expiry finding on #3989. * fix(csc): repoint batcher to survivor before stop The invalidation batcher captured its refresh binding at creation. When the active refresh owner closed, clearRefreshQueue restored the surviving sibling on h.refresh and stopped the batcher, but the stop-drain still offered evicted-hot keys through the closing owner's queue -- whose drainer had already gone, so the offers were dropped and the survivor never refetched them (they stayed evicted until a reader missed). setRefreshQueue had the same staleness on a rebind. Make cscInvalBatcher.refresh an atomic pointer and repoint it at the current binding (survivor / new owner, nil if none) BEFORE stop() in both set/clearRefreshQueue, so the stop-drain feeds the live refresher. An in-flight apply already holding the old pointer feeds its current batch to the old queue -- a bounded, benign residual (offers to a stopped queue are dropped, never panic: the queue channel is never closed). Addresses the cursor "batcher drain feeds closed refresher" finding on #3989. * fix(pool): relaxed timeout as one atomic snapshot Ofek's review asked for the relaxed-timeout window to be published as one value so a reader never sees it half-updated. Replace the mutex plus four atomics with a single atomic.Pointer to an immutable relaxedState snapshot. Set, SetWithDeadline, Clear, and expire install a new snapshot with a compare-and-swap; the read path (getEffective* and HasRelaxedTimeout) does one lock-free Load. This removes the writer-only mutex and the torn read a reader could see between the counter bump and the timeout store. It also fixes getEffective* returning the normal timeout on the call that observes an expired deadline while a notification holder is still active. The method now re-reads the snapshot after it retires the deadline holder and returns the surviving relaxed timeout. Addresses the codex relaxed-timeout thread and Ofek's snapshot request on #3989. * fix(csc): bound coalesced writes, harden miss retry Reader-miss coalescing and refresh both write a whole batch before they read the replies, and the reader waits until the write finishes. A batch larger than the connection write buffer (default 32 KiB) makes bufio flush in the middle of the batch, which can deadlock the write and the read on large payloads. The batch byte cap was 1 MiB. Bound each batch to the write buffer: - The miss coalescer caps a batch at min(write buffer, 1 MiB). An over-budget request is carried in the writer's own state and sent as the first request of the next batch, not put back on the shared queue. So a shutdown cannot strand it, and it also does not widen a pre-existing reservation-strand window on the drop-without-Close path. The first request always goes. - The refresh chunker applies the same byte bound next to its count cap. Also route a coalescer session or transport failure through the normal retry path. settleErr tags such errors as cscSessionError; processCached re-runs the read with MaxRetries and backoff instead of a raw io.EOF to the caller. A reply-level result (redis.Nil, WRONGTYPE, LOADING) stays untagged and is returned as-is. Addresses the Ofek review threads on #3989: write-before-read batches, the refresh batch, the byte-cap overshoot, and the coalesced-miss retry path. * fix(csc): drop ineffectual pending clear The grabInto carry change made the pending = nil at the top of the writer loop a dead store: grabInto now reassigns pending (with the next carry) before it is read again. Drop the clear. No behavior change. Fixes the golangci-lint ineffassign that CI flagged. * fix(csc): read coalesced replies while writing the batch The miss-coalescer wrote the whole batch and only then handed the requests to the reader, so the reader was idle for the entire write. A batch larger than the transport send capacity could deadlock: the server blocks writing early replies that nobody reads while the writer blocks flushing later requests. Bounding the batch to the write buffer did not fix this — the write buffer does not bound the socket send buffer (a custom Dialer or a large WriteBufferSize breaks the assumption), and with per-op timeouts disabled it can wedge. Hand the batch to the reader BEFORE writing it (as the autopipeline full-duplex engine already does), so the reader drains reply k while request k+1 is still being written and relieves the server's send backpressure. The send cannot deadlock the writer: a batch is at most cscMissBatchMax (128) and inflight is cscFullDuplexDepth (4096) deep, the writer is sequential so every prior batch was already flushed, and every reader-exit path either cancels the session context or follows the writer's own exit. The byte cap remains only as a batch-size bound, not as the deadlock fix. Addresses the Ofek and codex/cursor write-before-read findings on #3989. * fix(csc): stats without refresh, bound refresh write Two refresh-path fixes from the review: CSCRefreshStats returned an all-zero struct when the refresh queue was absent, so with ClientSideCacheRefreshOnInvalidate off (the default) but invalidation batching on it hid the cache-level Invalidations/Deletions/DeletionsNoop the handler still records. Read those cache counters independently of the queue; gate only the queue-specific counters on the queue. The refresh writes a whole chunk before it reads the replies, so a chunk that the server cannot drain (transport backpressure) blocks the flush. With per-op write timeouts disabled the flush is deadline-less and the 5s batch context does NOT interrupt it, so the refresher goroutine wedges — worse than the review reported. Bound the write with a positive deadline (shared with the batch context via one constant) so a stalled flush fails the refresh and the entries degrade to plain eviction (self-healing). Full read/write concurrency in the refresher is a possible follow-up, not done here. Addresses the codex refresh-stats and refresh write-before-read findings on #3989. * fix(csc): honor recycle/stop on the carry path grabInto can defer one over-budget request as a carry, held in the writer's pending slot and written as the first request of the next batch. While pending is non-nil the writer took the carry branch and skipped the select that watches recycle and mc.stop, so a producer whose commands are persistently over the byte cap could keep the held connection busy and starve a maintenance handoff or a lifetime recycle indefinitely (its OnPut hooks would never run). Check recycle/stop/session-cancel (non-blocking) between batches even when a carry is present, and settle the carry on bail — it lives only in the writer's state, so the teardown drain, which covers inflight, would otherwise leak it and hang its caller. Settle with the retry-uncached sentinel on recycle/stop so the caller re-runs on the pool. Addresses the codex carry-vs-recycle finding on #3989 (a follow-on from the carry introduced last round). * fix(csc): bound the refresh read too Last round bounded the refresh WRITE so a deadline-less flush could not wedge the refresher. The following WithReader still used the raw ReadTimeout: with per-op timeouts disabled (options.go maps -2 to -1, which WithReader treats as no deadline) a stalled reply or push drain parks refreshInvalidatedBatch forever, and stopCSCRefresher waits on that goroutine, so Client.Close never returns. Bound the read with a positive deadline (still capped by the 5s batch ctx, shared via cscRefreshBatchTimeout), symmetric with the write, so a stall fails the refresh and the entries degrade to plain eviction instead of hanging Close. Addresses the cursor refresh-read-hangs-Close finding on #3989. * fix(csc): per-chunk deadline for refresh round trips The refresh split a window into chunks (by count and by write-buffer bytes) but ran them all under one cscRefreshBatchTimeout context. When a window held more than one chunk and their AGGREGATE latency exceeded the budget, later chunks failed immediately even though each round trip was within the intended per-round- trip timeout — four 2s chunks under a 5s budget would refresh only the first two and leave the rest evicted. Create the bounded context PER refreshInvalidatedBatch call so the documented per-round-trip timeout applies to each chunk independently. The recover guard stays at the loop level. Addresses the codex per-chunk-deadline finding on #3989. * fix(csc): flush snapshot cache on FLUSH invalidation The FLUSHDB/FLUSHALL invalidation branch re-read the live h.cache under a second RLock and flushed that, while per-key deletes use the cache pointer snapshotted at handler entry. A last-user releaseLocked (drainer teardown / GC cleanup) can nil h.cache between the entry RUnlock and the flush RLock, so the guarded 'if h.cache != nil' skipped the wipe entirely. On an injected shared cache (cscOwnsCache == false) the teardown flush is gated off, so post-FLUSH entries kept serving until TTL / MaxStaleness. Flush the entry snapshot instead, matching the per-key branches. The second RLock stays — it guards the batcher epoch against a concurrent rebuild — but only the batcher is read fresh under it now; the cache uses the snapshot. The nil guard is dropped: the entry guard already returns on a nil snapshot. Addresses the cursor 'FLUSH can skip shared cache wipe' finding on #3989. * fix(csc): drain all pushes before a coalesced/refresh reply drainPushFrames in blocking mode, for a CUSTOM PushNotificationProcessor, peeked and processed once. A second invalidation still on the socket ahead of the reply was then read by the caller's ReadRawReply as the command value — published to the shared cache under the wrong key and shifting every later reply. Loop peek-then- process until a non-push (the reply) is next, matching the built-in blocking discipline; non-blocking probes keep the single-pass behavior. Covers the coalescer and refresh reply readers. * fix(csc): bound in-flight coalesced wire by bytes Miss coalescing serializes each caller's command into req.wire before it can enter the bounded queue; arbitrarily many callers can block on the send holding a full wire copy, and large cacheable commands (a big MGET) have no small encoded size, so a burst could exhaust memory — the queue's item cap does not bound bytes. Track total in-flight serialized bytes; before serializing, reserve this command's approximate size and shed over cscMissWireBudgetBytes (8 MiB) to the pooled path (errCSCRetryUncached), which is bounded by pool turns. Reserving before serialization means over-budget callers never allocate the wire; a single command larger than the budget sheds too (the pooled path runs it). TestCSCMissCoalescerWireBudget pins it. * fix(push): consume a push whose name can't be peeked The built-in blocking drain, on a frame already confirmed as a push, broke out of the loop when PeekPushNotificationName failed for a non-'too-long' reason (a non-string / malformed name). Breaking LEFT the push at the buffer head, so a reply-expected reader (CSC refresh / miss-coalescer / the single-command pre-read drain) then read the push as the command value — the intermittent 'redis: nil' and cross-command reply shift under load. Fall through to ReadReply and CONSUME the push instead (matching the name-too-long path): a genuine mid-frame read error still surfaces at ReadReply and retires the connection, and a non-string name is gracefully ignored (the downstream type assertion fails, so no handler runs). Buffered probes still stop cleanly on a boundary timeout. Affects every RESP3 client's pre-read drain, not only CSC. TestProcessor_ConsumesUnpeekableNamePush asserts the following reply is readable (the push was consumed, not left). * fix(csc): release miss wire budget when the wire is consumed The miss coalescer reserved wire-budget bytes before serializing a command, then released them in a defer that fired on every fetch exit — so a caller that abandoned after enqueue freed the budget while its wire snapshot was still retained in the session channel / inflight deque. A burst of large abandoned misses could thus drive the retained wire bytes far past cscMissWireBudgetBytes. Track the WIRE's lifetime instead of the caller's: store the reservation on the request (reserved) and release it exactly once, through a new settle helper, when the request is consumed (applyAndSettle, settleErr, shutdown drain). fetch releases in its defer only when the request never reached the queue (serialize error, or Close/ctx before enqueue), guarded by an enqueued flag. Also honor ContextTimeoutEnabled for the coalescer's enqueue and reply waits: when it is off the ordinary command path drives socket I/O on context.Background() (bounded by ReadTimeout, not the caller deadline), so a coalesced miss must not surface context.DeadlineExceeded on the caller's deadline. Wait on c.context(ctx) (caller ctx when the policy is on, Background when off); mc.stop stays the unconditional shutdown signal and returns still report ctx.Err(). Addresses the csc wire-budget and context-timeout review findings on #3989. * fix(csc): repopulate cache on coalescer-bail retry, order close Two fixes on the CSC integration path. Fallback populate: when the miss coalescer bailed (errCSCRetryUncached or a session error) processCached retried through a nil-capture processWithRetry — it returned the value but stored nothing, so a connection blip or a wire-budget shed left the key uncached and every later reader missed it. Re-Reserve and fall through to the capture path instead, so a successful retry repopulates the cache (Reserve may hand fetching to another reader, which the capture path already handles). Close ordering: stopBackgroundDrainer deactivated serving (cscActive=false) before joining the refresher, but the refresher's stop-path flush re-fetches the in-window invalidated keys and refreshInvalidatedBatch skips that work once cscActive is false — so the last collection window was dropped, and on a SHARED (unowned) cache those hot keys stayed evicted for siblings until a later miss. Stop the refresher FIRST (serving still on, pool still open), then deactivate before stopping the coalescer, which still shields a racing clone miss from an ErrClosed on the just-stopped coalescer. Addresses the csc fallback-populate and close-refresh-window review findings on #3989. * fix(csc): cap per-target key bytes in the refresh queue The refresh queue's depth cap (cscRefreshQueueDepth) bounds it by ITEM count, not memory: with large key encodings, up to that many queued targets could pin far more than the cache's own limit. Drop any target whose key bytes (cache key + Redis keys) exceed cscRefreshTargetMaxBytes before enqueue — like a full-queue drop, it just means a later reader does a normal miss fetch — bounding retained key memory to roughly cscRefreshQueueDepth * the cap. Addresses the csc refresh-queue-bytes review finding on #3989. * fix(csc): drain queued refresh targets on refresher stop The refresher's stop branch flushed only the collected window (pending); targets still buffered in q.ch — offered but not yet collected — were abandoned. Because stopCSCRefresher rebinds the invalidate handler away from this queue BEFORE signalling stop, nothing else drains q.ch, so on a shared cache up to cscRefreshQueueDepth hot entries stayed evicted for the surviving sibling until a later reader missed them. Extract drainQueue (the existing non-blocking window-fill, now reused by the q.ch branch) and, on stop, loop drain+flush until the channel is empty. The channel is quiescent at that point (no producer), so the loop is finite, and the pool is still open (closeResources runs after the join), so the final refetch can complete. Addresses the codex drain-queued-refresh-targets finding on #3989. * fix(csc): use live horizon for no-binding invalidations An invalidation enqueued while the batcher had no refresh binding stamped sinceToken 0. apply re-reads the refresh binding at apply time, so a refresh client attaching in between (setRefreshQueue repoints the batcher, then stop-drains it) applied those items against horizon 0 — treating every valid entry as hot and offering it for refetch. That is the cold-key refresh loop the recency horizon exists to prevent, and it re-registers cold keys with the server, pinning its tracking table. Stamp cscInvalNoHorizon (-1) when no binding exists at enqueue; apply falls back to the binding's LIVE horizon (seeded at the attaching client's start), a real recency test that can only under-refresh, never chase cold keys. LRUClock is a monotonic sequence >= 0, so -1 cannot collide with a real horizon. TestInvalBatcherRefreshAttachUsesLiveHorizon pins it (a cold pre-attach entry is deleted but not offered; a hot one is). Addresses the cursor refresh-attach-bypasses-recency-gate finding on #3989. * fix(options): normalize nonpositive DialerRetries Options.init normalized DialerRetries only when == 0, but the pool treats any value <= 0 as the default (ConnPool.dialConn). A negative DialerRetries therefore left the miss-coalescer's acquire deadline (which derives its budget from opt.DialerRetries) computing a smaller retry count than the pool actually uses. Normalize <= 0 so both see the same count; document the field accordingly. Addresses the codex normalize-nonpositive-retry-counts finding on #3989. * fix(csc): raise push-drain budget under relaxed timeouts Every push drain ran under a hard 50ms read cap (WithReaderHardDeadline) that deliberately bypasses maintenance relaxation. Right for a speculative probe on a healthy conn; wrong mid-failover: relaxation is active exactly because the server is slow, a push frame can fragment past 50ms, and a mid-frame timeout is a desync — so the pre-command path retired a HEALTHY tracked connection (evicting its CSC coverage) and, with MaxRetries=0, failed the command outright. The old pre-PR drain honored relaxation via WithReader. Add pushDrainBudget: the hard cap, raised to the connection's relaxed timeout while relaxation is active (EffectiveReadTimeout; one-directional — a smaller relaxed window never lowers the cap). Paid only when bytes are actually mid-frame: an empty or completed drain returns without blocking regardless of the budget. Pinned by TestPushDrainBudgetHonorsRelaxation. Addresses the pre-command-drain finding in the #3989 deep review. * fix(csc): join inval batcher on stop, wait on re-reserve loss Two review fixes on the invalidation/miss integration paths. Batcher join: stop() only signalled; the batcher's stop-drain and final flush ran asynchronously after Close (or a rebuild) returned — a straggler goroutine past Close, and on a shared injected cache a late apply could evict an entry a successor client just repopulated. Give the batcher a done channel (closed when run() exits) and a join(); every stop site (releaseLocked, setInvalBatchWindow, set/clearRefreshQueue) now stops AND joins, making teardown and rebuilds synchronous. join is safe under h.mu: run() takes no handler locks. Pinned by TestInvalBatcherStopJoinIsSynchronous. Also resolves the join-before-tighten review thread. Bail-loser wait: after a coalescer session failure, waiters raced the re-Reserve and every LOSER fell through to an independent pooled processWithRetry — one hot key turned into N pooled requests exactly mid-recovery. A loser now waits on the new owner (csc.Get parks on the in-progress entry, same shape as the first-Reserve loser path) and only falls back to the pooled path after a second Reserve loss. Addresses the codex drain-after-close and keep-same-key-waiters findings on #3989. * fix(csc): consume window demand nudge on refresh flush A demand signal buffered while a window was being flushed survived the flush: signalDemand fires only for keys in pendingSet, so the nudge referred to a key this flush was already refetching, yet it immediately flushed the NEXT window on its first collected key — defeating the coalescing window and inflating DemandFlushes. flush() now consumes one buffered nudge after clearing the window (collect runs only on the refresher goroutine, so no future-window demand can exist yet). Addresses the codex discard-stale-demand-signals finding on #3989. * fix(csc): emit coalesced reply-error metric exactly once applyAndSettle emitted the native error metric for every non-nil parsed reply while a caller whose context expired late emitted its cancellation independently — the same operation recorded twice with conflicting error types whenever the cancellation raced the reply. Centralize emission at the caller's req.done receive (emitReplyErr): reply errors (redis.Nil, WRONGTYPE, ...) are recorded once, where the caller consumes them; session errors and shed retries stay excluded (their processWithRetry re-run emits its own outcome); a caller returning ctx.Err() emits only its cancellation. req.servedBy is safe at the receive (the done receive is the field's happens-before edge), so the metric now also carries the serving conn on the paths that previously reported from the reader side. Addresses the codex double reply+cancel metric finding on #3989. * fix(csc): staleness warning covers injected LocalCache The ClientSideCacheInvalidationBatchWindow > MaxStaleness misconfiguration warning was gated on ClientSideCacheConfig != nil, so a pre-built cache injected via ClientSideCache was never checked — a 10ms-staleness cache with a 500ms batch window served pre-invalidation values 50x past its own contract, silently. When the config is nil, read the injected *LocalCache's effective bound (effectiveMaxStaleness) and run the same comparison. Addresses the injected-cache warning gap in the #3989 deep review. * fix(csc): budget refresh round trips by expected reply bytes The refresh chunker bounded only the REQUEST side (key bytes vs the write buffer); the reply side was unbounded. Within one chunk the server streams replies for the early keys while the client is still writing the late ones, and against a flow-controlling middlebox (a proxy or tunnel that stops reading when its buffer toward the client fills — vanilla Redis instead buffers replies in RAM) enough in-flight reply bytes stall the connection both ways: the classic pipeline write-read deadlock, surfacing as the per-chunk deadline failing the whole chunk and refresh silently degrading to eviction for large-value workloads. Budget each round trip's EXPECTED reply volume: collectHotAndDelete captures the dying entry's value size (free, under the same shard lock) on the refetch target, and the chunk boundary — extracted pure as cscRefreshChunkEnd and unit-tested — now caps count, request bytes, and expected reply bytes (32 KiB, comfortably inside typical kernel socket buffering). An estimate, not a guarantee (a value may have grown since eviction); the per-chunk deadline stays the backstop, and full read/write concurrency in the refresher remains the follow-up if the deployment path is confirmed to apply per-connection backpressure. Refs the refresh write-before-read review threads on #3989. * docs(csc): freshness contract, push spec, metrics TODO Outcomes of the push-consumption review discussion, no behavior change. - Document the invalidation-freshness contract on ClientSideCacheConfig: applied at arrival on full-duplex conns, at the next command on active conns, at the next drainer tick on idle conns; MaxStaleness is the hard bound everywhere. - Add .claude/specs/push.md: the checkpoint consumption model and its physics (frame completion, TLS peek opacity, custom-processor gating), the freshness guarantee, and the rules that keep the model closed — no new speculative drain sites; features needing fresher pushes own a reader (FD-session pattern). Also records the planned NotificationDrainer capability follow-up. - TODO(convergence) at the coalescer-bail re-run: start retry metrics at attempt 1 via the explicit-start plumbing once the full-duplex branch lands here. * fix(csc): non-blocking GC teardown, classify attributed refresh replies The drop-without-Close GC finalizer set cscActive false BEFORE signalling the refresher, so refreshInvalidatedBatch bailed and the stop-drain flush was a no-op — in-window hot keys stayed evicted for a surviving sibling. And clearRefreshQueue / setRefreshQueue / releaseLocked called the batcher join() while holding h.mu, blocking the GC finalizer (documented non-blocking) and stalling a sibling in ensureBatcher on the hot path (miss-coalescer reader). Detach the batcher under h.mu (detachBatcherLocked, stop-only) and join OUTSIDE the lock — on the Close / self-disable / attach paths, and NOT AT ALL on the GC finalizer, which is reordered (clearRefreshQueue -> refresher signalStop -> deactivate) so its async stop-drain is runnable while serving is still on. The guaranteed final-flush window stays the Close-path join; the finalizer stays non-blocking. setInvalBatchWindow gets the same treatment. Also: the refresher's cacheability check inspected raw[0], but a RESP3 attribute-prefixed error (|... frame) has raw[0]==RespAttr, so an error reply was published as a false refresh success (next reader must evict + refetch). Classify the FULL reply (cscRefreshReplyCacheable via classifyCachedReply), like the miss-coalescer. Addresses the GC-cleanup teardown findings and the attributed-error refresh finding on #3989. * fix(csc): cancel stranded miss on shutdown, charge wire, bound spill Three bounds/lifecycle fixes on the coalescer and invalidation batcher. Stranded miss: fetch's mc.stop wait branch cancelled the reservation and re-drained mc.ch, but the wctx.Done (caller-deadline) branch only claimAbandoned, leaving a post-drain request IN_PROGRESS so a later Get waited out StaleTimeout. cancelIfStopping now cancels + re-drains on that branch too — gated on mc.stop being observed, so a healthy late-cancel does not no-op a live reader's fulfillCached (token-guarded cache populate). Wire budget: cmdApproxBytes charges ~8 bytes for variable-size non-key args (BinaryMarshaler, *string), so the 8 MiB in-flight wire budget under-counted. reconcileWireBytes charges the delta between the estimate and the actual len(req.wire) after serialization, keeping req.reserved in lockstep (settle releases exactly once) so the counter reflects reality and the next reserve sheds. Invalidation spill: the spill was bounded by item count only; a burst of distinct large namespaced keys retained GBs below the item cap. Track retained key bytes (cscInvalSpillMaxBytes = 8 MiB) and take the same full-Flush fallback when either bound trips (a single oversized key trips it alone). Addresses the shutdown-strand, wire-size-budget, and invalidation-backlog-bytes findings on #3989. * fix(csc): shed a coalesced miss when its real wire exceeds budget The wire budget is reserved from cmdApproxBytes before serialization, then reconciled to the actual size after. Round 5 only corrected the counter and kept an over-budget wire, so N concurrent misses with undercounted variable-size args (BinaryMarshaler, *string, net.IP) each pass the pre-serialize gate and then charge their true size, drifting the in-flight total well past cscMissWireBudgetBytes while every snapshot stays queued. reconcileWireBytes now returns keep/shed: when charging the actual size would cross the cap it rolls the delta back, leaves req.reserved at the estimate (fetch's !enqueued defer releases exactly that), and fetch sheds to the pooled path — so the over-budget wire is dropped, not retained in mc.ch/inflight. A shrinking delta always keeps. Pinned by TestCSCMissCoalescerShedsWhenActualOverBudget. Addresses the recheck-wire-budget-after-serialization finding on #3989. * fix(csc): bound invalidation channel bytes, not just spill Round 5 bounded the overflow spill by retained key bytes, but the fast-path channel admits by item COUNT only: a burst of large keys — or one key larger than the whole byte cap — retains unbounded bytes in ch before the spill path ever engages. Track ch retained key bytes (chBytes): enqueue adds on a successful ch send and the worker subtracts on consume. A pre-admission gate diverts to the full-Flush fallback when this key alone exceeds the byte cap, or when the ch bytes plus this key would cross it, instead of admitting to ch. Pinned by TestInvalBatcherChByteGateTripsFlush and TestInvalBatcherOversizedKeyTripsFlush. Addresses the invalidation-byte-cap-before-channel-admission finding on #3989. * docs(csc): note refresh-stats scope on a shared cache CSCRefreshStats mixes cache-global counters (Invalidations/Deletions/DeletionsNoop, read from the shared LocalCache) with this-client's-queue counters (Enqueued/Refreshed/...). On a shared cache+processor the active refresh binding is the last-attached client's queue, so an earlier client reports every invalidation but ~zero refresh work — correct (it is not the refreshing client), but a naive refresh/invalidation ratio then reads as broken. Document that the ratio is meaningful only single-client or for the client holding the active binding. No behavior change (scoping the counters per-client would change what Invalidations means for existing readers). Addresses the keep-shared-cache-refresh-statistics-in-one-scope finding on #3989. * fix(csc): reserve invalidation channel bytes atomically The ch byte gate was a non-atomic Load then Add-after-send: concurrent producers (invalidations on multiple tracked conns) could each read a below-cap chBytes, all send a large key, then all Add — overshooting cscInvalSpillMaxBytes while the counter briefly read near zero. Reserve atomically before the send (add, and roll back if it crosses the cap or the ch is full, mirroring reserveWireBytes); an over-cap reservation diverts to the full-Flush fallback. Pinned by TestInvalBatcherChByteReservationIsAtomic. Addresses the reserve-invalidation-channel-bytes-atomically finding on #3989. * fix(csc): generation-tag refresh demand notifications A reader that passed signalDemand's pendingSet check just before flush cleared the window could send its nudge AFTER the one-time drain, and that stale nudge then flushed the NEXT (unrelated) window early — inflating DemandFlushes and defeating the coalescing window. Stamp each nudge with the window generation (read before the membership check); flush bumps the generation on clear; the refresher ignores a nudge whose generation is retired (demandIsCurrent). Pinned by TestCSCRefreshDemandGenerationIgnoresStaleSignal. Addresses the generation-tag-demand-notifications finding on #3989. * fix(csc): honor caller cancellation while enqueuing a miss fetch used wctx (Background when ContextTimeoutEnabled is false) in the enqueue select, so a caller that cancelled could not abort while merely WAITING FOR A QUEUE SLOT — pre-I/O backpressure — even though the ordinary path passes the caller ctx to ConnPool.Get. The enqueue-admission select now watches the original ctx; wctx stays only on the reply wait, which maps to socket I/O and follows the timeout policy. Pinned by TestCSCMissCoalescerEnqueueHonorsCallerCancel. Addresses the keep-caller-cancellation-while-waiting-to-enqueue finding on #3989. * fix(csc): don't re-run a limiter-denied coalesced miss When the coalescer's session acquisition failed because Options.Limiter.Allow REJECTED admission, the rejection was tagged cscSessionError like a transport failure, so processCached re-ran the command through processWithRetry -> getConn -> Limiter.Allow a SECOND time. A stateful or token-based limiter can permit that second call, executing the very miss whose original admission was denied. Split the admission from the dial (getConnLimited: getConn wraps it, byte-identical for existing callers) so the coalescer can tell an Allow rejection apart from a dial/transport failure. On a rejection, settle the caller with the error UNTAGGED (settlePlain/drainQueuePlain) so processCached returns it as-is — one Allow total, matching the ordinary path (shouldRetry is false for a generic limiter error). A genuine transport failure keeps its cscSessionError retry behavior. Pinned by TestGetConnLimitedSurfacesRejection. Addresses the preserve-limiter-rejections-during-session-acquisition finding on #3989. * fix(csc): unify teardown ordering across close and self-disable Three interrelated CSC teardown findings, resolved by one canonical order. New once-guarded stopCSCRefresherAndCoalescer runs: stop the coalescer (releasing its held pool connection) -> run the refresher's final flush while cscActive is still true -> deactivate. Called from both the clean Close path and the drainer goroutine's self-disable defer (neither is the refresher/coalescer goroutine, so the joins never self-block). - Self-disable (disableCSCServing / custom-processor damping / drainer seeing cscActive go false) previously exited without stopping the refresher or coalescer — the goroutines and a held pool conn leaked for the client's life. It now runs the same teardown. Start guards on startCSCRefresher/ startCSCMissCoalescer close a construction-time disable window. - Handler-initiated close (a push handler calling Client.Close) no longer preemptively deactivates + stops the drainer before the canonical close; it lets the async close drive the refresher-first-while-active ordering (mirrors the GC path). - Releasing the coalescer connection before the refresher's flush avoids a Close stall (up to ~160s on a small pool: refresh chunks waiting their deadline while the coalescer held the only conn). Reversing the old deactivate-first step is safe: every coalescer stop path returns errCSCRetryUncached/cscSessionError, so processCached re-runs on the still-open pool — no raw ErrClosed reaches callers. Tests + red-checks in csc_test.go. Deferred: quiescing invalidation producers that snapshotted the queue before unbind (needs a hot-path lock; the residual is a bounded teardown-only refetch, not a stale serve) — see the review reply. Addresses the self-disable-leak, handler-close-ordering, and release-coalescer-before-refresh findings on #3989. * fix(csc): preserve invalidation dedup across a size-cap flush run()'s size-cap flush (len(pending) >= the batch cap) cleared the epoch-scoped seen map without a full-cache Flush, so a duplicate key straddling the cap was applied twice; with refresh-on-invalidate the first apply refetched and published and the second evicted that fresh entry. Split flush into applyPending + clearSeen; the size-cap path applies WITHOUT clearing seen, so per-key dedup spans the whole window (timer/stop/full-flush still clear it). Epoch-bump was rejected: epoch is captured at enqueue and only valid paired with a full Flush, so bumping here would skip in-flight ch/spill items and lose their deletes. Pinned by TestInvalBatcherSizeCapPreservesDedup. Addresses the size-cap-flush-drops-dedup finding on #3989. * fix(csc): bound the shutdown refresh drain on first failure On Close with a large backlog against a dead/stalled server, the stop-drain gave each chunk a fresh 5s deadline and continued after errors — a full queue could delay Close ~160s (hours if large replies shrink chunks). The stop-drain now bails after the first failed chunk (those entries are evicted with this client's coverage on Close anyway). The normal (non-shutdown) refresh path is unchanged. Pinned by TestCSCRefresherStopDrainBailsOnFirstFailure. Addresses the cap-the-entire-shutdown-refresh-drain finding on #3989. * fix(csc): bound concurrent coalesced wire serialization cmdApproxBytes underestimates variable-size args, so many concurrent misses all passed the approx reservation and each allocated a full wire snapshot BEFORE reconcileWireBytes could shed — a concurrent burst transiently coexisted many multi-MB buffers and could exhaust memory even though none stayed queued. Add a serialize semaphore (cap = budget/batch) acquired before writeCmd and released after the reconcile decision, always before the enqueue select, so no slot is held across a channel send or reply wait (deadlock-free with the single worker); a caller that can't get a slot sheds to the pooled path. Pinned by TestCSCMissCoalescerBoundsConcurrentSerialize and the fetch-level shed test. Addresses the bound-concurrent-serialization finding on #3989. * fix(csc): share refresh queue into clones for demand flush clone() (WithTimeout/WithContext) copied the miss coalescer but not cscRefreshQueue, so a derived client's queue was nil and signalDemand no-opped — the clone never early-flushed the owner's in-window batch, so a demand-worthy miss waited the full cscRefreshWindow. Share the queue pointer into the clone (like the pool hook); lifecycle stays safe — the refresher/drainer handles are owner-only (nil in clones) so a clone only signals (nil-safe, non-blocking), never stops. Pinned by TestCloneSharesRefreshQueueForDemand. Addresses the clones-skip-refresh-demand-flush finding on #3989. * fix(csc): guard invalidation deletes by fetch-issue order Round-10 kept the batcher's per-window dedup (seen) across a size-cap flush so a key straddling the cap could not be applied twice — the second apply would evict the entry the first apply's refresh had just refetched. That traded a benign extra miss for two defects: seen grew unbounded under sustained load (the timer that clears it is deferred by every size-cap t.Reset), and a genuine re-invalidation of a refreshed key was deduped away and served stale. Worse, keep-seen could not fix the case where ONE cached command depends on SEVERAL Redis keys (e.g. MGET k1 k2): a refresh triggered by k1's invalidation republishes the entry, and k2's invalidation — a different seen entry — then re-evicts it. seen is keyed by Redis key, so it structurally cannot reason about whether THIS entry was refetched. Move the correctness out of dedup and into the delete. Stamp each entry with the global cscFetchSeq value at the moment its fetch is ISSUED (Reserve), carried through fulfill; snapshot cscFetchSeq when an invalidation is OBSERVED (enqueue). collectHotAndDelete now keeps a Valid entry whose fetchSeq exceeds the invalidation's snapshot: that fetch was issued after the write, so its reply already reflects it. In-progress placeholders are still removed unconditionally. Stamp at fetch ISSUE, not fulfill COMPLETION: invalidations and replies travel on different connections with no ordering, so a stale reply can fulfill AFTER the invalidate is observed (see deleteByRedisKey). A completion-order stamp would then retain that stale value; an issue-order stamp is sound because a fetch issued after the observe necessarily reached a server that had applied the write. With correctness in the guard, dedup lifetime no longer matters: the size-cap path goes back to flush() (apply + clearSeen), so seen is bounded and a genuine re-invalidation is re-applied. A straddling stale duplicate is now a no-op via the guard instead of via dedup. Tests: TestCollectHotMultiKeyRefreshSurvives (the MGET case dedup can't fix), TestCollectHotFetchOrderGuard (stale kept, genuine evicts — the case a fulfill-time stamp fails), and TestInvalBatcherSizeCapGuardKeepsRefreshed (end-to-end, with a DeletionStats noop delta proving the duplicate was applied and guard-skipped, not deduped). Existing collectHotAndDelete callers pass a max snapshot to isolate their recency/stats assertions from the guard. Addresses the size-cap-flush-dedup regression and the multi-key re-eviction on #3989. * fix(csc): extend fetch-order guard to in-progress fetches A delayed invalidation applied while a replacement fetch was still in progress removed the reservation even when entry.fetchSeq > fetchSnap — a fetch reserved AFTER the invalidation was observed, which cannot hold the pre-write value. Canceling it failed its racing Fulfill, woke every waiter as a miss, and could duplicate requests / defeat miss coalescing. The guard was Valid-only; drop that restriction so it keeps any entry (Valid or in-progress) issued after the observe. An entry reserved at or before the observe (fetchSeq <= fetchSnap) may predate the write and is still removed. Pinned by TestCollectHotGuardKeepsInProgress. Addresses the preserve-newer-in-progress-fetches finding on #3989. * fix(push): align idle CSC drain with the push-drain spec drainPushNotifications diverged from peekAndProcessPushNotifications / pushDrainWithin and .claude/specs/push.md on two points. 1. It drained under the raw cscDrainHardReadCap. During maintenance relaxation a push frame can fragment past that small cap, and a mid-frame timeout is a fatal desync that removes the connection and evicts its CSC coverage in the very failover/migration window relaxation is meant to tolerate. Route the drain through pushDrainBudget so the cap is raised to the relaxed timeout while relaxation is active (unchanged otherwise). 2. It invoked a custom push processor whenever the socket was readable (CheckForData), including kernel-only readability that over TLS can be a control record with zero RESP bytes; the custom blocking loop then hit the hard cap and its error — always fatal for a custom processor — retired a healthy connection. Gate the custom processor on the buffered state captured BEFORE the probe: a byte the probe just fetched from the socket/wrapper is not "already buffered", and handing it to a blocking loop is the exact spurious retire push.md warns of. So a custom processor drains only when RESP bytes were already in the reader buffer; kernel-only AND probed bytes both defer to the next pass. Mirrors peekAndProcessPushNotifications (buffered captured before any drain). The built-in buffered processor is unchanged — it still proceeds on kernel-only readability (a boundary timeout is benign). Custom skipping is bounded by the periodic fallback probe and the MaxStaleness backstop. Pinned by TestDrainPushNotifications_RelaxedBudgetToleratesFragmentedFrame and TestDrainPushNotifications_CustomProcessorSkipsKernelOnlyData. Addresses the apply-relaxed-budgets-to-idle-drainer and enforce-custom-processor-gate findings on #3989. * fix(csc): carry the latest fetch snapshot when deduplicating invalidations When two writes to the same key land in one batching window with a cache refetch between them (an entry near MaxStaleness expiring and being re-fetched), the second write's invalidation was deduped and DROPPED, leaving the retained item with the first write's older fetchSnap. At apply the fetch-order guard then preserved the entry refetched BETWEEN the writes (its fetchSeq exceeded the stale snapshot), serving it stale until the next invalidation or MaxStaleness. The dedup now records each key's index in pending and, on a same-epoch duplicate, carries the LATEST (max) fetchSnap onto the retained item, so the guard evicts an entry only refetched before the most recent write. seen and pending are always reset together, so a stored index never outlives its entry. Pinned by TestInvalBatcherDedupKeepsLatestFetchSnap. Addresses the retain-newest-invalidation- snapshot finding (e4xHA). * fix(csc): cancel partial refresh reservations on a CacheSizer panic refreshInvalidatedBatch armed its reservation-cancelling defer AFTER the Reserve loop. A user CacheSizer panics inside Reserve; the outer refresher recovery keeps the worker alive, but the targets reserved so far were left IN_PROGRESS with no defer installed, so every reader of those keys blocked until StaleTimeout. Arm the defer BEFORE the loop so a mid-loop panic cancels the partial set during unwind. Pinned by TestRefreshInvalidatedBatchCancelsReservationsOnSizerPanic. Addresses the cancel- partial-refresh-reservations finding (e8hzZ). * fix(push): keep the short first-byte boundary for speculative drains pushDrainWithin applied the relaxation-aware budget (pushDrainBudget) to the ENTIRE drain, including the first-byte wait. Under active maintenance relaxation a TLS control record — socket-readable with zero RESP bytes — made the speculative/pre-command drain block for the full relaxed timeout (seconds), stalling a ready command and the idle drainer. Bound the first-byte wait to the short cap: under relaxation, probe one byte under the short cap first (a timeout there means no frame began, benign) and grant the relaxed budget only to finish a frame already in flight — mirroring the idle drainer's probe-then-drain. Pinned by TestPushDrainWithinShortBoundaryUnderRelaxation. Addresses the keep-short-boundary-timeout finding (e8hzh). * fix(csc): pair the batcher with the flushed cache on full invalidation On a FLUSHDB/FLUSHALL (nil-payload) invalidation, HandlePushNotification snapshotted the cache at entry but read h.batcher FRESH under the flush lock. A last-user release + rebind to a different cache (A->B) running under h.mu.Lock between the snapshot and the flush left h.batcher = B's while cache = A: dropping B's batcher bumped B's epoch and skipped B's already-queued deletes while only A was flushed, so B served stale entries until MaxStaleness (zero by default). Extract fullFlush and drop the batcher ONLY when it still belongs to the cache being flushed (h.cache == cache). An unchanged binding still drops the fresh batcher — a same-cache batcher rebuild's queued deletes must be superseded — but a rebind no longer drops the new binding's batcher. Pinned by TestInvalidateHandlerFullFlushPairsBatcherWithSnapshotCache. Addresses the P1 keep-batcher-paired-with-the-snapshotted-cache finding (fCCqu). * fix(csc): attribute a cancelled coalesced miss to its sent connection When a caller's context expired after the coalescer session had sent the command but before the reply was claimed, fetch reported the cancellation with a nil connection and returned nil, so the error/operation-duration metrics dropped the peer attributes for a command Redis actually processed. servedBy is unsafe to read on that branch (it races the session's write; the happens-before edge is the req.done receive, which has not happened yet). Add sentConn (atomic.Pointer[pool.Conn]), stored by the session when the batch is assigned to a connection; the cancel branch reads it to attribute the metric and the returned conn, falling back to nil only for a true pre-send cancellation. Metrics-only, no behavior change; the exact send-vs-cancel race is not deterministically unit-pinned. Addresses the record-sent-coalesced-misses-before-cancellation finding (fCCqy). * fix(internal): drain pooled semaphore timer unconditionally putSemTimer gated its stale-fire drain on `!t.Stop()` and skipped it when Stop returned true. Stop's return cannot decide whether a fire is pending: it reports false for an already-expired timer, but a fire can settle into the channel while Stop runs, and Stop can report true in that window. A timer holding an undelivered fire then went back to the pool dirty, and the next Acquire's Reset-and-select read the stale value as an instant timeout. Drain unconditionally with a non-blocking select/default after Stop. It is safe under both timer-channel semantics and never hangs: after Stop the timer is disarmed, so at most one fire can be pending; the default branch runs when none is. Observed in CI on Go 1.24 (oldstable, GODEBUG unset) as a spurious timeout in TestFIFOSemaphoreNoStaleTimerFire. Not reproduced locally on go1.24.13 or go1.25.3 across tens of thousands of iterations; the fix holds by construction, since Stop's return cannot detect a pending fire. * fix(csc): recover coalescer callback panics; honor pre-I/O cancel; bound reply deadline Three coalescer robustness fixes from the #3989 review. Recover a panicking user CacheSizer (via fulfillCached) or custom Cmder reply parse on the full-duplex reader goroutine: applyAndSettle now defers a recover that calls settleErr, so the panic neither crashes the process nor leaves the caller blocked on req.done (the req is already off inflight, so the teardown drain cannot reach it). fetch now releases the serialize slot and cancels the cache reservation on a panic in the writeCmd window (folded into the !enqueued defer + a serializeHeld flag), instead of leaking the slot and pinning the key IN_PROGRESS until StaleTimeout. These are guardrails for the documented 'callbacks must not panic' contract; they mirror the refresher/invalidation- batcher recover pattern and are not separately unit-pinned. Honor caller cancellation while a coalesced miss waits for a session but has not yet reached a connection (req.sentConn == nil), even when ContextTimeoutEnabled is false. The ordinary path passes ctx to ConnPool.Get and only drops to the policy ctx for socket I/O; the reply wait now matches that, then follows the policy once I/O has begun. Arm the reply read under batchBudget (write + read), not bare ReadTimeout: the writer hands a batch to the reader before flushing it (deadlock avoidance), so a reply read can arm while its request is still being written, and a slow/large write within WriteTimeout would spuriously time out and tear down the session. The drain backstop's zero-progress interval is kept strictly above batchBudget so a legitimately slow single reply is not mistaken for a stall. This changes a user-visible reply-timeout bound; no test drives a reply slower than ReadTimeout. Addresses the coalescer-reader-panic (fCCni-689), fetch-panic-leak (532), post-enqueue-cancellation (492) and reply-deadline-during-write (392) findings. * fix(csc): safe cache identity, batcher rebind pairing, per-notification fetch order Three invalidation-handler correctness fixes from the #3989 review. Compare caches with sameCache() instead of ==: h.cache == cache in fullFlush panics on every FLUSHDB/FLUSHALL when the cache's dynamic type is non-comparable (a supported case — sameCache already exists for it), crashing the push goroutine. Pinned by TestInvalidateHandlerFullFlushNonComparableCacheNoPanic. Pair the windowed enqueue batcher with the snapshot cache: after a last-user release + rebind to a different cache (A->B) between the notification snapshot and ensureBatcher, the returned batcher belonged to B and deleted B's entries for a push meant for A. Guard with sameCache(b.cache, cache), else fall through to the inline delete on the snapshot; fullFlush already guarded the same pairing. Snapshot the fetch-order sequence once per notification, not per key: a live per-key cscFetchSeq.Load() included a fetch reserved after the push was observed, so collectHotAndDelete wrongly evicted the fresh value / cancelled its reservation. Mirrors cscInvalItem.fetchSnap (taken at enqueue). The rebind and fetch-order fixes are race-dependent and not separately unit- pinned; the CSC -race suite covers them. Known follow-up: the batcher enqueue path (csc_inval_batch.go) still snapshots cscFetchSeq per enqueue call rather than per notification — a deliberate, larger change left for a focused commit. Addresses the safe-cache-identity (468), enqueue-cache-rebind (409) and per-notification-fetch-order (434) findings. * fix(csc): boundary-probe idle drain; document built-in-processor and hook contract Route the background idle-drainer read through a relaxation-aware boundary probe, mirroring pushDrainWithin: under active maintenance relaxation, confirm a RESP byte under the short hard cap before granting the relaxed budget, so a TLS control record (socket-readable, zero RESP bytes) cannot stall the single-goroutine drainer for the full relaxed timeout (3068). Document the contract instead of chasing every corner case: a custom PushNotificationProcessor is not supported with client-side caching (CSC relies on the built-in processor consuming kernel-only data and tolerating boundary timeouts); the client logs a warning at init in that case. Hooks and user callbacks must call next, must not call Close or other client control methods (deadlock), must not panic, and must not mutate client/connection state. Behavior is undefined otherwise. Documented on Options.ClientSideCacheConfig, AddHook, .claude/specs/push.md, and AGENTS.md. Addresses the custom-processor-staleness (3057) and custom-processor-gate (push.md:35) findings by contract, and the relaxed-idle-drain-probe (3068) finding by fix. * Revert "fix(internal): drain pooled semaphore timer unconditionally" This reverts commit |
||
|
Nedyalko Dyakov
|
8010edc761 |
fix(client): don't leak pools on NewClient panic (#4003)
* fix(client): don't leak pools on NewClient panic NewClient creates the connection pools, then runs later steps that can panic: maintnotifications in ModeEnabled failing, or a custom push processor rejecting handler registration. The panic propagates to the caller (some, e.g. MultiDB AddDatabase, recover it), but the client is never returned, so the pools and their pooled connections leak with no reference left to close them. Add a deferred cleanup that closes the partially-built client if construction panics before NewClient returns. closeResources is nil-safe for a partial client, and the defer does not recover, so the panic still propagates. On the normal path the guard is set before returning and the defer is a no-op. * test(client): deflake NewClient panic-cleanup test The test asserted the pool dialed at least once before checking every dialed conn was closed. The min-idle dial runs in a pool goroutine, and the deferred Close in NewClient can win the race: dialConn then sees the closed pool and never calls the dialer, so opened=0 and the assertion fails even though nothing leaked. Failed on 6/8 CI matrix jobs and 5/30 local -race runs. Make the panic trigger (RegisterHandler) wait for the dialer's first call before panicking, so the leak scenario is what actually runs. Keep a 5s fallback so a missing dial still fails loudly instead of hanging. * fix(client): skip typed-nil pool in panic cleanup The panic-cleanup defer added for the partial-client leak called c.Close() even when newConnPool/newPubSubPool returned a nil pool plus an error: assigning that straight to the pool.Pooler field left a typed-nil interface, so closeResources treated it as present (its != nil check passes) and ConnPool.Close nil-dereferenced — replacing the original construction panic with a nil-pointer panic, visible to callers that recover from NewClient (e.g. MultiDB AddDatabase). Assign the pool fields only after the error check, via a local var, so a failed constructor leaves the field nil (untyped) and cleanup skips it. The pipeline pool already followed this pattern. |
||
|
Nedyalko Dyakov
|
32c371dffc |
fix(autopipeline): add WaitClosed; keep concurrent Close non-blocking (#3998)
AutoPipeliner.Close claims the shutdown with a CAS and drains the already- accepted batches. A concurrent Close that loses the CAS returns nil immediately, which by itself lets a wrapping client (e.g. MultiDBClient.Close) observe "closed" and tear down shared connection pools while the winning Close is still flushing accepted commands. Making the loser BLOCK on the drain would instead self-wait: a re-entrant Close from an in-flight dispatch (a batch's hook calling Close on its own executor goroutine) runs on a goroutine the winner's drain is waiting for, so it would deadlock until the ~30s backstop. So keep Close's loser returning immediately (no self-wait, no regression) and add WaitClosed(): it blocks until the winning Close's drain completes and returns that drain's result. A caller that must not act on "closed" until accepted commands are flushed calls Close and then WaitClosed. This change is additive (new closeDone/closeErr fields, the winner signaling them, and WaitClosed). The wrapping-client wiring — MultiDBClient.Close calling p.Close(); p.WaitClosed() before core.close() — is a follow-up once this lands on master and the MultiDB stack rebases onto it; until then this PR provides the mechanism but MultiDBClient does not yet call it. Test: a losing Close returns immediately; WaitClosed blocks until the drain signals closeDone and returns its result. |
||
|
Nedyalko Dyakov
|
2b238395c4 |
fix(pool): run conn teardown even when already CLOSED (#3985)
* fix(pool): run conn teardown even when already CLOSED initConn marks a rejected connection CLOSED to report an init/auth failure before any teardown runs. Conn.Close early-returned on CLOSED, so the pool's subsequent close leaked the socket and skipped the streaming-credentials unsubscribe and CSC close callbacks. Repeated failures accumulated open file descriptors until GC. Fall through to cleanup regardless of state: callbacks are cleared with an atomic swap (exactly once) and net.Conn.Close is double-close safe. Closes #3982 * fix(pool): close conn transport exactly once Review follow-up for #3985. Falling through to net.Conn.Close on an already-CLOSED connection meant a second or concurrent Close re-closed the socket and, on real transports, returned 'use of closed network connection' — a spurious error the pool's close paths (e.g. closeConnsIf) would propagate, where the pre-fix code returned nil. Claim the transport teardown with an atomic CAS so the socket is closed exactly once and repeat/concurrent closes return nil. The claim can never be stale: the socket is only replaced by SetNetConnAndInitConn, which gates on non-CLOSED states, so no replacement can follow a Close. Tests assert the transport closes exactly once and that repeat Close returns nil (recording net.Conn now errors on the second close, like a real socket). * fix(pool): close transport per socket generation The previous fix guarded net.Conn.Close with a per-Conn transportClosed flag. That flag is claimed once for the lifetime of the Conn, but the socket can be replaced afterwards: a Close racing SetNetConnAndInitConn in the INITIALIZING window closes the original socket and sets the flag, then init stores a replacement socket and unconditionally transitions the conn back to IDLE (Transition is a plain Store). Every later Close then fails the claim and never closes the replacement socket, leaking an FD under the same race TestConnOnCloseRaceWithInitConn exercises. Bind teardown to the transport generation instead: Close atomically swaps the current socket out of netConnAtomic and closes exactly what it took. Each stored socket is closed by the one caller that swaps it out; a socket a concurrent handoff installs afterwards is a fresh generation closed by the next Close. Repeat/concurrent closes swap out a sentinel and return nil, preserving the no-double-close-error contract. Adds a deterministic regression test for the replaced-socket case. The resurrection race itself (a Close during INITIALIZING overwritten by Transition(StateIdle)) is pre-existing and out of scope here; this change makes it not leak, not correct. * fix(pool): keep closed transport retrievable after Close The previous revision swapped a nil-conn sentinel into netConnAtomic on Close to bind teardown to the socket generation. That made getNetConn return nil after Close, changing a pre-existing contract: Conn.RemoteAddr and the connCheck health path then dereferenced nil. Sentinel's Filter closes an in-use conn while leaving it in the pool and later calls cn.RemoteAddr().String(); ConnPool.isHealthyConn calls connCheck(cn.getNetConn()). Both could panic. Bind teardown to the transport generation without clearing it: add a per-wrapper closed flag (atomicNetConn) that Close CAS-claims before calling conn.Close. The wrapper (and its closed conn) stays in place, so getNetConn keeps returning it and RemoteAddr/LocalAddr/connCheck behave as before. Each socket is still closed exactly once, a handoff installs a fresh wrapper so a replacement socket is a new generation closed by the next Close, and repeat/concurrent closes lose the CAS and return nil (no spurious double-close error). Adds TestConn_GetNetConnSurvivesClose for the restored contract; the existing generation and double-close tests still pass. |
||
|
Nedyalko Dyakov
|
c5cad058c7 |
test(tests): TCP-probe before Ping in skip gates (#4001)
Tests that skip when Redis is absent gated on a client Ping to a hardcoded/helper address. A Ping to an addr with nothing listening does not fail fast: it pays the dial-retry (DialerRetries, default 5) plus command-retry budget first, ~1.7s per call. A recent RE integration nightly spent ~1.6 min across 68 such skips. Probe the address with a short-timeout TCP dial first, which fails in microseconds on connection-refused, then keep the existing Ping so a reachable-but-unusable server still skips rather than proceeds. |
||
|
Nedyalko Dyakov
|
911fad2994 |
fix(pipeline): remove duplicate getPipelinePool (#3987)
* fix(pipeline): remove duplicate getPipelinePool PR #3959 branched before sibling #3962 merged; the merge kept both copies of getPipelinePool, and #3962's copy returns *pipelinePoolRef, which no longer satisfies pool.Pooler after #3959's field refactor. Drop the stale duplicate and adapt #3962's getter test: compare against pipelinePool.pool, and reach the nil path via the new always-on opt-out (PipelinePoolSize: -1). * test(command): warm pipeline pool before slowlog count The always-on pipeline pool (PR #3959) dials lazily on the first autopipelined flush, so its HELLO + CLIENT SETINFO handshake is recorded as slow queries once slowlog-log-slower-than is 0, inflating the counted total from 2 to 5 on the autopipeline-subjects run. Warm the pipeline-pool connection before lowering the threshold. The failure was only reachable after the master compile break was fixed. |
||
|
Nedyalko Dyakovandofekshenawa
|
bd0cea4ea8 |
chore(pipeline): Create pipeline pool even without specific buffer passed (#3959)
* fix(pipeline): PipelinePoolSize alone should create the pipeline pool
The dedicated pipeline connection pool was created only when
PipelineReadBufferSize or PipelineWriteBufferSize was set. Setting just
PipelinePoolSize therefore produced no pipeline pool at all, with no error and
no log line, and every pipelined operation silently kept using the main
connection pool — competing with ordinary commands for the same connections,
which is the opposite of what the option asks for.
The gating was inverted. The pool size is the primary intent ("give pipelines
their own N connections"); the buffer sizes are tuning. And the buffer sizes
already supplied a default for the size (10), so the dependency only ran one
way.
Any of the three options now creates the pool. Enabling it by size inherits the
regular ReadBufferSize/WriteBufferSize, so a caller who only wants to bound
pipeline connections no longer has to know that a buffer size is secretly
required.
Found while measuring client-side caching over a high-latency link: the CSC
background batched reads go through withPipelineConn, which falls back to the
regular pool when none exists, so they were contending with the very reader
misses they exist to prevent.
Tests cover all four option combinations and were verified to fail with the
gate reverted.
* feat(pipeline): give autopipelined clients a dedicated pipeline pool by default
Autopipelining is pipeline-heavy by definition, yet the dedicated pipeline
pool still had to be enabled separately; without it every batch competes
with regular commands for main-pool connections. Two paths now default it in:
Config time: Options.init defaults PipelinePoolSize to DefaultPipelinePoolSize
(new exported const, 10) when AutoPipelineOptions is set. Cluster node clients
do not inherit AutoPipelineOptions (the cluster autopipeliner lives at cluster
level and dispatches into the node clients' pipeline hooks), so clientOptions
applies the same default there. Sentinel and failover node clients inherit
AutoPipelineOptions and are covered by Options.init; Ring does not support
autopipelining. A negative PipelinePoolSize opts out explicitly.
Runtime: the common pattern configures the autopipeliner after NewClient
(AutoPipelineWithOptions on a plain client), where pool creation has already
happened. ensurePipelinePool now builds the pool lazily when an autopipeliner
is created. baseClient.pipelinePool becomes a pointer to an atomic slot
(pool + pool-name ref) so the lazy publication is race-free against
concurrent Pipelined callers on the hot path, and WithTimeout clones share
the slot — two sharers cannot install two different pools (CAS; the loser
closes its candidate). Streaming-credentials and maintnotifications hooks are
wired before publication; otel registration happens for the winner. Conn, Tx
and SentinelClient never allocate the slot and keep using the main pool.
The extraction into buildPipelinePool also surfaced that NewFailoverClient
kept a duplicated creation block with the pre-fix gate (buffer sizes only),
so PipelinePoolSize alone silently created no pipeline pool on failover
clients; the shared helper fixes that, with a regression test. Stale
RingOptions/FailoverOptions doc lines still describing the old gate are
corrected.
* feat(pipeline): 64KiB default buffers and no pre-dial for the pipeline pool
The options the dedicated pipeline pool is built with are now resolved by
pipelinePoolOptions, a pure function with pinned tests:
Buffers default to DefaultPipelineBufferSize (new exported const, 64 KiB)
rather than inheriting the regular 32 KiB. Pipeline connections move whole
batches per round trip and earn bigger buffers than per-command traffic:
measured on the autopipeline engine, throughput plateaus around 64 KiB,
gains nothing past ~128 KiB, and very large buffers can regress it. An
explicitly larger regular buffer is kept rather than shrunk, and explicit
pipeline buffer sizes always win. The RESP3 minimum clamp applies as on the
main pool.
MinIdleConns is forced to 0 on the pipeline pool. It previously inherited
the main pool's value through the options clone, so a client with
MinIdleConns set pre-dialed that many pipeline connections at pool creation
— a silent doubling of the idle footprint. The pipeline pool is burst
capacity: its connections dial on demand, and the pool size
(PipelinePoolSize or DefaultPipelinePoolSize) is only a cap, so an unused
pipeline pool holds zero connections.
* test: make pipeline-conn specs aware of the dedicated pipeline pool
The autopipeline-subjects job builds its suite subject by calling
AutoPipelineWithOptions on the shared rawClient, which now (correctly)
creates the dedicated pipeline pool lazily - so client.Pipelined and
client.Pipeline run on pipeline-pool connections, not the main pool's.
Two specs assumed the old sharing:
should Auth asserted main-pool stats (Hits=2/Misses=1/TotalConns=1) after
two Pipelined calls. It now branches on PoolStats.PipelineStats: when the
dedicated pool exists, the two pipelines account there (first dials,
second reuses) and the main pool holds only the BeforeEach FlushDB dial;
in the default subject mode PipelineStats is nil and the original
assertions run unchanged.
should ClientSetInfo issued CLIENT SETINFO through a pipeline and then read
CLIENT INFO through rawClient - with a pipeline pool those are different
connections, and SETINFO is per-connection state. It now reads CLIENT INFO
through the same pipeline that issued the SETINFOs, which is equivalent on
a single-pool client and correct on a split-pool one.
Verified against redis:8 in all three GOREDIS_TEST_SUBJECT modes
(default, ap-blocking, ap-async): Ran 2 of 1543 Specs, 2 Passed.
* test: warm the pipeline-pool conn in AP tests that instrument connections
Two autopipeline unit tests assumed batches execute on the main pool's
already-initialized connection; with the dedicated pipeline pool now
created lazily at autopipeliner creation, a batch's first dispatch dials
and initializes a fresh pipeline-pool connection instead. Both tests now
warm that connection with a real two-command batch first (a lone command
takes the solo fast path through Process on the main pool and would not
touch the pipeline pool):
TestAutoPipelineRetriesOnNetworkError counted the pipeline pool's first
dial as if it were a retry redial (3 dials observed, 2 expected). The
warm-up runs before the write failure is armed, so the count again
measures exactly the one redial the retry performs.
TestAutoPipelineHookPostNextErrorPartialBatch installed a hook that
unconditionally injects a post-next error before the pipeline-pool
connection existed. Connection init runs its handshake pipeline through
the client's hook chain (newConn shares hooksMixin), so the hook failed
the pipeline conn's init itself — retries exhausted, the injected error
was stamped on every command, and the test measured init poisoning
rather than the post-next outcome rule it pins. The warm-up initializes
the connection before the hook is installed.
* docs: TODO(v10) — ClientTracking* and ClientMaintNotifications are stateful
CLIENT TRACKING and CLIENT MAINT_NOTIFICATIONS mutate per-connection
state, exactly like CLIENT SETNAME/SETINFO, but live on cmdable: on a
pooled client they land on an arbitrary pool connection that is
immediately returned to the pool, so the flag applies to a connection
the caller cannot address again. They belong on statefulCmdable, next
to ClientSetName — but the move removes the methods from
Client/UniversalClient/Cmdable (and defining them on both cmdable and
statefulCmdable makes every Pipeline selector ambiguous), so it is a
breaking change deferred to v10. Documented now; the move happens in a
follow-up PR. No behavior change.
* docs: drop the TODO(v10) notes on the per-connection commands
The ClientTracking*/ClientMaintNotifications move to statefulCmdable ships
as its own PR against master (feature/stateful-conn-commands) rather than
waiting for v10, so the TODO notes here are obsolete. Docs only.
* feat(pipeline): always create the pipeline pool; spill on exhaustion
The dedicated pipeline pool is now created unconditionally at
NewClient/NewFailoverClient — like the pubsub pool — instead of behind a
matrix of triggers (pipeline options set, AutoPipelineOptions declared,
or lazily when an autopipeliner was built). It is pure burst capacity:
no pre-dialing (MinIdleConns forced to 0), a small cap
(DefaultPipelinePoolSize) and 64 KiB buffers, so an unused pipeline pool
holds zero connections and costs nothing, while pipelines stop competing
with regular commands for main-pool connections. PipelinePoolSize < 0
opts out and restores the old pipelines-on-the-main-pool behavior.
To keep this from capping heavy concurrent Pipelined callers — who
previously shared the much larger main pool — withPipelineConn spills to
the main pool when the pipeline pool is exhausted (ErrPoolTimeout):
bursts wider than the cap keep their old capacity, spilled pipelines
simply run with regular buffers, and total connections stay bounded by
PoolSize + PipelinePoolSize.
Pipeline-pool connections are excluded from CLIENT TRACKING during init.
Pipelined commands never consult or populate the built-in client-side
cache (only the single-command cached path on main-pool connections
does), so tracking pipeline reads only grew the server's tracking table
and produced invalidation pushes for keys the cache does not hold.
With creation unconditional, the lazy machinery is deleted: the atomic
slot and CAS publication, ensurePipelinePool, the autopipeliner-factory
hooks, the Options.init AutoPipelineOptions defaulting and its cluster
clientOptions mirror. pipelinePool is a plain field set before the
client is visible to any goroutine; WithTimeout clones share it. Net
diff of this commit is negative.
Tests: the creation-rule table becomes always-on plus opt-out; new e2e
coverage for the spill (a pipeline pool of one, held busy by a pipelined
BLPOP, must not queue a second pipeline behind PoolTimeout) and for the
tracking exclusion (CLIENT LIST shows the main-pool connection tracked
and the pipeline-pool connection untracked).
* test: keep HImport mock choreography on the main pool; doc the pool default
The HImport mock tests choreograph exact per-connection sequences — a
pool of one, booms armed to fire on the Nth replayed PREPARE, session
counts pinned per connection. The always-on pipeline pool adds a second
connection whose init replays fieldsets (correct production behavior:
HIMPORT pipelines work on pipeline-pool connections precisely because of
that replay), which fired the armed booms during the pipeline conn's
init instead of the reissue under test and shifted the session counts.
Opt every client in the file out with PipelinePoolSize: -1: the tests
pin HIMPORT reissue/session semantics, not pool routing.
Also states the PipelinePoolSize default plainly in the option docs:
DefaultPipelinePoolSize (10) connections (review feedback).
* test: skip conn-init handshakes in the cluster shard pipeline hooks
The shard-level pipeline hooks in the cluster hook-order specs asserted
Expect(cmds).To(HaveLen(...)) unconditionally. With the always-on
pipeline pool, a node's first pipeline dials a pipeline-pool connection
whose init runs its handshake pipeline through the node client's hook
chain, so the assertion fired on the handshake and the resulting Ginkgo
panic aborted the whole suite. Guard both hooks with the idiom
ring_test.go already uses for the same situation: defer GinkgoRecover()
and pass connection-initialization pipelines (hello/client) through.
The cluster-level hooks are untouched — node connection init never runs
through the cluster client's chain.
* test: adapt plain-pipeline tests to the always-on pipeline pool
TestNoPipelinePoolStats pinned the stats shape of a client without a
pipeline pool by creating a client without pipeline options — a premise
the always-on pool inverts. Having no pool now requires the explicit
opt-out, so the test sets PipelinePoolSize: -1, which is exactly the
configuration whose stats shape it verifies.
TestPipelineRetriesOnNetworkError is the plain-pipeline twin of the AP
dial-count test adapted earlier: pipelines now run on the dedicated
pipeline pool, so the first Exec dialed a fresh pipeline-pool connection
and the test counted it as a phantom retry redial. Warm the pipeline
connection with one Pipelined round trip before arming the write
failure, so the count again measures exactly the one redial the retry
performs.
* fix(pipeline): address #3959 review (spill, caps, UniversalOptions)
- UniversalOptions now exposes PipelineReadBufferSize / PipelineWriteBufferSize /
PipelinePoolSize and maps them through Cluster()/Failover()/Simple(), so
UniversalClient users can size or opt out (PipelinePoolSize < 0) of the now
always-created pipeline pool (ofek's CHANGES_REQUESTED blocker).
- withPipelineConn no longer double-counts the Limiter on spill: it acquires the
main-pool conn directly (not via withConn) and records which pool owns the
conn, so the single deferred ReportResult+release runs once, in order (cursor
HIGH).
- The pipeline pool no longer inherits MaxActiveConns (reset to 0, so the ceiling
is MaxActiveConns + PipelinePoolSize, not ~2x) and uses a short
DefaultPipelinePoolTimeout so a saturated pipeline pool spills promptly instead
of queueing; spill also triggers on ErrPoolExhausted (defensive) and the doc is
corrected. Stale "lazy ensurePipelinePool" comments reworded (eager, set-once).
Adds tests: spill accounts the Limiter exactly once, the clone's cap/timeout
overrides, and the UniversalOptions passthrough.
* fix(pipeline): review follow-ups (docs, de-flake, init-spill, stats)
- Options/FailoverOptions/RingOptions docs: pipeline pool is created by default
(opt out PipelinePoolSize < 0); buffers default to max(regular, 64 KiB).
- pipeline_pool_gate_test.go: poll PoolStats instead of a fixed Sleep (de-flake).
- withPipelineConn now spills to the main pool on a pipeline-conn INIT failure
too (e.g. a fresh dial refused with maxclients), not just Get saturation;
non-saturation Get errors (ctx cancelled / pool closed) still do not spill.
- ClusterClient.PoolStats / Ring.PoolStats aggregate each node/shard pipeline
pool into PoolStats.PipelineStats.
* fix(pipeline): review follow-ups (docs, de-flake, init-spill, stats)
- Options/FailoverOptions/RingOptions docs: pipeline pool is created by default
(opt out PipelinePoolSize < 0); buffers default to max(regular, 64 KiB).
- pipeline_pool_gate_test.go: poll PoolStats instead of a fixed Sleep (de-flake).
- withPipelineConn now spills to the main pool on a pipeline-conn INIT failure
too (e.g. a fresh dial refused with maxclients), not just Get saturation;
non-saturation Get errors (ctx cancelled / pool closed) still do not spill.
- ClusterClient.PoolStats / Ring.PoolStats aggregate each node/shard pipeline
pool into PoolStats.PipelineStats.
- Ring.PoolStats now folds WaitCount/WaitDurationNs/StaleConns into both the
main-pool and pipeline-pool accumulators, matching the cluster fold (were
dropped before).
- pipelinePoolOptions caps the pipeline PoolTimeout at DefaultPipelinePoolTimeout
but honors a caller's SHORTER PoolTimeout (min), so a client tuned to spill
faster is not forced to wait the full default before falling back.
* fix(pipeline): accept pipeline pool settings in URL options
ParseURL/ParseClusterURL/ParseFailoverURL now consume pipeline_pool_size,
pipeline_read_buffer_size and pipeline_write_buffer_size, so a URL such as
redis://host:6379?pipeline_pool_size=-1 applies the documented opt-out instead of
being rejected as an unexpected option. Adds TestParseURLPipelinePoolOptions.
* ci(govulncheck): use stable Go to pick up security patches
setup-go's "1.26.x" resolved to go1.26.5, which govulncheck flags for two
standard-library vulnerabilities fixed in go1.26.6: GO-2026-6090 (crypto/tls)
and GO-2026-5972 (encoding/asn1). Track the latest stable toolchain so future
security patches are picked up automatically instead of pinning a patch.
* fix(pool): size maintnotif defaults for the pipeline pool too
Handoff worker and queue defaults were derived from PoolSize alone, but
the manager hooks the dedicated pipeline pool as well, so its
connections competed for capacity sized as if only the main pool
existed. Options.init now derives the defaults from the combined
ceiling (PoolSize + effective PipelinePoolSize) and widens the
MaxActiveConns bound the same way, since the pipeline pool sits outside
it.
---------
Co-authored-by: ofekshenawa <104765379+ofekshenawa@users.noreply.github.com>
|
||
|
Nedyalko Dyakovandofekshenawa
|
3f417e77fe |
perf(autopipeline): better pool utilization on stranglers - *Client (#3962)
* perf(autopipeline): bound the straggler-hold when the pipeline pool has a free connection The async autopipeline held 1-3 queued "straggler" commands until the in-flight batch's reply landed (~1 RTT) whenever any batch was executing, so on a slow link an uncached command that enqueued behind an in-flight batch paid ~2x RTT. A phase trace on a deterministic 50ms link pinned it: service (round trip) was clean at 1 RTT and the concurrency permits were never exhausted, but the straggler-hold re-armed for a full RTT (uncached p95 stuck at 107ms / 2x RTT at low-to-mid concurrency). Bound that hold to a few silence gaps (stragglerHoldGaps * gap, which tracks the round trip) ONLY when the pipeline pool has a connection to spare (an idle conn, or room to dial — the pipeline pool runs MinIdleConns=0). When the pool is saturated, keep the original long hold: flushing tiny batches into a full pool thrashes it because they cannot coalesce into the deep pipelines the scarce connections need (an unconditional few-ms bound cut throughput ~7x at a squeezed pool). The 30s autoPipelinePermitBackstop remains the flush-path safety ceiling. The pipeline pool is captured once at construction via an in-package assertion and is nil for clients without one (e.g. *ClusterClient), which then keep the prior long hold. Measured (50ms proxy, churn, inflight 1): default pool uncached p95/p99 111->65ms; squeezed pool (pipeline-pool 2, uncached-heavy) no throughput regression; real WAN uncached p99 314->177ms (residual is link jitter, present on the untouched cached path too). * perf(autopipeline): add getPipelinePool accessor so the straggler gate is live The straggler-hold pool gate reaches the pipeline pool via an in-package interface assertion (getPipelinePool() pool.Pooler) in newAutoPipeliner. On master that accessor does not exist, so the assertion fails silently: ap.pipelinePool stays nil, pipelineHasFreeConn() always returns false, and the gate degrades to the conservative long hold — the fix compiles and passes CI while doing nothing. Add the accessor (returns the baseClient pipeline pool) so the assertion succeeds and the gate actually engages. Tests assert the AutoPipeliner captures a non-nil pool (gate live) and the safe nil fallback when no pipeline pool is configured. Note: PR #3959 introduces its own getPipelinePool over a pipelinePoolRef refactor; when both land keep one definition (the #3959 ref-based form) and drop this plain accessor. * fix(autopipeline): honor MaxActiveConns in straggler-hold gate pipelineHasFreeConn() used `IdleLen()>0 || Len()<Size()`, which ignores MaxActiveConns: a pipeline pool with MaxActiveConns < PoolSize and no idle connection reported free even though the flush's Get would hit ErrPoolExhausted, so the straggler-hold shortened its hold into a pool it could not actually acquire from (ofek's named change-request blocker; codex #3962). Add ConnPool.HasFreeCapacity(): it keeps the existing idle / dial-able terms and adds the one the heuristic missed -- MaxActiveConns hard-caps dials on poolSize (newConn returns ErrPoolExhausted at poolSize >= MaxActiveConns). The gate prefers this probe (every *ConnPool implements it) and keeps the old heuristic as a fallback for any other Pooler, so with MaxActiveConns == 0 the result is unchanged. Adds internal/pool tests for the max-active-saturated and no-max-active branches. * fix(autopipeline): tighten straggler gate; route solo via pipeline pool - HasFreeCapacity now counts only USABLE idle conns (usableIdleLen) and requires a free pool turn (new FastSemaphore.Available(), which accounts for in-flight dials) before declaring capacity. Adds an unusable-idle test. - The solo-straggler flush now dispatches via processPipeline (one-command pipeline) on the pipeline pool, so it uses the same pool the gate probes; processPipeline falls back to the main pool when none exists (no-op for default clients). The solo goroutine dispatch (the 2xRTT phase-lock fix) is unchanged. * fix(autopipeline): tighten free-capacity gate; keep CSC on solo flush - HasFreeCapacity: when a dial is required, also require a free dial slot (MaxConcurrentDials below PoolSize can leave all slots busy while a turn is free); and usableIdleLen excludes idle conns marked ShouldHandoff (an OnGet hook would divert them). Adds TestHasFreeCapacityExcludesHandoffIdle. - Solo autopipeline flush routes a cacheable command through the single-command Process path so client-side caching is honored (processPipeline bypassed the processCached branch, dropping CSC hits for one-command flushes); non-cacheable solos still dispatch on the pipeline pool. * fix(autopipeline): keep non-CSC cacheable solos on the pipeline pool A cacheable solo straggler was routed through Process (main pool) whenever the command type was cacheable, even with client-side caching off (the default) — so it ignored the dedicated pipeline pool the straggler gate probed and could contend on a saturated main pool while pipeline capacity sat free. Gate that routing on cscEnabled (captured at construction): only a CSC client sends a cacheable solo through the cache-honoring path; otherwise it dispatches on the pipeline pool. * ci(govulncheck): use stable Go to pick up security patches setup-go's "1.26.x" resolved to go1.26.5, which govulncheck flags for two standard-library vulnerabilities fixed in go1.26.6: GO-2026-6090 (crypto/tls) and GO-2026-5972 (encoding/asn1). Track the latest stable toolchain so future security patches are picked up automatically instead of pinning a patch. * chore(autopipeline): compact review-round comments Comment-only: HasFreeCapacity/usableIdleLen docs reduced to their check invariants; the cacheable-solo CSC gate comments deduplicated into the cscEnabled field doc; test headers condensed. * fix(autopipeline): breaker-aware capacity; live CSC solo gate - HasFreeCapacity reports no capacity while the dial circuit breaker is open (dialErrorsNum >= PoolSize, dialConn's fail-fast condition): room under PoolSize is not dialable capacity when every dial errors immediately, so the straggler gate keeps the conservative hold instead of flushing into a failing dial path. - The cacheable-solo CSC gate is live (the client's autopipelineCSCActive func, consulted per dispatch) instead of a construction-time bool: when CSC disables itself mid-life, cacheable solos return to the pipeline pool. The routing test pins the mid-life flip. * fix(pool): check semaphore turns before idle in HasFreeCapacity Get acquires a turn before popIdle, so with the semaphore exhausted even a usable idle conn is not immediately servable — and putConnWithoutTurn can re-pool a conn while its turn stays held, so a positive idle count does not imply a free turn. The turn check now runs first; the idle short-circuit only applies when a turn is available. --------- Co-authored-by: ofekshenawa <104765379+ofekshenawa@users.noreply.github.com> |
||
|
Nedyalko Dyakov
|
216593cc01 |
test(command): skip CLIENT INFO tracking-flag assert on RE (#3981)
Against a Redis Enterprise database the proxy does not surface the client tracking flags in CLIENT INFO, so 'info.Flags & ClientTracking' (and BCAST) read back zero even though CLIENT TRACKING ON returned OK and client-side caching works. The two CLIENT TRACKING specs failed with 'ClientFlags: 0 not to be zero-valued'. Assert the CLIENT INFO tracking flags only against a direct server (guard with !RECluster). The rest of each spec — the command returning OK, and the flag clearing after TRACKING OFF — still runs everywhere. |
||
|
Nedyalko Dyakov
|
96554bd5a4 |
test(autopipeline): scale timing assertions to measured RTT (#3978)
On a remote endpoint (e.g. the Redis Enterprise integration run) the plain autopipeline tests that assert a fixed local latency bound fail on network latency alone — e.g. TestAutoPipelineSequentialSingleThread: '10 sequential commands took too long: 144ms (should be < 100ms)'. Add apBaselineRTT() (min of a few warmed PINGs) and apLatencyBudget() (scales a per-round-trip budget to the measured RTT, with a floor so localhost stays tight), and use them in the latency-bound checks: the single-command, sequential and MaxFlushDelay plain tests and the ginkgo blocking-command spec. The bounds now hold on both localhost and a remote database while still catching a real per-command coalescing regression. Also guard TestDoBypassesPipeline (autopipeline_internal_test.go) with the same 'no redis' skip its siblings use — it was the last internal test that hard-failed on an unreachable :6379. |
||
|
Nedyalko Dyakov
|
ceba4fbb9b |
fix(command): tolerate unknown CLIENT INFO flags (#3977)
parseClientInfo (shared by CLIENT INFO and CLIENT LIST) failed the whole reply on any client-flag character it did not recognize: redis: unexpected client info flags(...) New flag characters can appear over time, so a version-safe client must handle them gracefully. Skip unrecognized flag characters instead of erroring, matching the 'skip unknown fields' behaviour already used for unknown fields in the same parser. Known flags still set their bits. |
||
|
Nedyalko Dyakov
|
c37c3485d8 |
test(autopipeline): reach RE database in plain tests (#3976)
Follow-up to #3975, which fixed only the ginkgo Describe blocks. On the Redis Enterprise integration run (RE_CLUSTER=true) the plain TestXxx funcs still built clients from apTestAddr() (hard-wired to :6379) and ran before BeforeSuite, so those without a skip guard hard-failed with 'dial tcp :6379: connect: connection refused'. - new apTestOptions() helper lazily loads reConn (RE_CLUSTER + REDIS_ENDPOINTS_CONFIG_PATH) without depending on BeforeSuite, so the plain redis_test AutoPipeline tests reach the RE database (addr, creds, TLS); - four guardless white-box timing tests in autopipeline_internal_test.go (package redis, which cannot see the redis_test helpers) get the same 'no redis' skip guard as their siblings. DB-15, custom-dialer, ring/cluster and pure-config tests are intentionally left on apTestAddr()/their existing skip guards. |
||
|
Nedyalko Dyakov
|
2a037fd2c9 |
test(autopipeline): use suite DB in Ginkgo specs (#3975)
The three AutoPipeline Ginkgo Describe blocks built their client from
&redis.Options{Addr: apTestAddr()}, which is hard-wired to :6379 (or
REDIS_PORT). Against a Redis Enterprise endpoint (RE_CLUSTER=true) the
address, credentials and TLS come from redisOptions()/BeforeSuite, so
those specs dialed :6379 and failed with 'connection refused'.
Base the Ginkgo clients on redisOptions() like the rest of the suite so
they reach the proper database (RE endpoint + creds + TLS via
applyREConnection). apTestAddr() stays for the plain go tests, which run
without BeforeSuite and self-skip when Redis is unreachable.
|
||
|
Nedyalko Dyakov
|
36d97525cd |
fix(pool): log buffered push data at debug level (#3948)
The buffered-push-data notice in isHealthyConn fired unconditionally. With client-side caching every tracked write parks an invalidate frame on idle pooled conns, so at default log level the line floods the log on nearly every pool Get under write-heavy workloads. It documents the normal healthy path, not a problem - gate it behind LogLevelDebug like the equivalent notices in maintnotifications. |
||
|
Nedyalko Dyakov
|
1acd19a61b | docs: point security reports at Redis VDP (#3949) | ||
|
Nedyalko Dyakov
|
c7f59a2a95 |
chore(release): prepare 9.22.0 (#3947)
- add 9.22.0 release notes (consolidates 9.22.0-beta.1) - bump version to 9.22.0 in version.go and all extra/example go.mod files |
||
|
Nedyalko Dyakov
|
c994cfc85f |
feat(autopipeline): automatic command pipelining (#3942)
* ci: run PR workflows for feature/** base branches PRs stacked against feature/autopipeline/integration (and any feature/** integration branch) didn't trigger build, e2e, govulncheck or codeql because their pull_request.branches filters only matched master/v9/ndyakov/**/etc. A pull_request event filter matches the BASE branch, so PRs targeting a feature branch were skipped. Add 'feature/**' to those filters. * feat(autopipeline): #1 base types and client wiring (#3864) * perf(proto): read payload+CRLF in one shot in ReadStringInto When the caller-supplied buffer has room for the payload plus the trailing CRLF, read both in a single io.ReadFull instead of a separate line read. This is the zero-copy fast path used by GetToBuffer. * feat(autopipeline): add AutoPipeliner with deferred typed results Introduce AutoPipeliner, a background batcher that coalesces concurrent commands from many goroutines into Redis pipelines for high throughput. Engine (autopipeline.go): - Sharded command queue (per-shard mutex + flusher + FIFO semaphore), round-robin enqueue to cut contention under concurrency. - Batch completion signalled by a single close(batch.done) instead of per-command channel sends. - AutoPipelineConfig{MaxBatchSize, MaxConcurrentBatches, Unordered, MaxFlushDelay, AdaptiveDelay}. Default is ordered (MaxConcurrentBatches=1). Validate() rejects MaxConcurrentBatches>1 without Unordered:true so callers can't silently lose ordering; NewAutoPipeliner panics on an invalid config. Deferred results (command.go): - baseCmd gains a ready channel + setReady/await/rawErr. Every exported value-accessor (Val/Result/Text/Int/Bytes/Bool/Float/Time/Scan/First/Item/ ...) awaits the result before reading cmd.val/cmd.err; delegating accessors inherit the await through the method they call. Execution-path reads use rawErr() to avoid awaiting the very batch they produce (self-deadlock). - Cmder interface extended with setReady/await/rawErr. Pooling (pipeline.go) reuses Pipeline/cmd-slice objects across batches. osscluster.go and redis.go use rawErr() on the pipeline execution path. * feat(autopipeline): return an error from constructors instead of panicking newAutoPipeliner / NewAutoPipeliner validated the config and panicked on a bad one. Since these run from the post-init AutoPipeline / AsyncAutoPipeline calls (not client construction), a panic is the wrong failure mode — a deterministic config mistake should surface as a returned error the caller can handle. Return (*AutoPipeliner, error); the config is rejected before any flusher goroutine starts, so the error path leaks nothing. * fix(autopipeline): don't cap the shutdown flush with a hardcoded 2s timeout flushBatchSliceShutdown wrapped the final drain in context.WithTimeout(2s). Because the effective wire deadline is min(ctx deadline, conn timeout), that 2s capped everything — including a relaxed read/write timeout set by maintnotifications during a failover/migration. In-flight commands on shutdown would then time out at 2s instead of honoring the relaxed window the relaxation was designed to protect. ap.ctx is already cancelled at this point, so use a plain context.Background() with no artificial deadline, matching the normal flush path and a plain client Exec: the connection's ReadTimeout/WriteTimeout (and any relaxed override) governs the wire deadline. A user who wants shutdown bounded sets those, as for any command. * fix(autopipeline): correct permit-wait timeout and error in flushBatchSlice The flush waited a hardcoded 5s for a concurrency permit and, on any non-nil result, errored every command in the batch with ErrClosed. Two problems: - ErrClosed is wrong when the wait simply timed out — the client is not closed. Callers saw a misleading closed error for a deadline. - 5s is below a maintnotifications relaxed window, so a legitimately slow in-flight batch (e.g. during a failover) holding the permits could make waiters fail spuriously. Distinguish a real shutdown (ap.ctx cancelled -> ErrClosed) from the backstop firing (-> the timeout error as-is), and raise the backstop to 30s — well above the default ReadTimeout and a relaxed window. It remains only a safety net: every permit holder releases via defer and each batch Exec is bounded by the conn timeout, and the wait still ends immediately when Close cancels ap.ctx. * docs(autopipeline): fix usage example to the error-returning API The AutoPipeliner type doc showed 'ap := client.AutoPipeline()', the old single-return form. AutoPipeline / AsyncAutoPipeline now return (*AutoPipeliner, error); update the example to assign and check err so the base reflects the final API surface. * refactor(autopipeline): drop orphan AutoPipeline() accessor stubs (*Pipeline).AutoPipeline() returned a silent nil and (*AutoPipeliner). AutoPipeline() returned itself, with a comment claiming they satisfy the Cmdable interface. They don't — neither Cmdable nor Pipeliner declares AutoPipeline(), so these methods satisfy nothing and the nil-returning Pipeline stub is a latent nil-deref footgun. The real entry points are Client/ClusterClient.AutoPipeline(config...) (*AutoPipeliner, error). Remove both stubs; the var _ Cmdable assertion still holds. * fix(autopipeline): Process honors mode; unexport constructor; doc fixes - Process() now routes through ap.cmdable like Do and the typed methods, so on a blocking autopipeliner it blocks instead of always running async. Previously it hardcoded processAsync, diverging from every other entry point. - Unexport NewAutoPipeliner -> newAutoPipeliner. The exported form took an unexported cmdableClient param and skipped cluster slot-sharding, silently degrading a *ClusterClient to cross-node batches. The public entry points (Client/ClusterClient.AutoPipeline/AsyncAutoPipeline) wire sharding correctly. - Fix stale docs: Do no longer claims it always 'returns immediately' (it blocks in blocking mode); TxPipeline/TxPipelined drop the false 'will panic for other clients' note (cmdableClient embeds Cmdable, so TxPipeline always exists). * ci: run PR workflows for feature/** base branches PRs stacked against a feature/** integration branch (e.g. feature/autopipeline/integration) didn't trigger build, e2e, govulncheck or codeql: a pull_request event filter matches the BASE branch, and those filters only listed master/v9/ndyakov/** etc. Add 'feature/**' so stacked PRs run CI. * fix(json): avoid autopipeline deadlock and stale reads in JSON commands JSONCmd/JSONSliceCmd readReply called baseCmd.Err(), which now awaits the autopipeline batch's done channel. readReply runs inside that batch's Exec, before done is closed, so a deferred JSON command would deadlock. Use rawErr() there instead. Also add the missing await() to JSONCmd.Expanded and JSONSliceCmd.Val/Result so deferred callers block for the result instead of reading empty/stale data. Reported by Cursor Bugbot on PR #3864. * ci: exempt autopipeline engine files from the unused linter The autopipeline stack defines its engine on lower branches and wires the callers in on top, so those symbols read as unused until the stack is assembled, failing golangci-lint on the base/pool PRs. Scope the unused exemption to autopipeline.go and pipeline.go only; unused stays enforced everywhere else. * docs(autopipeline): MaxBatchSize default is 200, not 100 DefaultAutoPipelineConfig and the zero-value fill both use 200; the field doc said 100. Reported by Copilot on PR #3865. * fix(search): await deferred FTHybrid cursor/raw accessors; clarify MaxBatchSize - FTHybridCmd.CursorResult/CursorVal/RawVal/RawResult read their fields without await(), unlike Val/Result. On a deferred autopipeline that returns empty/stale data before the command executes. Add await() to match. (readReply sets the fields directly, so this can't re-introduce the batch-Exec deadlock.) - Clarify the MaxBatchSize doc: it is a soft accumulation threshold, not a hard cap; under burst the batch can exceed it and execute as one larger (safe) pipeline. Reported by Cursor Bugbot on PR #3864. * fix(autopipeline): drain shards in Close after flushers exit A command could pass enqueue's under-lock closed-recheck moments before Close's CompareAndSwap, then append to a shard AFTER that shard's flusher had already drained and exited — leaving its batch.done unclosed, so the caller's blocking accessor hung forever (lost command). Close now drains every shard once more after wg.Wait() (flushers provably gone). The drain loop checks queue emptiness UNDER s.mu instead of via the unlocked s.Len(), so it cannot miss a command appended concurrently. This pairs with the closed-recheck in enqueue: s.mu serializes them, so either the late enqueue appends first and the sweep flushes it, or the sweep empties first and the enqueue then sees closed==true and rejects with ErrClosed. Fixes the hang in TestAutoPipelineCloseRace. * address pr comments * search needs to wait on raw result * feat(autopipeline): #2 add pipeline pool (#3865) * perf(autopipeline): flush once the queue stops growing on the default window accumulateBatch waited the full defaultAccumulateWindow (200µs) whenever the queue was below MaxBatchSize, even for a single blocking caller with nothing left to coalesce, so every low-concurrency command paid the whole window. The doc described a "stops growing" early exit that was never implemented. Implement it for the implicit default window only: a burst timer fires when no new command arrives for defaultAccumulateGap (20µs); each enqueue resets it, so concurrent load keeps coalescing up to the window cap while a lone caller flushes ~one gap after enqueuing. An explicit MaxFlushDelay / AdaptiveDelay is an intentional accumulation window and is still waited in full. defaultAccumulateWindow becomes a var so tests can enlarge it deterministically. * feat(autopipeline): default the blocking face to a single ordered batch stream DefaultBlockingAutoPipelineConfig used MaxConcurrentBatches:50 "for throughput", but that is pessimal. With many flush permits, each command finds a free permit and flushes on its own before others accumulate, collapsing batch size toward one command per round-trip — i.e. non-pipelined throughput at high latency. Backpressure from a single in-flight batch (MaxConcurrentBatches:1) instead lets callers whose commands return re-enqueue and flush together as the next batch, so batches stay deep (a near-continuous, double-buffered pipeline). Measured (get=80/set=20, 64B, localhost, 128 blocking workers): before (b=50): ~59k ops/s, p50 2.9ms, avg batch 1.6 after (b=1): ~420k ops/s, p50 0.30ms, avg batch ~94 Higher throughput AND ~10x lower latency, and it is naturally ordered. The async default (DefaultAutoPipelineConfig) is already MaxConcurrentBatches:1; for maximum throughput use AsyncAutoPipeline with a window of in-flight commands (inflight>1). * feat(pool): add optional dedicated pipeline connection pool Add an opt-in separate connection pool for pipeline/autopipeline execution (PipelinePoolSize), so heavy pipelining doesn't starve the regular command pool. withPipelineConn draws from it when configured and falls back to the main pool otherwise, draining push notifications before returning a connection. Attach the existing per-pool listeners to the dedicated pool as well, so its connections get the same lifecycle handling as the main pool: - streaming credentials re-auth hook - maintnotifications handoff hook (MOVING/MIGRATING). Because that hook's failed-handoff removal target is its own single pool field, the pipeline pool gets an independent hook (Manager.InitPoolHookForPool) bound to it, rather than sharing the main pool's hook — otherwise a pipeline-pool connection that failed handoff would be removed from the wrong pool. The extra hook shares the Manager, so MOVING/MIGRATING tracking stays central, and is shut down and detached on Manager.Close. * docs(options): correct pipeline buffer recommendation to 64-128 KiB The recommended 512 KiB overstated the benefit. Benchmarks show pipeline throughput climbs from the 32 KiB default to ~64 KiB and then plateaus; beyond ~128 KiB there is no further gain and very large buffers (>=512 KiB) can regress throughput and waste memory. The only effect of the buffer is holding one batch's wire bytes so it flushes in a single syscall, so size it to roughly MaxBatchSize x average-command-bytes. Update the recommendation, example, and memory-impact figures accordingly. * feat(pool): register the dedicated pipeline pool with OTel The optional pipeline connection pool was created, used and closed but never registered with the OTel recorder, so its connection-count / pending-request metrics were invisible. Add a pipelinePool param to otel.RegisterPools / UnregisterPools (registered as a regular pool under a _pipeline name suffix) and pass c.pipelinePool from NewClient and the client teardown. nil when no pipeline pool is configured, so default clients are unaffected. * fix(pool): limiter on pipeline-pool path, hook-attach race, test addr Address Copilot review on PR #3865: - withPipelineConn now honors opt.Limiter (Allow() before acquiring, ReportResult() on every exit incl. the early init/re-acquire failures), mirroring getConn/releaseConn. Enabling the pipeline pool no longer silently bypasses throttling and failure reporting. - InitPoolHookForPool now holds hooksMu across the closed-check, append and AddPoolHook, closing a (construction-only) window where Close could tear the hook down between append and attach, leaking it. AddPoolHook is lock-free and never re-enters the manager, so this is deadlock-safe. - pipeline_buffer_test.go uses the shared redisAddr instead of a hardcoded localhost:6379 so it respects the env/CI Redis address. * perf(autopipeline): decouple shard count from MaxConcurrentBatches Shard count was derived as min(GOMAXPROCS, MaxConcurrentBatches, 16), which silently fragmented the command queue as soon as a user raised MaxConcurrentBatches for throughput: 16 shards each accumulate a sliver of the load, so batches collapse toward one command per flush and pipelining disappears. Measured on a 14-core host (get=80/set=20, 64B, 128 goroutines, async window 8): MaxConcurrentBatches=50 meant 16 shards and 413k ops/s at 1.4ms p50, while the identical permit budget on ONE shard reaches 1.75M ops/s at 0.51ms p50 — the fragmentation, not the permit count, was the regression. Add AutoPipelineConfig.NumShards (0 = auto: one shard) and stop deriving the shard count from the permit budget. One deep queue plus 2-4 permits is the sweet spot: batch N+1 accumulates and executes while batch N's replies are in flight (async, window 8: 1.78M ops/s at 0.53ms p50 versus 1.0M at 1.0ms with a single permit). Every shard keeps at least one permit, so the effective global concurrency is max(NumShards, MaxConcurrentBatches). numAutoPipelineShards stays as the cluster-wiring default, where slot-routed shards keep each batch on one node. * perf(autopipeline): stripe the enqueue queue where ordering permits Profiling the windowed-GET throughput benchmark showed the shard enqueue mutex as the hottest lock in the engine: with 128 concurrent callers on one shard, lock/unlock slow paths (parking and waking contended waiters) accounted for roughly half of all CPU samples. Split the shard's queue into 8 stripes, each with its own mutex, queue, and batch-completion signal; enqueue picks a stripe round-robin, and the flusher drains every stripe into one merged pipeline so batches stay as deep as before. Striping applies where reordering is impossible or waived: unordered configs, and every blocking-face shard (a blocking caller waits for each command before issuing its next, so stripes cannot reorder its stream). The async ordered mode keeps a single stripe — with several stripes a caller's consecutive commands could land on opposite sides of an in-progress drain and execute out of order. Stripes are padded so neighbouring mutexes do not share a cache line. Measured (14-core, in-process throughput bench): windowed GET 5.2M -> 6.0M ops/sec; blocking mixed load 0.60M -> 0.77M ops/sec. External-process harness shows +2-7% with lower p99 (client CPU is not the bottleneck there). * fix(autopipeline): resolve pre-ship review findings in the engine Review of the feature (five independent passes; the two ordering defects were reproduced against a live server) surfaced correctness holes. Engine fixes: - Do executes on a NORMAL connection, outside the pipeline. Raw command names can carry connection state (SELECT/MULTI/SUBSCRIBE/CLIENT ...) or block the connection (BLPOP ...); batched onto a shared pipeline conn they silently poisoned it for every later batch, or stalled and retried unrelated commands. Do now carries plain Client.Do semantics (documented); both faces keep their call shape (blocking waits; async defers via a ready channel). Zero-arg Do reports a real error instead of ErrClosed. - Submit is rejected on the blocking face. Submit does not wait, so a windowed caller defeats the invariant the blocking face's enqueue striping relies on ("every caller waits per command") — reproduced as a deterministic reorder of 25% of adjacent submissions from a single goroutine. The deferred face is the intended home for windowed submission and is unchanged. - NumShards > 1 without Unordered fails construction on the deferred face: round-robin sharding flushes concurrently and does not preserve submit order (reproduced: 42% of windowed pairs reordered at NumShards=4). The blocking face is exempt (callers wait; Submit rejected), as is cluster slot sharding via the new unexported contentSharded marker (same-key commands share a shard, so per-key order holds). - Close no longer errors already-accepted commands: batch execution and permit waits run on a background context (wire timeouts still bound them; Close waits via wg/batchWg before pools tear down), and the shutdown drain now acquires the shard permit so a drained command cannot execute concurrently with — and be observed out of order against — a still-in-flight batch. - Validate rejects AdaptiveDelay with MaxFlushDelay <= 0, which silently disabled batch accumulation entirely while reading as a smart-tuning knob. - numAutoPipelineShards (the cluster wiring default) no longer derives from MaxConcurrentBatches: that coupling collapsed cluster slot routing to a single shard at the default budget, making per-node sharding dead code. - Minors: apStripe padded to a 128-byte stride (the 88-byte stride false-shared hot fields with the next stripe's mutex), 32-bit-safe unsigned modulo in the shard pick (int conversion panicked after counter wrap on GOARCH=386/arm), zero AutoFuture.Wait returns an error instead of nil-panicking, stale "maximizes latency" wording corrected. * fix(pool): pipeline pool survives clone; maintnotifications retires it too Two integration gaps from the pre-ship review: - baseClient.clone() dropped pipelinePool, so a WithTimeout clone silently fell back to the main pool: configured pipeline buffers and pool sizing evaporated with no error, and pipeline traffic polluted main-pool metrics. - The maintnotifications runtime downgrade retired tracked connections only through the primary pool; pipeline-pool connections (tracked identically via initConn) stayed in service with the feature enabled but no hook attached, silently dropping MOVING/MIGRATING pushes. Retire now offers the snapshot to every pool the manager attached a hook to — RetireConns skips connections a pool does not own, so this is safe by construction. * polish(autopipeline): pre-ship review polish — accessors, WaitContext, tidy From the deep review (improvements/docs/dead-code passes): - AutoFuture.WaitContext: ctx-aware wait (abandons the wait, never the command — per-command contexts are still not honored after enqueue, and now there is an escape hatch for callers that need one). - IsBlocking() and Config() accessors: code handed an *AutoPipeliner can tell which face it holds and what the effective (default-filled) config is, instead of probing Submit for an error. - Operator breadcrumb when the 30s permit backstop expires (both the flush and shutdown paths) — previously the one "should never happen" wedge case failed batches silently, with no internal.Logger call anywhere in the engine. - Static errors are errors.New sentinels (errSubmitBlockingFace, errZeroAutoFuture, errDoNoArgs) per house style. - Micro: putQueueSlice zeroes only the used prefix (tail invariant documented; was memclr'ing up to 8 KB per flush for small batches); flushBatchSlice skips provably-empty stripes without locking (safe only in that path — the wake protocol re-observes racing enqueues; the shutdown drain keeps locking). - Docs: the AutoPipeliner type doc's flush-trigger list rendered broken in godoc and implied a lone command never flushes — rewritten, plus a retry caveat (a batch is retried whole on a network error, so non-idempotent commands can execute twice on a dropped connection) and a pointer that Do runs outside the pipeline. Stale backstop comment (claimed Close cancels the permit wait; it deliberately does not) corrected. MaxFlushDelay doc no longer references an unexported identifier. - Dead code: commented-out process() stub removed; unused FIFOSemaphore methods removed (AcquireBlocking, Len, and Close — which would have panicked Release had anyone called it); defaultAccumulateGap is a const; closedBatch renamed completedBatch (it also signals successful direct execution, not just closure). * docs(autopipeline): WaitContext — result may become available later Spell out the post-deadline contract: a ctx error means the wait was abandoned, not the command — the batch is still in flight and may populate the command at any moment. Reading Cmd()'s value or error directly at that point races the executing batch; Wait again and only read after a non-context-error return. * perf(autopipeline): RTT-adaptive coalescing on the default window On a real network the fixed coalescing constants fragment batches. Measured against a Redis Enterprise QA endpoint (RTT ~52ms), 512 workers: when a batch's replies land, hundreds of goroutines wake over several milliseconds, but the flusher flushed 20µs after the first re-enqueues — the herd split into partial batches whose serialization costs a full RTT per split (blocking p50 174ms ≈ 3.3 RTT; manually setting MaxFlushDelay=5ms recovered 7.6k ops/s @ 67ms from 3.4k @ 174ms). On loopback the same constants are exactly right, so no fixed value serves both. The engine executes every batch, so it already knows the round trip: keep an EWMA (alpha 1/8) of batch execution time and derive the default-window timings from it — gap = clamp(RTT/64, 20µs, 3ms) window = clamp(RTT/4, 200µs, 15ms) Loopback RTTs clamp to the historical 20µs/200µs (behavior bit-identical); WAN RTTs widen both so reply-wakeup herds coalesce into one batch per round trip, automatically doing what a hand-tuned MaxFlushDelay did. Explicit MaxFlushDelay / AdaptiveDelay configurations are untouched. The derivation is a pure function (adaptiveCoalesce) for unit testing; EWMA updates are deliberately racy read-modify-writes (losing a sample is harmless, keeps the batch path CAS-free). * perf(autopipeline): merge reply cohorts after a blocked permit wait At MaxConcurrentBatches=1 the flusher takes the queue and THEN blocks on the permit for a full batch round trip. Callers whose replies land just after the take re-submit into the fresh queue during that wait; executing without them splits the herd into two alternating cohorts — each observing two round trips at half throughput — and the state is stable once entered. Measured on a 52ms link: p50 pinned at 120ms (2xRTT) for entire runs at 64 workers x inflight 10 while neighbouring worker counts ran at 1xRTT, and run-to-run bistability at other counts (whichever cohort structure forms at startup persists). After a blocked permit acquire on the default window, debounce the resubmission herd (accumulateBatch's RTT-adaptive gap) and fold the late arrivals into the already-taken batch before executing. The herd collapses back to one batch per round trip. TryAcquire successes (uncontended, or MaxConcurrentBatches>1 with free permits) skip the top-up entirely, and explicit MaxFlushDelay/AdaptiveDelay configs keep their own timing. * perf(autopipeline): replace debounce timer with cohort-tracked coalescing The default coalescing path armed a ~20us debounce timer on every flush. A Go timer that short fires ~1ms late on an idle or low-core host (wakeup latency dominates the requested delay), taxing every low-concurrency command ~5x its round trip: on an in-network endpoint (0.25ms RTT) a lone caller measured p50 1.31ms vs 0.24ms for a plain client. Replace wall-clock guessing with two facts the engine already has: - resubmitDebt: a completed batch of N commands wakes its N waiters together; in a closed loop they resubmit. Credit N at batch completion, debit 1 per enqueue, and hold the flusher until the debt drains - the herd flushes as one deep pipeline, exiting the moment it has landed. An exact per-herd count cannot ratchet into fragmentation the way a smoothed estimate does (an EWMA exit undershoots its own herd and locks into fragmented flushes; measured on a 57ms WAN link). - inFlight: executing batches mean more wakeups are coming. Hold stragglers while anything is in flight (gap-bounded, and below coalesceMinFlush hold them for the next herd rather than burning a pipeline connection on a near-empty flush); when nothing is in flight and no debt is outstanding, the shard is idle and flushes immediately with no timer armed. The silence-gap fallback (200us floor, execEWMA/8, 5ms ceil) only ends waits for callers that left; it never fires in steady state. Also dispatch the single-command fast path in its own goroutine: running Process inline blocked the flusher for a full round trip, and on a slow link a solo straggler then held a landed herd hostage for one RTT whose flush delayed the straggler again - a stable phase-lock at exactly 2x RTT (~25% of runs on a 57ms link). Measured (get=80/set=20, 64B, unordered, MaxConcurrentBatches 50): - loopback: w1 p50 0.156->0.130ms; w64 +16% tp; w256 +14%; w1024 even - in-network 2-vCPU (0.25ms RTT): w1 p50 1.31->0.235ms, 4.7x tp; w16 3.2x tp; w64 2.3x tp; w1024 +4%; w256 -20% (fast-link partial-batch overlap the old timer bought; accepted, fixing it re-fragments WAN) - WAN 57ms: w64/w256/w1024 within baseline variance, phase-lock gone The stops-growing/adaptive-window tests are superseded by autopipeline_coalesce_test.go; autopipeline_stops_growing_test.go and autopipeline_adaptive_rtt_test.go (pipeliner branch) test removed symbols and are replaced by it on rebase. * fix(autopipeline): bound the straggler hold; never erase a fresh herd's debt Two review findings on the cohort coalescing: - The gap-fire stale-debt reset was a blind Store(0) on the pipeliner-global counter. A batch completing on another shard (or racing the fire) between the flusher's last look and the reset would have its just-credited herd erased, fragmenting a herd that is really coming. Reset now requires the debt not to have grown since the timer was last armed, and uses CompareAndSwap so a credit racing the reset itself also survives. - The straggler hold (gap-fire with a near-empty queue while batches are in flight) re-armed unboundedly. With read timeouts disabled a wedged batch pins inFlight forever and up to coalesceMinFlush-1 queued commands would hang with it. The hold is now bounded by autoPipelinePermitBackstop, mirroring the permit wait's safety net. Also drop a stale test comment claiming Options.AutoPipelineConfig is not consulted by the blocking face (it is, when no config argument is given). * refactor(autopipeline): rename debt/herd vocabulary to expected arrivals/wave resubmitDebt reads as if something is owed; nothing is — the counter is an expectation: how many commands the engine knows are about to arrive because it just woke that many waiters. And 'herd' named the woken group by analogy rather than by mechanism. Rename for the next reader: resubmitDebt -> expectedArrivals (announce on completion, account coalesceCohort -> awaitExpectedArrivals per enqueue) cohortGap -> silenceGap (+Floor/Ceil consts) herdExpected -> expectingWave herd / cohort -> wave (comments; 'wave merge' at the permit site) Behavior unchanged; comments rewritten in the same vocabulary. * perf(autopipeline): pad stripes with x/sys/cpu.CacheLinePad The stripe pad was a hand-tuned [80]byte to reach a 128-byte stride. Use golang.org/x/sys/cpu.CacheLinePad instead: it is sized per GOARCH (64 B on x86-64/arm64, 128 B on ppc64, 256 B on s390x), so stripes land on their own cache line on every target rather than a constant that happens to fit the common ones and still false-shares on s390x. x/sys was already in the module graph (indirect); this promotes it to a direct dependency. A contended microbenchmark (8 goroutines, adjacent stripes) shows padded vs unpadded ~16x, and CacheLinePad vs the old [80]byte within noise on arm64 — so no regression, just arch-correct. * perf(autopipeline): lower silence-gap ceiling from 5ms to 2ms * chore(examples,extra): go mod tidy for golang.org/x/sys indirect dep * fix(autopipeline): filter the pipeline pool on Sentinel failover onFailover only filtered connPool, so after a Sentinel promotes a new master the dedicated pipeline pool kept idle connections dialed to the old master and pipelined traffic could keep using them. Filter pipelinePool with the same predicate. Also reword the TestPipelineBufferSizes comment: it verifies end-to-end pipelining with the buffer options set, not the actual socket buffer sizes (which are not observable here). * ci(autopipeline): govulncheck on go 1.26.5 (GO-2026-5856) The govulncheck workflow pinned go-version "1.26.x", which resolves to 1.26.4 and reports GO-2026-5856 (crypto/tls). Pin 1.26.5 (the fixed release), matching the pipeliner/benchmarks branches. * feat(autopipeline): #3 wire into Client and ClusterClient (#3867) * feat(autopipeline): wire AutoPipeliner into Client and ClusterClient Add Client.AutoPipeline() and ClusterClient.AutoPipeline() returning a deferred *AutoPipeliner, plus WithAutoPipeline() and propagation of the pipeline-pool options. Options gains AutoPipelineConfig. * test(autopipeline): cover correctness, ordering, config and cluster routing Standalone and cluster tests for the deferred AutoPipeline API: per-goroutine ordering, no cross-talk or lost commands, error isolation, config validation (panic on MaxConcurrentBatches>1 without Unordered), zero-copy GetToBuffer/SetFromBuffer, adaptive delay, and cluster cross-slot routing. * docs(autopipeline): add README section and 4-way decision-guide example README documents the deferred AutoPipeline API and the ordered-default / Unordered-for-parallel rule. example/autopipeline benchmarks four usage styles (normal blocking, ordered blocking-read, ordered read-later, unordered read-later) and prints a decision guide with throughput and ordering. * feat(autopipeline): AutoPipeline/AsyncAutoPipeline return an error The four AutoPipeline / AsyncAutoPipeline methods (Client and ClusterClient) now return (*AutoPipeliner, error) instead of *AutoPipeliner. These run from post-init calls, so an invalid config must surface as a returned error the caller can handle rather than a panic. On error no instance is cached, so a later call with a valid config still succeeds. Tests updated to the two-value form; the unsafe-config test now asserts an error instead of a panic. * test(autopipeline): drop test for removed AutoPipeline() self-accessor The (*AutoPipeliner).AutoPipeline() stub was removed (orphan, satisfied no interface); drop the test that asserted it returns itself. * test(autopipeline): use AsyncAutoPipeline after unexporting the constructor NewAutoPipeliner is now unexported; the unsafe-config test calls c.AsyncAutoPipeline (same error-on-bad-config contract). Update a stale doc comment reference too. * style: gofmt redis.go and osscluster.go Remove a stray double blank line after the AsyncAutoPipeline methods so golangci-lint's gofmt check passes. * ci(spellcheck): add autopipelining, goroutine(s) to the wordlist The README autopipeline section uses 'autopipelining' and 'goroutine(s)', which aren't in the spellcheck dictionary, failing check-spelling on the PR. Add them to .github/wordlist.txt. * test(autopipeline): fix CI failures on the pipeliner PR Three independent test-job failures, all in test code / the example module: - Blocking-face ginkgo specs panicked: ap.Do returns a generic *redis.Cmd, but the specs type-asserted *StringSliceCmd/*StringCmd. Use *redis.Cmd with its StringSlice()/Text() accessors. - Cluster autopipeline tests failed in the standalone test-ce job: the :7000 tests had no skip guard, and the :16600 tests guarded only on Ping (which passes against unjoined nodes that then return CLUSTERDOWN/MOVED). Add a shared skipIfClusterUnhealthy that gates on CLUSTER INFO cluster_state:ok, and use it in all cluster autopipeline tests. - example/autopipeline didn't compile (govulncheck 'all modules' load error): it called AutoPipeline()/AsyncAutoPipeline() in single-value context after they gained the error return. Assign (ap, err) and panic on err. * ci(spellcheck): add async, runnable to the wordlist The README autopipeline section uses 'async' and 'runnable' in prose; add them to .github/wordlist.txt. Verified the full README passes pyspelling via the exact CI image (jonasbn/github-action-spellcheck:0.62.0). * docs(readme): autopipeline examples use the (ap, err) return AutoPipeline()/AsyncAutoPipeline() return (*AutoPipeliner, error); the README snippets showed single-value calls that wouldn't compile. Capture and check err. Reported by Copilot on PR #3867. * test(autopipeline): cover stops-growing early exit and explicit-delay gating Two white-box tests for the default-window stops-growing behavior: - StopsGrowingDefaultWindow enlarges defaultAccumulateWindow to 1s and asserts a lone caller still flushes in well under it (deterministic, no sub-ms timing). - ExplicitDelayWaitsFullWindow guards the gating: an explicit MaxFlushDelay is still waited in full, so stops-growing never short-circuits a configured window. Config is passed as an argument to AutoPipeline (the blocking face does not read Options.AutoPipelineConfig). * docs(autopipeline): AutoPipeline default is a single ordered stream, not parallel batches * feat(autopipeline): cluster keeps multi-shard default; cover NumShards The engine now defaults to a single shard (deep queue) for standalone clients. Cluster clients still want several shards — commands are routed to shards by slot so each batch stays on one node — so the cluster wiring fills in the numAutoPipelineShards default when the user leaves NumShards at 0, copying the config rather than mutating the caller's struct. Tests cover: standalone defaulting to one shard regardless of the permit budget, an explicit NumShards override, negative-NumShards validation, and the cluster default (copy semantics + explicit passthrough). * test(autopipeline): adaptive-delay test pokes stripe 0 after enqueue striping The white-box adaptive-delay test set the shard's queueLen directly; that counter now lives on the shard's stripes. Point it at stripe 0 (these configs are ordered, so the shard has exactly one stripe). * fix(autopipeline): cluster review fixes — push drain, slot-shard default, docs Cluster-side resolutions from the pre-ship review: - ClusterClient.pipelineReadCmds drains buffered RESP3 push notifications before reading each reply, matching the standalone pipeline and cluster TxPipeline read loops. It was the only push-blind reader: a push frame (e.g. a maintnotifications MOVING notification) arriving mid-batch was consumed as a command's reply, shifting every subsequent reply in the pipeline by one command — and the autopipeliner routes all cluster traffic through this loop. - Cluster wiring uses the decoupled numAutoPipelineShards() default and marks the config contentSharded, restoring per-node slot sharding at the default configs (it had collapsed to one shard, making installAutoPipelineSharding dead code) while remaining exempt from the async NumShards ordering check — slot routing keeps every key on one shard, so per-key order holds. - Docs: ClusterClient.AutoPipeline's godoc was fused into the unexported clusterAutoPipelineConfig helper (the public method rendered undocumented); split and given a per-key ordering qualifier. Stale wording fixed in redis.go ("maximizes latency"), the example ("parallel-batch config"), and command.go (references to never-shipped FutureAutoPipeline/ChanAutoPipeline). - Tests: Submit rejected on the blocking face; NumShards>1 requires Unordered on the async face (blocking and cluster slot-sharded configs exempt); AdaptiveDelay without MaxFlushDelay rejected; Do bypasses batches (a MULTI via Do can no longer corrupt co-batched commands); cluster default config keeps multiple slot-routed shards; stops-growing tests use typed calls now that Do no longer enters the engine. * polish(autopipeline): unify getters, closed-client guard, config fallback, docs Wiring polish from the deep review: - The four AutoPipeline/AsyncAutoPipeline getters (Client and ClusterClient) shared a hand-rolled 15-line caching protocol each; they now delegate to one getOrCreateAutoPipeliner helper, which is also where the new closed-client guard lives: calling a getter after Close returns ErrClosed instead of silently building a fresh pipeliner against closed pools whose flusher goroutines nothing would ever stop. - Options.AutoPipelineConfig now feeds BOTH faces (it was silently ignored by the blocking AutoPipeline — a generically-named option honored by only one of two identically-shaped methods). Each face still falls back to its own default when the option is nil. FailoverOptions gains the AutoPipelineConfig field (failover clients could reach the feature but not configure it) and plumbs it to both the sentinel-backed client and the failover cluster client. - Docs: getter godocs state the Options fallback; ClusterClient.AsyncAutoPipeline documents per-key ordering across nodes; the README's stale "parallel pipelines" claim is corrected to the ordered-stream default, throughput figures are qualified, and the section gains the connection-count benefit plus a caveats paragraph (per-command contexts, blocking commands and Do bypass batching, whole-batch retry duplication, shared cached instance). Buffer-size option docs mention both pipeline entry points. - The scoped "unused" linter exemption for the stacked-PR period is removed: the wiring lands on this branch, so every engine symbol is referenced. Example build artifact gitignored. * docs(example): autopipeline example becomes a usage tour + benchmark The example was a throughput harness only — valuable, but it never showed how to actually use the API (nothing read a value back). It is now two acts: Act 1, a usage tour: the blocking face as a drop-in (goroutines Set+Get with plain-client call shape), the async face's submit-a-window-read-later pattern, Submit/AutoFuture for raw commands (async-only by design, with WaitContext noted), the Do escape hatch (runs on a normal connection outside the pipeline — plain Client.Do semantics), in-code tuning notes (Unordered + MaxConcurrentBatches 2-4, NumShards stays 0, first-call-config-wins caching, dedicated pipeline pool options), and an env-gated cluster tour (REDIS_CLUSTER_ADDRS) showing slot sharding needs no extra code. Act 2, the existing four-way throughput comparison, with review fixes: the unused rdb parameter dropped, the bench-tuned MaxConcurrentBatches: 80 magic number replaced by the recommended 2-4 (with a comment where big numbers belong), the baseline's PoolSize asymmetry explained where it stands, panic replaced by the file's print-and-exit convention, and the stale "parallel pipelines" package comment corrected to the ordered-stream default. Adds the README the example was missing (15 of 17 examples have one): run instructions, what each act shows, a sample results table, and the caveats users must know (per-command contexts, whole-batch retry duplication, per-key cluster ordering). * test(autopipeline): re-point tests that the Do change made vacuous AutoPipeliner.Do now executes on a normal connection outside the engine, so every test that drove ap.Do to exercise batching/flushing/queueing was passing vacuously against the plain-client path. Re-pointed at the engine: - ap.Do(ctx, "SET", ...) becomes the typed ap.Set(...) throughout the ginkgo suite, TestAutoPipelineBasic/MaxFlushDelay/Concurrency/SequentialSingleThread, the sequential tests, and the cluster tests (which now genuinely exercise slot sharding). - The error-isolation spec uses a typed WRONGTYPE failure inside a batch instead of Do("INVALID_COMMAND"), which no longer touches a batch at all. - The queue-length spec moves to the deferred face with a wide explicit window, making Len assertions deterministic and meaningful (they previously observed an untouched queue: always zero). - TestAutoPipelineSecondaryAccessors moves to the deferred face — the missing await() regression it guards is only expressible there (the blocking face waits inside the call, so it could never catch it). Stale reference to the removed autoPipelineCmd type dropped. - The sequential submit-then-read tests move to AsyncAutoPipeline (that is the deferred pattern they describe); TestOrderedModeWindowed likewise, making its "never blocks between submits" claim true. TestFutureFaceOrdering deleted — it duplicated TestBlockingFaceExecutesOnCall. Note: the ginkgo AutoPipeline specs require the full test rig (BeforeSuite spins up cluster/sentinel processes) and are validated in CI; everything reachable locally runs green under -race. * docs(readme): list Automatic pipelining under Features (experimental) Adds the feature-list entry linking to the section, marked experimental and scoped to its intended audience: high-throughput / high-load / scale use cases. The section itself now carries the same experimental note so readers landing via the anchor see it too. * docs(readme): point to the example app from the autopipelining intro The example pointer sat at the end of the caveats paragraph where nobody scrolling the section would find it; it now lives in the section's opening note (and the duplicate tail mention is removed). * test(autopipeline): pin the RTT-adaptive coalescing derivation Unit tests for adaptiveCoalesce (loopback clamps to the historical constants, WAN scales gap/window, ceilings bound the tax, enlarged-floor interaction with the stops-growing test hook) and the execEWMA smoothing behavior. * test(autopipeline): drop tests of removed coalescing internals autopipeline_stops_growing_test.go and autopipeline_adaptive_rtt_test.go exercised defaultAccumulateWindow / adaptiveCoalesce, which no longer exist; autopipeline_coalesce_test.go covers the replacement mechanism. * fix(autopipeline): non-pipelineable command fails itself, not the merged batch mapCmdsByNode returned a batch-wide error when any command's policy failed CanBeUsedInPipeline, so under the opt-in dynamic resolver one caller enqueueing e.g. a multi-shard MGET errored every unrelated caller co-batched in the same autopipeline flush (review finding M9). Set the error on the offending command and continue mapping the rest; the skipped command never joins a node pipeline, is not retried, and its waiter reads the policy error while everyone else proceeds. * docs(example): autopipeline example catches up with the engine - Do guidance: typed blocking commands (BLPop, XRead Block, Wait, ...) are diverted to a normal connection automatically — Do is not the escape hatch for them; and stop suggesting SELECT/SUBSCRIBE through Do (connection-state commands poison a pooled conn with plain Do semantics too — Options.DB and the PubSub API are the answers). - Tuning notes: extra MaxConcurrentBatches permits add overlapping batches bounded by the pipeline pool; with wave-boundary coalescing they no longer fragment batches. Reworded in main.go and the bench comment. - README: refresh sample numbers on the current engine (blocking face 552k -> 633k on the same loopback box) and note the pipeline-hook instrumentation shape in caveats. * ci(autopipeline): wordlist loopback/autopipeliner; go mod tidy example/autopipeline * chore: stop tracking .claude/worktrees gitlinks; gitignore them * perf(autopipeline): dispatch drained batches straight to processPipelineHook The flusher built a Pipeline per batch: one Pipeline alloc, a nil cmds slice growth-reallocating ~log2(N) times, and one interface Process call per command — then Exec handed the slice away, nothing reused. Extend cmdableClient with processPipelineHook (the exact hook-wrapped entry Pipeline.Exec wires, so hooks/OTel see the identical call) and hand the drained queue slices to it directly: zero-copy for single-stripe drains (every ordered shard), one pooled merge for striped drains. Shutdown flush uses the same path. Local A/B (M4 Max, redis 6379, count=6): B/op -12-14% on windowed workloads (p=0.002); blocking-face throughput +10% (1.09M -> 1.21M ops/s); windowed throughput unchanged (RTT-bound). Note: hooks now receive a pooled slice — retaining []Cmder past the hook call was never sanctioned; contents are zeroed on reuse. * perf(autopipeline): skip the shard round-robin atomic when there is one shard len(shards)==1 is the standalone default; every enqueue still bumped ap.next — a shared cache line — for a pick that is constant. Same guard the stripe pick already has. Also adds a goroutine-leak regression test (open/close many autopipelines, assert goroutines return to baseline). Local A/B: unordered dispatch ~-3% ns/op, windowed throughput +3-5%. * test(autopipeline): de-flake TestDoBypassesPipeline Force a real multi-command batch (MaxFlushDelay) so the batched SETs run on the dedicated pipeline pool, instead of racing the single-command fast path onto the MULTI-poisoned normal-pool connection. Fixes intermittent failures under -race and on loaded CI runners. * fix(autopipeline): surface retry-exhaustion errors and route streaming re-auth to the owning pool generalProcessPipeline only set command errors on the early-exit branch, not when the retry loop was exhausted. Manual Pipeline surfaces that via Exec's return value, but AutoPipeline discards the pipeline error and observes results only per-command, so a network failure that exhausted retries silently returned a nil error and zero values. Mirror the early-exit setCmdsErr on exhaustion, guarded by !isRedisError so a per-command redis error keeps its own reply. Streaming-credentials re-auth hardcoded c.connPool even for pipeline-pool connections, so a failed pipeline conn was closed and accounted against the main pool. Route the SingleConnPool base and the auth-error CloseConn to the owning pool via poolForConn (pool.Conn.PoolName). * test(autopipeline): retries, cluster MOVED, and pipeline-pool feature coverage - pipeline and autopipeline retry re-drive: success-after-retry, exhaustion surfaces the error, WRONGTYPE isolation, context-cancel not retried (scriptable bad-conn dialer, no external deps) - cluster MOVED redirect inherited by AutoPipeline via the shared execer, with a MaxRedirects=-1 anchor proving the redirect path is exercised - dedicated pipeline pool: ACL AUTH + SELECT handshake, StreamingCredentialsProvider re-auth, and maintnotifications all verified on pipeline-pool connections * feat(autopipeline): expose AutoPipeline on the UniversalClient interface Add AutoPipeline/AsyncAutoPipeline to the UniversalClient interface. *Client (including sentinel-backed failover clients) and *ClusterClient already implement them; *Ring returns errRingAutoPipelineUnsupported so it satisfies the interface and fails explicitly instead of being silently unavailable. * test(autopipeline): context contract, reply-shape demux, fuzz, backpressure, UniversalClient - AutoPipeline context contract: WaitContext abandons only the wait, caller ctx does not cancel execution, blocking commands honor ctx, submit-after-Close - heterogeneous reply-shape demux (bulk/array/int/map/nil/double/large/binary) under RESP3 and RESP2 - differential fuzz: identical results plain client vs autopipeline over random type-diverse command streams - blocking-command isolation, double-Close, windowed backpressure, bounded soak - AutoPipeline reachable via the UniversalClient interface (Ring returns unsupported) * chore(autopipeline): fix CI — remove unused putPipeliner, pin govulncheck to go 1.26.5 - lint: putPipeliner (pipeline.go) was documented but never wired to a caller, so golangci-lint flagged it (unused); remove it. - govulncheck: pin the scan's Go toolchain to 1.26.5, fixing the crypto/tls standard-library advisory GO-2026-5856 (was resolving to 1.26.4). * feat(autopipeline): AutoPipeliner implements UniversalClient (drop-in) Widen the internal cmdableClient to UniversalClient and delegate the non-batched surface (AddHook, Watch, Subscribe/PSubscribe/SSubscribe, PoolStats, Do, AutoPipeline/AsyncAutoPipeline) to the underlying client, so an *AutoPipeliner can be used anywhere a redis.UniversalClient is expected: data commands are batched, everything else runs on the underlying client. Narrow Do to return *Cmd to match the interface; drop the now-redundant .(*redis.Cmd) assertions in the tests. Close() still closes only the autopipeliner, not the underlying client. This lets tests (the command-suite matrix and maintnotifications e2e) parametrize a single UniversalClient subject that includes the autopipeliner. * refactor(autopipeline): take *AutoPipelineConfig (nil=default) not variadic Address review feedback (cxljs, #3867). The variadic `config ...*AutoPipelineConfig` reads like the functional-options pattern but isn't; a single `*AutoPipelineConfig` where nil selects the default is clearer. Changes AutoPipeline / AsyncAutoPipeline on *Client, *ClusterClient, *Ring, the UniversalClient interface and the AutoPipeliner drop-in, plus getOrCreateAutoPipeliner's override parameter. Call sites now pass nil instead of no argument. README, example and doc comments updated. * fix(autopipeline): PR #3867 review fixes - clone(): copy pipelinePoolName so timeout-derived clients keep the dedicated pipeline pool for re-auth routing (poolForConn) instead of falling back to the main pool. - clusterAutoPipelineConfig: mark contentSharded even when the caller sets an explicit NumShards. A cluster always routes by slot, so per-key order holds regardless of shard count; the async face was previously rejected. - TestFutureFaceTyped: build the deferred face with AsyncAutoPipeline so it actually exercises await()/deferred accessors. - Options docs (sentinel/ring/osscluster): correct the pipeline-pool enable condition — set by PipelineReadBufferSize/PipelineWriteBufferSize; PipelinePoolSize alone does not enable it. - README: define ctx in the AutoPipeline examples so they compile as-is. - Update the cluster shard-config test for the corrected contentSharded behavior. * refactor(autopipeline): AutoPipelineOptions + no-arg / WithOptions methods Rename the config type AutoPipelineConfig -> AutoPipelineOptions (matching the Options naming family: Options, ClusterOptions, ...) and split each face into a clean no-arg method plus an explicit variant: AutoPipeline() // Options.AutoPipelineOptions or default AutoPipelineWithOptions(*AutoPipelineOptions) AsyncAutoPipeline() AsyncAutoPipelineWithOptions(*AutoPipelineOptions) Drops the awkward AutoPipeline(nil) for the common default case while keeping an explicit per-instance override. Both faces remain independent (a client can hold a blocking and an async autopipeliner at once). Also renames Default{,Blocking}AutoPipelineConfig -> ...Options and the *Options AutoPipelineConfig fields -> AutoPipelineOptions. Applied across *Client, *ClusterClient, *Ring, the UniversalClient interface and the AutoPipeliner drop-in; call sites, README and example updated. * docs(autopipeline): reword no-arg -> no-argument (check-spelling) check-spelling flagged "arg" (tokenized from "no-arg") in the README. Use "no-argument" instead. * fix(autopipeline): hooks reading deferred results no longer deadlock A pipeline (or process) hook that reads a deferred command's result — cmd.Err()/Val()/Result() — blocked on the batch's done channel, which the dispatching goroutine closes only after the hook chain returns: a deterministic self-deadlock on the async face (#3867 review). Any hook that inspects per-command results inside the chain — a common pattern in custom logging/metrics instrumentation — triggers it. The blocking face was unaffected. Fix, leaving the execution path untouched: - Complete batches at the innermost exec seam. The async dispatch now runs the user-hook chain via withProcessPipelineHook around the base processPipeline, closing the batches (and announcing expected arrivals, preserving the wave-merge ordering) the moment replies are populated — still inside the chain, so a hook reading results after next() (the span-status/metrics pattern) finds them ready. The lone-command fast path and Do get the same treatment through withProcessHook. The flusher's deferred close stays as a panic backstop (apBatch.close is now idempotent). The blocking face keeps its previous flow unchanged. - Guard await() against self-deadlock for reads BEFORE next(): the batch records its dispatcher's goroutine id while inside the hook chain; a reader on that goroutine gets the not-yet-executed view — exactly what a plain pipeline hook sees before next() — instead of blocking on a channel only its own goroutine can close. The goroutine id is parsed only on the about-to-block slow path, never on the fast path. - Make cmd.ready an atomic pointer: processAsync wired it after the command was already published to the flusher, racing a hook's read. The atomic store keeps that race benign — a hook that wins sees nil and takes the non-blocking not-yet-executed view — with nothing added under the hot stripe enqueue lock. - Arm the goroutine-id guard only when user hooks are installed (hooksMixin.hookCount): hook-free clients never stamp the batch, so a blocked reader pays one atomic load, not a runtime.Stack parse. Benchmarked against the unfixed base (benchstat; sequential n=8 rounds plus drift-controlled interleaved runs for the sensitive benches): DispatchPath ordered/unordered/solo, Submit, FutureFace and Throughput are all statistically unchanged (~, p>0.05); allocs identical (4/op dispatch). An earlier iteration that put setReady under the hot stripe enqueue lock benched at -9% Submit ops/sec and was rejected; the shipped shape adds nothing under that lock and keeps the guard to one atomic load unless hooks are installed. Note: results become readable the moment execution completes, so a hook that mutates a command's result after next() is racy on the async face; hooks should treat results as read-only there. Regression tests cover before/after-next hooks across the batch, solo and Do dispatch paths plus the blocking face, race-enabled; the deadlock reproduces on the unfixed tree. * test(autopipeline): consolidate per-topic test files into two Collapse the 29 autopipeline_* / osscluster_autopipeline_* test files into: autopipeline_test.go all blackbox tests (package redis_test), including the cluster autopipeline tests autopipeline_internal_test.go whitebox tests (package redis): shard-count decoupling, coalescing, review-fix regressions Pure reorganization: no test bodies changed, only regrouped (matches the consolidation already done on the benchmarks branch). Counts preserved (68 Test funcs, 3 Describe suites). The pipeline_*_test.go files are left as-is: they cover the dedicated pipeline pool and plain-pipeline behavior, not the autopipeline engine. * fix(autopipeline): deep-review fixes — hook short-circuit, cluster guard, tests, docs Engine: - A hook that returns WITHOUT calling next (allowed by the Hook contract; circuit-breaker shape) used to wake every waiter with a silent zero value: the chain's error was discarded on all dispatch paths. The innermost now records execution; when a hook short-circuits, its error (or a dedicated sentinel) is set on every command before the batch completes. Covers both faces, the lone-command fast path, Do and the shutdown drain. - The async self-deadlock guard is now always armed on cluster clients: node clients carry their own hooks (OnNewNode + AddHook, the redisotel pattern) that hookCount() cannot see. Node-level hooks run on node-worker goroutines, which the goroutine-id guard cannot protect pre-next(); that limitation is documented (results are safe to read after next()). - AutoFuture.Wait/WaitContext get the same dispatcher-goroutine guard as the command accessors. - Lone-command dispatch releases its permit and inFlight slot before waking the caller, so a closed-loop lone caller's next command takes the immediate-flush path instead of arming the silence-gap wait. - AddHook mutates the hook slice under hooksMu, and the dispatch snapshots the slice under RLock (background flushers iterate it concurrently). Tests: - TestAutoPipelineConcurrency asserted a locally-incremented counter and could not fail; it now checks every command and verifies DBSize. - The autopipeline reply-demux test ran on the blocking face sequentially, so every command took the lone-command fast path and the batch demux it claims to cover never executed; it now uses the async face with a flush window so all nine heterogeneous commands land in one pipeline batch. - The three cluster autopipeline tests dialed :7000-:7002 and always skipped; they now use the test harness's :16600-:16602. - Plain go tests use apTestAddr() (REDIS_PORT-aware, :6379 default): the suite variable redisAddr defaults to :6380 and is only corrected inside the ginkgo BeforeSuite, which plain go tests may run without. - Cold-start latency tests warm up dial+handshake before timing; goroutine leak checks poll with a deadline instead of fixed post-GC sleeps; the flush-on-timer test waits on a completion signal instead of sleeping. - New coverage: hook short-circuit (both faces, batch+solo), hooks added after the autopipeliner exists (client and delegate AddHook), Close flushing pending deferred futures, client-level Options.AutoPipelineOptions taking effect on both faces, BLPOP on the async face (readTimeout diversion), and AdaptiveDelay end-to-end traffic. API/docs: - UniversalOptions gains AutoPipelineOptions, threaded through Simple/ Failover/Cluster (the other option structs already had it). - ErrRingAutoPipelineUnsupported is exported (it was named in public godocs while unexported and unmatchable with errors.Is). - ClusterOptions.AutoPipelineOptions doc corrected: it configures BOTH faces, not only AsyncAutoPipeline. Client/ClusterClient Close docs now mention both cached autopipeliners. - README: stale WithConfig name fixed, the parallel-batches example uses a realistic MaxConcurrentBatches (8) with a note that more permits don't deepen batches; godoc throughput claims hedged as local loopback measurements; stale setReady/await comments refreshed; MaxBatchSize doc notes the blocking face's 300 preset. - Remove three stray benchmark screenshots (513 KB) accidentally committed. * fix(osscluster): route cluster pipelines through the dedicated pipeline pool Cluster pipelines acquired connections via node.Client.getConn — the node's main pool — so the dedicated pipeline pool was never used on ClusterClient, even though node clients inherit PipelineReadBufferSize / PipelineWriteBufferSize / PipelinePoolSize through clientOptions (review finding by cxljs on #3867). processPipelineNode and processTxPipelineNode now acquire through withPipelineConn: the node's dedicated pipeline pool when configured, with the exact previous behavior as the fallback (withPipelineConn delegates to withConn when no pipeline pool exists). An entered flag distinguishes acquisition failures — which keep the old MarkAsFailing / mapCmdsByNode / setCmdsErr handling — from execution errors, which processPipelineNodeConn already handles. TestClusterPipelineUsesDedicatedPool covers both the manual pipeline and the autopipeline batch path against a real cluster and asserts the node pipeline pools actually carried the traffic. * fix(autopipeline): DoRaw escape hatch + node-hook short-circuit surfacing Address review feedback (ofekshenawa, #3867): - AutoPipeliner embeds cmdable, so DoRaw and DoRawWriteTo rode the batching engine even though Do deliberately runs outside it. Raw commands carry Do's caveats, and DoRawWriteTo streams to caller-provided io.Writers — user I/O that must not execute inside a shared batch's reply loop. Both now override with Do's exact escape-hatch shape through a shared runOutsidePipeline helper: synchronous Process on the blocking face; returns-immediately with hook-safe batch completion (and hook short-circuit surfacing) on the deferred face. Do is refactored onto the same helper. - processPipelineNode and processTxPipelineNode discarded the return of the node-level hook chain: a node hook returning without calling next (the circuit-breaker shape the Hook contract allows) left every command error-free, so a cluster pipeline could report success for commands that were never sent. The chain now records execution; when it never reached the transport, the chain's error — or the errHookShortCircuit sentinel — is set on every command. Deliberate aborts are not remapped for retry: a retry would re-run the same hook. Tests cover DoRaw/DoRawWriteTo on both faces under result-reading hooks (including the no-args guard) and the node-level short-circuit against a real cluster, using an armable pipeline-only breaker installed via OnNewNode — disarmed during warmup because connection-init handshakes are themselves pipelines through the hook chain. * fix(autopipeline): retry tests honor REDIS_PORT, doc nits - pipeline_retry_test.go hard-coded ":6379" four times, silently skipping (or hitting the wrong server) when tests run with REDIS_PORT set; use apTestAddr() like the rest of the suite - drop a duplicated sentence in the ClusterOptions doc - the example benchmark's comment promises a goroutine-sized pool for the plain-client baseline but set 50 for 500 goroutines; size the pool to the goroutine count so the baseline is not pool-starved * fix(autopipeline): honor a hook's post-next verdict; complete batches after the chain A hook that called next and then returned an error had that verdict discarded (review finding by ofekshenawa on #3867): batches completed at the innermost exec seam, so waiters were already awake and the executed flag suppressed the chain's error. Batches now complete at the caller's deferred closes, after the whole hook chain has returned. The dispatcher-goroutine-id guard in await()/Wait() carries the self-deadlock protection instead: a hook on the dispatch goroutine reading a command before next() gets the not-yet-executed view, and after next() the populated results — without blocking on a channel only its own goroutine completes. This restores plain-pipeline hook semantics across the board: - a post-next injected error (exec succeeded, hook returned an error) reaches every command that has no error of its own; per-command errors recorded by the exec itself always win, and a hook's rewrap of an exec failure is dropped rather than clobbering precise per-command results; - the lone-command fast path and Do record the chain's final verdict exactly like Client.Process — short-circuits, rewrites and suppressions are all honored; - hooks may adjust results before any waiter wakes, removing the previous 'results are racy to mutate after next()' caveat; - the expected-arrivals announce still precedes the closes on both faces, preserving the wave-merge ordering. Also: the differential fuzz passed a raw 1000 (nanoseconds) to Expire, which truncates to EXPIRE 0 (a delete) — now a real 100s TTL (review finding by cursor). New TestAutoPipelineHookPostNextError covers both faces across the batch, solo and Do paths; the full hook battery (deadlock before/after next, short-circuit, hooks-added-late, DoRaw) stays green through the guard, as do the AP race suite and the cluster tests against a live cluster. * chore(examples): tidy the new example modules against this branch's root The master merge added four example modules (ft-aliaslist, himport, search-aggregate-collect, ts-querylabels) tidied against master's root go.mod. This branch's root additionally requires golang.org/x/sys (cache line padding in the pipeline pool), and the examples replace the root module, so on this branch they need 'go mod tidy' — which made the govulncheck CI job fail while loading packages (no vulnerability involved). Tidy adds the x/sys indirect requirement to each. * fix(autopipeline): skip the dispGid stamp on the blocking face's solo path The lone-command fast path stamped the dispatcher goroutine id (a runtime.Stack header parse) whenever hooks were installed, without the !ap.blocking guard the multi-command and shutdown dispatch paths already have. On the blocking face the stamp is never consulted — blocking commands never carry a batch, so their result accessors don't block — making it pure overhead on every single-command flush (review finding by cursor on #3867). Also document, at the verdict guard in dispatchCmds, why a post-next hook error is applied only to an all-clean batch: hooks conventionally return next's error, so after a partial failure the chain error is presumed to be that echo, and stamping it on the commands that did succeed would overwrite valid replies with their batchmates' failure. Behavior unchanged. * fix(autopipeline): never read user command results on the execution path A full re-review of the PR found that the async autopipeliner deadlocked in the DEFAULT configuration. pipelineProcessCmds re-read the first command's error after the himport re-issue pass via Err(); on the async face Err() awaits the command's own batch — which this very goroutine is executing and only closes after the hook chain returns. With no hooks installed the dispatcher-goroutine-id guard is unarmed, so a multi-command batch whose first reply was any error (a plain redis.Nil cache miss is enough) blocked the dispatch goroutine on itself forever: shard permit never released, every waiter hung, Close() hung on the batch WaitGroup. Reproduced with 50 async GETs on missing keys — the runtime reports 'all goroutines are asleep'. The same execution-path Err() reads existed at five himport sites (himportAfterBatch, himportRetryFailedSets, the cluster re-queue), which additionally run on per-node worker goroutines where the guard can structurally never match — any async batch containing an HIMPORT command wedged even when every command succeeded. All six now use rawErr(), matching the already-swept neighboring sites, with comments stating the rule. Two regression tests pin the redis.Nil-first batch and the himport-in-batch cases on the hook-free default. Also from the re-review: - TSNRangePivotRowSliceCmd.Val()/Result() were the only command accessors missing await(): async-face TSNRange/TSNRevRange returned zero values while racing the flusher's readReply. Added; a mechanical sweep of every accessor confirms no other misses. - Client.WithTimeout copied the client struct while AutoPipeline()/Close() can concurrently mutate the mutex-guarded autopipeliner fields — a data race under -race; the snapshot is now taken under the guard. - NewFailoverClient created the dedicated pipeline pool but never set pipelinePoolName, so poolForConn would route failover pipeline conns to the main pool (latent until failover clients gain streaming credentials; wrong-pool CloseConn/accounting once they do). - AutoPipelineOptions coerced non-positive MaxBatchSize and MaxConcurrentBatches to defaults BEFORE Validate ran, so the documented negative-value construction errors were unreachable. Validate now runs first; zero still selects the default. Test-quality fixes (found by auditing what the suite actually pins): - The pipeline/AP 'retries on network error' tests were vacuous: the broken conn they seeded into the idle pool embeds a zero net.TCPConn, which the pool health check discards before use — both tests passed with retries disabled. They now use a dialer whose conn fails exactly one write at the wire after the handshake, and assert the failure was consumed and exactly one redial occurred, so the first attempt genuinely dies and the re-drive is exercised. - The differential fuzz drove the blocking face sequentially, so every command took the lone-command fast path: zero multi-command batches, meaning batch demux had no randomized coverage. Added an async windowed phase (submit all ops, then read — the ordered default preserves submit order, so the sequential oracle still applies) with a counting hook that fails the test if no real batch formed. Validated by mutation: swapping two replies in dispatchCmds now fails the fuzz. - The cluster NumShards default test asserted >= 2, which fails on single-vCPU runners (the default is clamp(GOMAXPROCS,1,16)); it now asserts the computed default. - README: the AutoPipeline getters also return ErrClosed on a closed client, not only invalid-config errors. * fix(autopipeline): async-face parity for metric-only result reads; CI and doc fixes Publish/SPublish and XRead read their command's result only to record OTel metrics. On the async autopipeline face a result read awaits execution, so with no recorder installed these silently turned fire-and-forget publishes into blocking calls. internal/otel gains Enabled() and the three sites are gated on it; TestAsyncAutoPipelinePublishDoesNotAwait pins submission completing inside the flush delay. The Eval family's post-issue read stays: it normalizes NOSCRIPT to ErrNoScript, which Script.Run/RunRO depend on — now documented as synchronous on the async face. The failover client now registers its pools (main, pubsub, pipeline) for OTel gauge metrics, matching NewClient; it previously registered none while still unregistering on close. example/client-side-caching (new with the csc master merge) is tidied against this branch's root module — the govulncheck job failed loading it (go mod tidy needed; no vulnerability involved), the same pattern as the four examples fixed before the rebase. Documented from the cross-cutting review: a hook must never issue a command on the same autopipeliner and synchronously wait for it (the nested command needs the dispatch slot the hook chain holds; the engine recovers only by failing that flush after the 30s permit backstops, taking co-drained bystanders with it); Options.Limiter is consulted once per batch dispatch like a manual pipeline, not once per command; an autopipeliner created on a WithTimeout clone is not stopped by the parent's Close and must be closed explicitly. README caveats plus the armSelfDeadlockGuard and WithTimeout doc comments carry the warnings. * chore(autopipeline): consolidate pipeline test files; mark the feature experimental Consolidate the five loose pipeline test files into two, mirroring the engine PR's test consolidation: the dedicated-pool battery (handshake, maintnotifications, streaming credentials) into pipeline_pool_test.go and the execution-behavior tests (heterogeneous reply-shape demux, the retry/re-drive loop) into pipeline_exec_test.go. Pure move — ten test functions before and after, with section markers keeping provenance. Mark autopipelining EXPERIMENTAL using the repo's existing marker line: the API may still change as feedback lands. The marker covers every public surface — the eight client getters (Client and ClusterClient, blocking and async, no-arg and WithOptions), the AutoPipeliner and AutoPipelineOptions types, both Default*Options constructors, the UniversalClient interface block and Options.AutoPipelineOptions — plus banners in the README autopipelining section and the example. * chore(autopipeline): assert AutoPipeliner's UniversalClient conformance in universal.go The compile-time assertion existed in autopipeline.go; move it into universal.go's assertion block next to Client, ClusterClient and Ring, where the interface and its implementors are declared. * chore: gofmt — drop the double blank line left by the assertion move * chore(autopipeline): skip the fallback options allocation when an override is supplied getOrCreateAutoPipeliner called the fallback constructor (which allocates a default AutoPipelineOptions) before checking for an explicit override, discarding the result whenever one was provided (review finding by cursor on #3867). Check the override first. * fix(autopipeline): run SHUTDOWN and MONITOR outside pipeline batches Both are on the typed command surface and rode pipeline batches. A batched SHUTDOWN kills the server before it replies, so every batchmate failed with EOF and the whole batch was retried against a dead server (late review finding by cxljs on #3865). A batched MONITOR rebinds the shared connection into a monitor stream, desyncing every batchmate reply behind it. Both now execute directly on their own pooled connection, exactly like blocking commands — one gate in submit(), shared by both faces. The rest of the typed surface audits clean: Quit is a deprecated panic stub; the connection-state commands (Auth, Select, Hello, SwapDB, ClientSetName, ClientSetInfo) live on statefulCmdable and are not reachable through the AutoPipeliner; ClientTracking is the same footgun on a plain pooled client (client-side caching is the supported path); CLIENT PAUSE and the BGSAVE/FLUSH family are server-wide and batch-safe; blocking commands were already routed out via their read timeout. TestAutoPipelineShutdownRunsOutsidePipeline pins both routings with path-recording hooks against a guaranteed-closed port, so nothing real is shut down by the test. * fix(osscluster): honor a node-level hook's post-next verdict on clean sub-batches processPipelineNode surfaced a node-level hook's error only for short-circuits (hook never called next). A hook that called next and then returned its own error — the node-level twin of the post-next case fixed in dispatchCmds — had that verdict dropped: on an all-clean sub-batch every command kept reporting success (suppressed Copilot finding on #3867). Apply the same rule as AutoPipeliner.dispatchCmds: a post-next error on a fully clean sub-batch can only be the hook's own verdict, so it is applied to the commands; on a mixed batch the exec-recorded outcomes win (hooks conventionally echo next's error, and stamping the echo would overwrite successful replies). No remap for retry — retrying would re-run the same hook. cmdsFirstErr reads rawErr, so the check cannot await on the node goroutine. TestClusterNodeHookPostNextError pins the behavior against a live cluster with an armable post-next hook (disarmed during warmup so conn-init handshake pipelines pass). Also: example/autopipeline/go.mod pinned v9.21.0 while every other example pins v9.22.0-beta.1 — aligned and tidied. * fix: deflake the release-time push peek and the handoff-removal test TestReleaseConnRemovesConnectionAfterPartialPushRead failed randomly across matrix cells, and the root cause is in product code: peekAndProcessPushNotifications set a 10µs read deadline after MaybeHasData() had already confirmed kernel-buffered bytes. On a loaded machine the deadline routinely expired before the first peek syscall ran; the push processor treats a zero-consumed timeout as 'no pending data', so a connection with buffered push bytes went back into the pool undrained instead of being drained or removed. The first read never waits on the network here — the deadline only needs to survive scheduler pauses — so raise it to 1ms. Reproduced on the base branch (300 plain / 100 -race iterations both fail); the same loops are fully green with the fix. TestConnectionHook/ConnectionRemovalOnHandoffFailure slept a fixed 100ms and then asserted the connection had been removed — racing the handoff worker's three retries with backoff on slow runners. Poll for the removal with a bounded deadline instead. * feat(autopipeline): #4 tests and benchmarks (#3892) * feat(autopipeline): #4 benchmarks and parametrized command-suite runs Rebuilt on the post-#3867 integration: the branch's history carried pre-rebase copies of pipeliner content that conflicted with the squashed merge, so this is the branch's actual payload as one commit. - autopipeline_bench_test.go: eight honest benchmarks — plain-client and manual-pipeline baselines, the dispatch path (per-command ns/op and allocs/op), the future face, Submit, zero-copy reads, cluster throughput, and the plain-vs-blocking-vs-windowed headline comparison — updated to the shipped API. Every fixed-duration driver divides by the timed region (the previous fixed divisor inflated cluster windowed ~2x and the plain baseline ~27%), zero-copy read buffers live outside the timed region so B/op reflects the command path, and every bench skips without a server. The earlier tuning sweeps (batch/flush-delay/buffer sizes) were deleted: at low parallelism they reported the configured MaxFlushDelay timer for every swept value — noise dressed as signal. autopipeline_bench_README.md rewritten multiplier-first: absolute numbers are machine/load-dependent, so the documented signal is the within-run ratio vs the plain-client baseline (plus environment-independent allocs/op); the main README's throughput claims got the same treatment. - iterator.go: ScanIterator.Next read the command's page/cursor fields without awaiting the fetch, which only enqueues on the deferred autopipeline face — iteration spun re-issuing SCANs with a stale cursor forever, flooding the engine with an unbounded batch and hanging the caller. Next now awaits the fetch before reading (a no-op on every other path); TestAsyncAutoPipelineScanIterator pins it. - Parametrized-suite fixes: BFScanDump/CFScanDump compared command structs (each async command carries its own batch pointer, so equality is structurally false there) — compare values; the SlowLog argument-count spec and the BLMoveM timeout spec now keep admin/conn calls on the raw client like their siblings; the commands/bitmap suites build the subject closer before their first assertion so a failed BeforeEach cannot leave AfterEach with a nil or stale closer. - CI: the subjects job focus covers all nine parametrized suites (the old case-sensitive focus silently ran one), each subject run carries an explicit -timeout, and the run-tests action refuses an empty make target. - Command-suite parametrization (#3885): the commands/array/bitmap/ iterator/json/probabilistic/search/timeseries/vectorset suites run against a subject selected by GOREDIS_TEST_SUBJECT (client, ap-blocking, ap-async) via the newUniversalSubject harness, so the whole surface can be replayed through either autopipeliner face without touching specs. - maintnotifications: a failed manager Close no longer drops the not-yet-processed additional pool hooks — they are put back so a retried Close still sees them. - pipeline_buffer_test.go: skip, rather than fail, when the docker test server is absent. - CI: a test-autopipeline-subjects job replays the command suite through both faces via GOREDIS_TEST_SUBJECT (run-tests action gains an injection-safe make-target input; Makefile gains the target). Dropped from the old branch: stale pre-rebase engine-file copies (superseded by #3867), its pre-consolidation AP test files (integration's are newer and complete), and three unreferenced PNGs. * chore(autopipeline): review polish — close leaked test clients, align the docs The ScanIterator regression test closes its autopipeliner explicitly and the RESP3 search suite closes client2 in AfterEach (it leaked one client per spec) — both flagged by Copilot. From the cold-read review: the main README attributed the parallel-batch ~10x multiplier to the blocking face's ordered default (the bench README says the default lands lower — the two now tell one story); pipeline_buffer_test.go gets its honest pre-existing comment back; the Makefile comment says the subjects focus 'covers' every parametrized suite rather than 'lists explicitly'. * fix(bench): feasible zero-copy load, exact b.N accounting; CI polish The zero-copy benchmark at 64KiB payloads drove 500 goroutines through a PoolSize-10 client — on a 4-vCPU CI runner that blows the default 3s write deadline, and the newly surfaced errors (previously swallowed into the ops/sec count) correctly failed the bench on two matrix cells. Large-payload sizes now run 100 goroutines with generous deadlines, so the benchmark measures the command path rather than write-deadline queueing; the B/op result is unchanged (~100x less at 64KiB). From Copilot's follow-up review: benchWindowed and the zero-copy driver partition b.N exactly across workers (the first b.N%goroutines take one extra op) instead of floor-and-min-1, so ns/op and allocs/op are per command at every -benchtime — the old split over-ran for small b.N and dropped the remainder for large; and the run-tests action invokes make -- "$MAKE_TARGET" so a target value starting with a dash cannot inject make options. Also backtick MaxConcurrentBatches/Unordered in the README prose the previous commit added — the spellchecker exempts code spans. * fix(autopipeline): full review round for the master merge Required fixes from the integration review: ReadStringInto's fast path gates on len instead of cap, so the trailing CRLF can never be written past the slice the caller handed over (a sub-slice of a packed buffer exposed trailing capacity belonging to the next segment — the pattern the zero-copy example itself teaches); cluster mapCmdsByNode is strictly all-or-nothing again for user pipelines, with fan-out-policy commands rejected at autopipeline submit through a new engine preflight hook (so a merged batch cannot be poisoned and the lone-command path agrees with batched dispatch); both pipeline read loops carry push-frame-mid-batch regression tests (the cluster one mutation-validated); and Submit() marks commands ready so result accessors before Wait() await instead of racing the dispatcher. Codex findings: a bad-conn push-drain error on the dedicated pipeline pool removes the connection instead of pooling a desynced reader; PipelineReadBufferSize gets the RESP3 minimum clamp on both construction paths; a CSC WithTimeout clone's Close delegates to the owner's Client.Close so cached autopipeliners stop and the closed flag flips. Engine: AutoPipelineOptions.MaxBatchBytes caps flushes by approximate payload volume and dispatch splits oversized drains into byte-bounded chunks (batches still complete once, after all chunks) — many large values flush as several bounded writes instead of one multi-megabyte burst that can stall a constrained link past its write deadline; dispatch goroutines recover panics into per-command errors instead of crashing the process (a plain client unwinds a hook panic into the caller; the engine's internal goroutines had nobody to recover); the outside-pipeline guard covers the full connection-hostile command set for raw Submit/Do; internal backstop timeouts surface ErrAutoPipelineTimeout rather than a misleading context.DeadlineExceeded; ErrHookShortCircuit and ErrSubmitBlockingFace are exported; the content-shard index is MinInt-safe. Docs and tests per the review: pending-command String() formatting documented as unsynchronized; AfterEach drains subjects before flushing in the module suites; the CI subjects job claims results parity rather than batch coverage; new tests cover RESP2 batched demux, blocking-face ctx cancel, dispatch-panic containment, byte-capped batching, Submit misuse under -race, push drains on both read loops, policy rejection on both pipeline kinds, and the never-past-len reader guarantee; assorted minors (govulncheck minor-series pin, apTestAddr in buffer tests, slotCache happens-before note, twin-path drift warning, Close worst-case and Validate caveats, EXPERIMENTAL markers on new exports). * test: flush through the subject before closing it in the module suites The previous commit ordered closeSubject() before rawClient.FlushDB — but the closer closes rawClient, so the flush silently failed on a closed client and state leaked across specs (SEARCH_INDEX_EXISTS on the next protocol iteration, red on every CI test cell). Flush through the SUBJECT instead, in the original order: on the async autopipeline face the flush is ordered after any queued writes and closeSubject's drain guarantees it executes, which is what the reorder was trying to achieve; on the plain and blocking paths it is identical to the old rawClient flush. * fix(autopipeline): deferred face returns immediately for diverted commands; timeseries AfterEach Typed blocking and connection-hostile commands (BLPOP, SHUTDOWN, ...) were executed inline on the deferred face, stalling the submitter for the full block duration — Do already honored the deferred contract via its background-goroutine escape hatch. The diverted submit path now routes through the same runOutsidePipeline, keeping each face's call shape: the blocking face runs synchronously, the deferred face returns immediately with a ready batch and the result accessors block (review finding by cursor on #3942). A regression test pins submission returning immediately while BLPOP still blocks server-side for its full window. Also: the timeseries suite gets the same subject-flush AfterEach the previous commit applied to json/probabilistic/vectorset — flushing via rawClient before closing the subject let a queued async write land after the flush. * test: count solo dispatches in the MaxBatchBytes test The counting hook only watched the pipeline hook, but a drain cycle that catches exactly one command takes the engine's lone-command fast path (ProcessHook, no pipeline) — on a loaded runner one of the ten commands split off that way and the total came up short. Record solo dispatches as size-1 entries, filter to the test's own SET commands so a fresh-conn handshake pipeline cannot pollute the counts, and allow the uncapped control one straggler split. * test: widen the sentinel resolution budget for loaded runners The regression test demanded the full sentinel resolution chain inside a 500ms context — routinely missed on loaded CI runners over docker networking. Three seconds keeps the signal (the retry-storm exhaustion it pins burned the whole context regardless of size) without the flake. * fix: legacy DisableIndentity mapping in cluster node options; align checkout action version The cluster node clientOptions mapped the deprecated DisableIndentity field from the NEW DisableIdentity option, silently ignoring a user-supplied legacy value — Ring and the failover client already copy the legacy field. Also bump the run-tests composite action's internal checkout to v7, matching the workflow-level version. Both pre-existing upstream, folded in on the way to master. * fix(autopipeline): delegate cluster-wide commands, abort chunks after failed prefix, share the closed flag across clones Three review findings (codex on #3942) plus a test seam: AutoPipeliner now delegates ClusterClient's cluster-wide method overrides (DBSize, ScriptLoad, ScriptFlush, ScriptExists) to the underlying client — the embedded cmdable shadowed them and sent each to a single random node. Chunked dispatch (MaxBatchBytes) stops dispatching remaining chunks once an earlier chunk dies on a transport-class failure, failing them with the same error: the unchunked path fails or retries the batch as a unit, so in the ordered stream later commands must not overtake a failed prefix. Per-command redis errors are normal chunk outcomes and do not abort. The pool-closed flag (baseClient.apClosed) becomes a shared *atomic.Bool copied by WithTimeout/clone: closing a clone closes the SHARED pools, so the parent's AutoPipeline getters now see ErrClosed instead of building a fresh flusher against dead pools. ClusterClient passes nil — its per-node clients own their pools. The maintnotifications Manager gets a shutdownTimeout field (default 10s, behavior unchanged) so the Close put-back path — a failed shutdown returns the pool hook and reopens the manager for a retry — is pinned by a test. Tests: TestAPClusterWideCommandsDelegate, TestDispatchChunkedAbortsAfterFailedPrefix (unit-level: through the public API the byte cap wakes the flusher mid-submission, so one logical batch may split into several drains and a single-dispatch assertion races), TestWithTimeoutCloneCloseBlocksParentResurrection, TestCloseRetriesAdditionalHookAfterShutdownError. * fix: String() awaits deferred commands; share pooled-conn helpers between main and pipeline pools Public String methods on commands read their result fields before awaiting, so formatting an in-flight deferred-autopipeline command from another goroutine raced with reply processing. Every String method now calls await() first — the same accessor contract as Err/Val/Result. The dispatch-gid guard in await keeps a hook that formats its own batch on the dispatch goroutine safe (it sees the not-yet-executed view instead of deadlocking), and cmdString remains the non-waiting internal formatter on rawErr. The hook-peek regression tests now format commands from inside hooks as well (review finding by ofek and cursor on #3942). withPipelineConn duplicated getConn/releaseConn by hand and had drifted again: the dedicated pipeline pool missed the client-side-cache post-read probe on release and the connection-create-time metric. Conn preparation and release now go through pool-parameterized helpers shared with the main path (initPooledConn, releaseConnToPool), closing the drift class. This also fixes a main-path slot leak — a conn that cannot be re-acquired after its handshake is now Removed from the pool instead of abandoned. The reported Acquire hang on Go 1.23+ timer channels is refuted empirically and pinned by regression hammers: under synchronous timer channels Stop returns TRUE for an expired-but-undelivered fire (the delivery is aborted), returning false only once the value was actually received — and the sole receiver is Acquire's own select, so a taken token implies an unreceived fire and the drain branch is unreachable. Under GODEBUG=asynctimerchan=1 (a consumer main module on go < 1.23) the fired value sits buffered and the drain consumes it before pooled Reset-reuse. The hammers race Release against timeout expiry under both semantics. * fix(autopipeline): register cluster node executors on batches; harden pooled semaphore timers A node-level hook reading a command result before next() — exactly what redisotel's tracing does on cluster node clients via OnNewNode plus rediscmd.CmdsString's Err() peek — permanently wedged the deferred cluster face: the read blocked on a batch that completes only after the node call returns, and the single dispatch-gid guard cannot vouch for the cluster fan-out goroutines. Batches now carry a registry of node executor goroutines, entered and left around each node call's hook chain, and the accessor guards (await, Wait, WaitContext) treat registered goroutines like the dispatcher: result reads get the not-yet-executed view instead of self-deadlocking. Reproduced against a live cluster and pinned by TestClusterNodeHookResultPeekNoDeadlock. The pooled semaphore timers get a portable drain: it moves to a single putSemTimer at pool put-back using select-with-default, correct under both timer-channel semantics. Under go >= 1.23 synchronous channels Stop discards a fired-but-undelivered value, so the drain branch is unreachable — the default arm is a safety net so no semantics shift can ever turn it into a blocking receive. Under GODEBUG=asynctimerchan=1 (a consumer main module on go < 1.23) the buffered fire is consumed before Reset-reuse. Stale-fire hammers pin both semantics against spurious instant timeouts. * test: make the search RESP2/RESP3 equivalence suite hermetic The subtests drop only the index between runs, so earlier subtests' hashes were re-indexed by the next one (the aggregate subtest saw the search subtest's third document), and waitForIndexing polled FT.INFO a hundred times without sleeping — returning in microseconds on a partial count. On a slow runner the index was still churning between the RESP2 and RESP3 queries, which surfaced as a transient extra empty row in the RESP3 aggregate reply. Each subtest now starts from a flushed keyspace, and the wait requires the exact document count with ingestion settled (Indexing == 0), sleeping between tries. * test: wait for search ingestion in the home_json doctests The search examples write three documents and FT.SEARCH them back to back, but indexing is asynchronous — on a loaded runner the hash example's query raced ingestion and returned zero results. Both examples now wait for the exact document count with ingestion settled before querying. The wait lives in a separate non-example file and is called inside REMOVE_START/REMOVE_END markers, so the generated documentation snippets are unchanged. * test: assert module-suite teardown; scale semaphore hammers under -short The json, probabilistic, timeseries and vectorset suites ignored the results of their AfterEach FlushDB and closeSubject calls. On the async autopipeline subject the flush is enqueued and its failure only observable through the command's Err(), while AutoPipeliner.Close reports nil — so a failed teardown could silently leak keys into later specs. Both steps are asserted now, and awaiting the flush also pins it to execute at teardown time rather than inside the close drain. The semaphore timer hammers scale down under -short (200k -> 2k race iterations, 200 -> 20 stale-fire rounds), following the repo's existing short-mode convention for stress tests; full runs keep full coverage. * perf(autopipeline): restore the accessor guards' no-executor fast path The node-executor guard replaced the dispatch-gid short-circuit in the accessor guards (await, Wait, WaitContext) with a check that parsed the goroutine id and took the batch mutex unconditionally. Every blocking-face command and every pre-done future passes that guard once per wait, so concurrent waiters serialized on one batch's mutex with a stack parse each: the blocking face dropped from ~830k to ~138k ops/sec (6x) on the loopback throughput benchmark, while the windowed face was unaffected (its reads mostly land after batch completion). isExecutorGoroutine now answers the common case — dispatch gid unset and no registered node executors — from two atomic loads before any goroutine id parse or lock; the mutex is reachable only inside a cluster batch's node fan-out window. Blocking-face throughput measured back at the pre-regression band, and the node-hook deadlock regression test still passes: the guard still vouches for cluster node executors. * fix(autopipeline): honor shared pool closure at submit; stamp ready under the stripe lock; divert conn-state cluster commands Three review findings (codex on #3942): A cached AutoPipeliner kept accepting enqueues after a WithTimeout clone closed the SHARED pools — the shared closed flag guarded only the getters. It now threads into the pipeliner and every submit path (batched and diverted) checks it, so work is refused with ErrClosed at submit instead of failing dispatch by dispatch against dead pools. The deferred face published a command to its shard before setReady stamped the gating batch, so a fast flush could take a command whose readyBatch was still nil — the cluster node-executor registration would skip that batch, and a node hook reading the command mid-dispatch could block on a batch its own call chain completes. enqueue now stamps ready under the stripe lock, before the command is visible to any drain; the ordering is guaranteed by the lock both sides take. readonly, readwrite and asking join the outside-pipeline divert set: they mutate connection-scoped cluster state (replica reads, a pending redirect) that must not leak onto shared pipeline connections — and ReadOnly/ReadWrite are reachable from the typed surface, not just Do. Tests: TestWithTimeoutCloneCloseStopsCachedAutoPipeliner, TestEnqueueStampsReadyUnderStripeLock, TestConnStateClusterCommandsRunOutsidePipeline. Throughput bands re-measured unchanged after the enqueue-path change. * fix(autopipeline): bound Close on tracked diverted dispatches; divert raw blocking commands; delegate HIMPORT Six review findings (codex, cxljs and ofek on #3942), all reproduced in the code before fixing. Close and diverted commands (one coupled change). Commands that run outside a batch — blocking and connection-hostile ones — executed on untracked goroutines, so Close could return while such a command still held a pooled connection. They are tracked now. Tracking alone would trade that leak for a worse bug: an accepted command is deliberately never cancelled by Close (the flush contract the batched path documents), so a zero-timeout blocking command, or a stalled read with ReadTimeout disabled, would hang Close forever. The wait is therefore bounded by the same backstop the permit wait uses, and on expiry Close returns an error naming what is still in flight rather than blocking its caller. Raw blocking commands. Typed helpers mark themselves with a per-command read timeout, which submit already diverts on; a raw Cmder built by hand carries no such marker, so BLPOP and friends could be queued onto a shared pipeline connection and hold it for the block duration. They are recognized by name now, with XREAD/XREADGROUP classified by their BLOCK argument so the common non-blocking form stays batched. Cluster-wide overrides. The HIMPORT admin trio joins the delegation set: like DBSize and the SCRIPT admin commands, ClusterClient fans them out to every master and updates the shared fieldset registry, so running them on a single routed node lets a later HImportSet for a key on another master fail. A reflection test now walks ClusterClient's method set and fails when an override that shadows the generic cmdable implementation has no delegate — this class has been found by review twice. Maintenance notifications. A delayed idle handoff for a connection from a dedicated pipeline pool queued on the primary pool hook, whose request carries the primary pool; a failed handoff then closed the connection without freeing its slot in the pipeline pool. Handoffs now route to the hook that owns the connection's pool, matched by the pool name already stamped on every connection. Close's bound is its own named constant rather than a borrowed one, with the reasoning recorded next to it: the bound is only reachable when a dispatch cannot end by itself, so a tighter value would not speed up healthy shutdowns — it would make Close report failure while a legitimate slow drain (a large final batch, a relaxed maintenance window during failover) is still finishing. Docs. The slot-to-shard mapping is described as what it is — a batch-depth heuristic. With non-contiguous slot ownership a shard's batch can still span nodes; correctness is unaffected because the node router resolves each command's own slot, and a key always lands on one shard so its relative order holds. Also lands the truth battery: thirteen end-to-end tests that check order, reply demultiplexing, exactly-once execution, at-most-once under retries, chunked-dispatch integrity, pipeline-pool isolation, cancellation honesty and the Close drain against an INDEPENDENT ground truth — a client-side model value, a server-side structure recording the real execution order, or a differential run of the same randomized workload through a plain client. * fix(autopipeline): close the divert registration race, honor shared close everywhere, order limiter reporting Five review findings on the previous commit (cursor Bugbot and codex on #3942), each reproduced in the code first. Shared close. Do, DoRaw and DoRawWriteTo guarded on this pipeliner's own closed flag rather than the combined check, so once a WithTimeout clone had closed the shared pool set they still accepted commands and dispatched against pools that were already gone. IsClosed had the same gap while its documentation promised that the owning client's close counts. Every guard that gates accepting new work now goes through the combined check, with the invariant recorded beside it: only Close's own compare-and-swap reads the instance flag directly, because it claims the shutdown rather than testing for it. Divert registration race. A diverted command could pass its closed check, Close could then observe an empty wait group and return, and only afterwards would the goroutine register — leaving an accepted command holding a pooled connection past Close, and racing Add against Wait. Registration now happens under a mutex with a closed re-check, and Close passes through that mutex once after marking itself closed, so a registration either completes before the wait or is rejected. The gate precedes publishing the command's batch: publishing first and then rejecting would leave the command gated on a batch nobody closes. Blocking detection. The BLOCK token is matched the way the encoder renders it — a raw Cmder may carry RESP keywords as []byte or *string, and a string-only type assertion let those XREAD forms be batched onto a shared connection. Cluster preflight. Diversion is decided before the preflight runs. The preflight keeps fan-out-policy commands out of shared pipelines, but a diverted command never rides one: it goes through Client/ClusterClient Process, where the cluster-wide aggregation works. Running the preflight first rejected typed WAIT/WAITAOF on a cluster whenever command policies were enabled. Limiter ordering. The dedicated pipeline-pool path registered the limiter report and the connection release as two defers, which run LIFO — the connection went back to the pool before the result was reported, so a circuit breaker could admit another pipelined operation before observing the failure. The main pool reports first; both paths now share one deferred exit that preserves that order and still reports on the acquire and handshake failure paths. * fix(autopipeline): bound the whole close drain; gate raw Do and telemetry on the divert contract Four review findings from codex plus two from Copilot, each reproduced in the code first. Raw Do on the blocking face. Do, DoRaw and DoRawWriteTo carried their own inline blocking-face branch that called Process directly, bypassing runOutsidePipeline and therefore the divert registration gate added in the previous commit. A raw blocking Do was untracked, so Close could observe no diverted work and tear down shared resources while it still held a pooled connection. Both faces now go through the one path that registers. Close's bound covered only the last wait. A dispatch that nothing can end sits just as easily in a flusher the wait group is waiting for, or in the shutdown sweep's own dispatch, as in the final wait — so Close hung before ever reaching the bound it documents. The whole drain tail (flusher exit, final shard sweep, batch and diverted waits) now runs under a single bound, and on expiry the error names the precise stage still outstanding, because a wedged flusher and a wedged blocking command call for different operator responses. Telemetry no longer changes the async call shape. The metric emissions after Publish, SPublish and XReadGroup read the command's outcome immediately, which AWAITS on the deferred face: with a recorder installed, an autopipelined publish blocked until Redis replied, and an XReadGroup with the default Block: 0 could block indefinitely in what should be a submit. They now consult a non-blocking readiness check and skip the metric for a submission that has not executed; emitting these from the execution path is a follow-up in the OTel wiring rather than something a command wrapper can do. Two Copilot findings: the ConnPool.Name doc block added last round swallowed Size()'s Godoc, leaving an exported method undocumented — restored; and the pipeline-buffer tests left fixed keys behind, one of which (test_key) is also used by another test file in the package, which is exactly the order-dependence that produced the search-suite flake earlier in this PR. * fix(autopipeline): stop Config leaking internal sharding; keep Eval non-blocking on the deferred face Two review findings from codex plus one from Copilot. Config leaked an internal invariant. The effective config returned to callers carried contentSharded, the bit cluster wiring sets to record that shards are slot-routed and therefore cannot reorder same-key commands. Copying that config into a standalone deferred autopipeliner silenced the check that requires Unordered for NumShards>1 — and round-robin shards genuinely do flush concurrently and break submit order. Internal-only fields are now cleared before the copy is handed out. Eval was synchronous on the deferred face. Normalizing NOSCRIPT to ErrNoScript read the command's error immediately after submitting, which awaits execution, so the whole Eval family lost the deferred call shape — a caller submitting a window of scripts, or running one slow script, blocked in the submit. The normalization now runs only when the result is already readable, and Script.Run/RunRO accept the server's raw NOSCRIPT error so the EVAL fallback keeps working when it was skipped. The synchronous and blocking faces are unchanged; on the deferred face the raw error is byte-identical to ErrNoScript, so errors.Is and MatchError assertions still hold. Copilot: the pipeline-buffer key cleanup added in the previous commit ran before the reachability check, issuing a DEL against a server that might not be there and turning a unit-only run into a failure instead of a skip. The four call sites skip first now, and the helper documents that ordering. * fix(autopipeline): keep a no-retry command's policy off its batchmates; complete the blocking set A command that forbids retries — today the zero-copy reads, whose reply decodes into a caller-owned buffer that a retry could not un-write — disables retries for the entire slice it is dispatched in. In a shared autopipeline batch that silently stripped retries from unrelated callers' ordinary commands, which would have been retried had they been flushed on their own. Mixed batches are now partitioned by retry policy and each group dispatched separately; uniform batches are returned untouched, so only the mixed case pays a second dispatch. Splitting rather than diverting the no-retry commands keeps zero-copy reads pipelined — measured after the change at 507k ops/sec and 497 B/op for a 4 KiB value, still ahead of the regular path. The raw-blocking allowlist was incomplete: blmovem, migrate, and the argument-driven ts.read BLOCK form could be queued onto a shared pipeline connection when submitted as raw Cmders, holding it for the block duration. The set is now derived from the typed helpers that call setReadTimeout rather than enumerated by hand, with that derivation recorded beside it and a test that pins every name and both arg-driven forms. Also rewords PipelineReadBufferSize's tuning guidance, which described the write side — sizing so a batch flushes without overflowing mid-write — and cross-references the option that guidance actually belongs to. codex's remaining finding, that ClusterClient.PoolStats does not fold in the per-node pipeline pools, is a monitoring gap rather than a correctness one and is left as a follow-up. * fix(otel): read the global recorder under its lock when registering pools RegisterPools and UnregisterPools read globalRecorder directly while SetGlobalRecorder writes it under recorderMu, so installing telemetry concurrently with creating or closing a client is a data race. A regression test reproduces it — two race reports under -race before the change, clean after — and both helpers now go through getRecorder. Also fixes the pipeline-buffer key cleanup added earlier in this PR: its post-test sweep ran in a t.Cleanup, which executes after the test function's defers, and every caller defers client.Close(). The DEL therefore landed on a closed client and silently did nothing, leaving exactly the keys the helper exists to remove. The sweep now uses its own short-lived client and reports a failure rather than swallowing it (both found by Copilot on #3942). * fix(autopipeline): isolate retry policy without reordering; honor successful hook short-circuits The no-retry isolation added in the previous commit split a mixed batch into two policy groups and dispatched the retryable one first, which REORDERED the stream: a zero-copy read submitted before a write to the same key executed after it, so the read observed the new value on a face that promises submit order. The batch is now sliced into contiguous runs and dispatched in order — each dispatched pipeline is still policy-uniform, so a no-retry command cannot strip retries from batchmates, and no command overtakes another. Uniform batches are still one dispatch with no allocation. Both new tests fail if the grouping is reintroduced, including the end-to-end one that reads the wrong value. A hook that returns nil WITHOUT calling next has short-circuited successfully: it served the batch itself and set the command values, which is exactly what a caching or mocking hook does and what plain Pipeline and Client hooks are permitted to do. The engine rewrote that success into an error, so such a hook worked on a pipeline but failed every autopipelined batch. All five sites now propagate only the hook's own error, and the sentinel this PR added for the synthesized case is retired. Cluster commands whose routing is not slot-derived — ReqSpecial, such as FT.CURSOR READ, which is sticky to the node holding the cursor — are now diverted instead of coalesced. A solo flush reached ClusterClient.process and its special routing, but inside a batch mapCmdsByNode routes by slot and can send the cursor read to the wrong shard, which only shows up under concurrent traffic. Diverting keeps them working on their own connection rather than rejecting them (all three found by codex on #3942). * fix(autopipeline): share the prefix-abort rule; report submit rejections from Process Breaking a batch into ordered pieces happens in two places now — the MaxBatchBytes chunker and the retry-policy runs — and each independently got the same rule wrong: once a piece dies on a transport-class failure, the remaining pieces must not be dispatched, because the unchunked path fails or retries the batch as a unit and later commands must never overtake a failed prefix. The runs loop dispatched them anyway. Both callers now go through one dispatchSequential helper that owns the rule, so a third variant cannot drift out of line; per-command redis errors remain normal outcomes that do not stop the sequence. The deferred face's Process discarded submit-time rejections. A closed pipeliner or a cluster preflight refusal sets the error on the command and queues nothing, so returning nil told the caller the command was accepted when it would never run — and code reaching the engine through UniversalClient.Process sees only that return value. Those rejections are returned now, identified by the shared completed batch rather than by probing for an error, so execution failures still stay off this path: the call must not wait. The type documentation still said String()/%v on a pending deferred command does not wait. It has waited since the accessor round, and a test pins exactly that, so the doc was steering users toward a race that no longer exists. It now describes the real contract, including the dispatch-goroutine exception, and points at Name()/Args() for logging a submission without waiting (all three found by codex on #3942). |
||
|
Nedyalko Dyakov
|
82b0213da3 |
chore(release): prepare 9.22.0-beta.1 (#3940)
- add 9.22.0-beta.1 release notes - note Redis 8.10 support in the README - bump version to 9.22.0-beta.1 in version.go and all extra/example go.mod files |
||
|
Nedyalko Dyakov
|
90fd08805d |
chore(ci): point 8.10 testing at custom client-libs-test image (#3938)
Switch the 8.10.x version mapping in CI, the doctests service image, and the local Makefile/docker-compose defaults from 8.10-rc2 to custom-30445126297-debian. |
||
|
Nedyalko Dyakov
|
93f961af67 |
feat(timeseries): support multiple aggregators per key in TS.NRANGE (#3937)
RedisTimeSeries (RedisTimeSeries/RedisTimeSeries#2079) lets TS.NRANGE and TS.NREVRANGE apply several aggregators to one key: each key gets its own AGGREGATION spec token, and a spec may list multiple comma-separated aggregators (e.g. AGGREGATION avg,max sum <bucketDuration>). The Go API could only express one aggregator per key. Change TSNRangeOptions.Aggregators and TSNRevRangeOptions.Aggregators from []Aggregator to [][]Aggregator: one spec per key, comma-joined into a single wire token. Validation now also rejects empty specs, and the pivot row docs describe the flattened (key, aggregator) value layout. Breaking for the unreleased TS.NRANGE API only; matches redis-py (redis/redis-py#4225). |
||
|
Nedyalko Dyakovandofekshenawa
|
49e004104c |
feat(himport): HIMPORT command with lazy per-connection prepare (#3919)
* feat(himport): HIMPORT command family with lazy per-connection prepare replay Fieldsets are server-side session state scoped to one physical connection, so pooled HIMPORT SET needs the fieldset prepared on whichever connection executes it. Instead of replaying PREPARE in the handshake, the client keeps a versioned fieldset registry and a per-connection prepared flag, and injects the PREPARE into the same round trip right before a SET that needs it - at most once per connection session. A 'no such fieldset' reply for a registered fieldset invalidates the flag and grants one retry that re-prepares. * fix(himport): drain push notifications between injected PREPARE and command replies proto.Reader.ReadReply consumes a RESP3 push frame as a normal reply, so a notification arriving between the injected PREPARE's reply and the command's reply was read as the command's reply and desynced the connection. Drain pending notifications again after reading the PREPARE replies, mirroring what the pipeline read path does before every reply. The regression test drives a RESP3 mock server that emits a push frame between the two replies; without the drain the SET fails with "can't parse reply=\">2\"". * fix(himport): wire registry into failover and tx clients, invalidate stale pipeline flags Two review fixes for the lazy prepare-replay machinery: - NewFailoverClient built its baseClient without the HIMPORT registry, so HImportPrepare appeared to succeed but never registered and pooled HImportSet replay never engaged. Tx (client.Watch) now shares the parent client's registry the same way Client.Conn() does, since its sticky pool borrows connections from the parent pool. - A pipelined HImportSet failing with "no such fieldset" left the connection's prepared flag stale, so even a caller-initiated retry of the pipeline failed forever on that connection. The failed exec now invalidates the flag and the retried pipeline replays the PREPARE. Pipelines are deliberately not auto-retried: re-running the batch would repeat side effects of commands that already succeeded. * test(himport): skip integration specs when the server lacks HIMPORT The 8.10.x CI image is a pre-release build without the HIMPORT patch series, so the version gate alone let the specs run into "ERR unknown command 'himport'". Probe with HIMPORT DISCARDALL and skip when the command family is absent. * docs(himport): add example app Walks through prepare/set basics, pipelined bulk ingestion timed against plain HSET, concurrent pooled writers relying on lazy replay, the template encoding surfaced by OBJECT ENCODING and INFO memory, and fieldset discard semantics. Exits gracefully when the server does not support HIMPORT. * feat(himport): propagate discards lazily to all pooled connections HIMPORT DISCARD/DISCARDALL execute on one borrowed connection, so other pooled sessions kept their fieldset copies: a subsequent HImportSet was nondeterministic (it succeeded against a session that still held the fieldset and failed elsewhere), and the leftover copies held server memory until the connections closed. Discards now propagate the same way prepares do. A discarded name stays in the client registry as a tombstone and DISCARDALL bumps a registry epoch; before any HIMPORT command on a connection, the client injects HIMPORT DISCARD for tombstoned fieldsets that session still holds, or a single HIMPORT DISCARDALL when the session predates the current epoch. Re-registering a name clears its tombstone - the replayed PREPARE already replaces the fieldset server-side. Post-discard behavior is now deterministic: HImportSet fails with "no such fieldset" on every pooled connection, exactly as on a single connection. Non-HIMPORT traffic never pays for the cleanup; sessions that stop running HIMPORT commands are cleaned up by connection close, as before. * fix(himport): guard root-cause swap, discard stale fieldset - the single-command path replaced the SET reply with an injected PREPARE failure even when the SET succeeded or failed with an unrelated error; gate it on "no such fieldset" like the batch path - an injected re-prepare that fails leaves the session holding the older fieldset version, and the pipelined SET behind it then writes the old field names silently; discard the held version first so the SET answers "no such fieldset" and the root cause is surfaced - move the test-only registry accessors epoch/empty into the test file: the linter runs with tests excluded and flags them as unused * fix(himport): align recovery and discard semantics with HLD v21 - Pipelines: HIMPORT SETs of a registered fieldset that fail with "no such fieldset" are re-prepared and re-issued once on the same connection - and only those SETs; HIMPORT SET is a full replace, so the re-execution is idempotent and no other command of the batch runs again. The error therefore never surfaces for managed fieldsets (NF.4). Transactions keep surfacing it - an executed MULTI/EXEC cannot be partially re-run - and the invalidated flags make the caller's transaction retry succeed. - HImportDiscard/HImportDiscardAll report the registry lifecycle per the HLD method shapes: 1 when the fieldset was registered on this client and removed, and the count of registered fieldsets removed. The executing session's reply passes through only for names never registered through the managed API. - Document that HIMPORT commands sent through generic interfaces (Do, raw Cmd values) bypass the registry and carry none of the managed guarantees; the root README generalizes the caveat to every command that alters connection session state (SELECT, CLIENT SETNAME, RESET, CLIENT TRACKING, ...) and points session-scoped work at Conn(). The working tree also carries the remote "guard root-cause swap, discard stale fieldset" changes verbatim, so the merge below resolves cleanly. * feat(himport): cluster and ring support via shared registry and fan-out Every ClusterClient node client and Ring shard client now shares one HIMPORT fieldset registry, adopted at client creation - replicas included, since roles change with the topology: a promoted replica's connections carry no prepared flags, so their first HIMPORT SET replays the PREPARE lazily, and MOVED/ASK redirects re-prepare on the target node through the existing per-connection machinery with no cluster-specific code. On top of the shared registry, user-issued PREPARE, DISCARD, and DISCARDALL fan out eagerly to all masters (R.4) or all ring shards: one connection per node is prepared or cleaned up front, server-side validation surfaces immediately, and leftover session state stays bounded. Fan-out copies carry a pre-assigned registry version/epoch so the registry is touched exactly once per operation; a deterministic server rejection (e.g. duplicate field name) withdraws the registration, while transport failures keep it and lazy replay covers the connections the fan-out missed (all-succeeded reporting). * chore(ci): add fieldset to the spellcheck wordlist * fix(himport): wire cluster pipelines into the managed layer, close review findings Review findings from three independent passes (wire protocol, concurrency/state, HLD compliance): - ClusterClient pipelines and tx pipelines wrote commands directly to node connections, bypassing HIMPORT entirely: no PREPARE replay (the fan-out prepares one connection per master, so pooled pipelines hit "no such fieldset" nondeterministically), no registry updates for pipelined PREPARE/DISCARD (a pipelined DISCARD was later resurrected by lazy replay), no re-prepare after MOVED/ASK re-queues, and no stale-version defense. processPipelineNodeConn and processTxPipelineNodeConn now inject and consume the same HIMPORT bookkeeping commands as the standalone paths; failed SETs of registered fieldsets re-queue through the pipeline's attempt budget, which re-prepares lazily on the next attempt. - A transport failure inside the standalone pipeline's HIMPORT retry round trip signalled canRetry after the first round trip had been fully consumed, so the outer loop re-executed the whole batch and double-applied non-idempotent commands whose results were already delivered. The retry now reports the error without requesting a batch retry; the connection is still discarded. - The handshake Conn handed to OnConnect (and the re-auth Conn) had a private HIMPORT registry whose versions restart at 1: an OnConnect prepare could mark a pooled connection at a version number colliding with the client registry's, silently skipping the replay of a different fieldset definition. Both now share the client's registry. - A fan-out PREPARE rejected by one master withdrew the registration without a tombstone, leaving unaddressable fieldsets alive on the masters where the fan-out had already succeeded. The withdrawal now tombstones, so those sessions are cleaned on their next HIMPORT use. - NewRing stored the ring-wide registry in the caller-owned RingOptions, so two rings built from one options struct shared and clobbered each other's registry. NewRing now copies the options. - Rewrite the stale pre-cluster godoc on HImportPrepare that claimed ClusterClient/Ring do not fan out, and document the best-effort fan-out failure semantics on the cluster/ring methods. - newConn now takes the HIMPORT registry as an explicit parameter instead of allocating a throwaway one that every call site overwrote. * fix(himport): skip set re-queue when the batch retry already carries it * fix(himport): scope re-issue failures, unmark while holding the conn Three review findings: - A transport error inside the pipeline set re-issue round trip was returned to generalProcessPipeline, which stamped it onto every command with a nil error - including commands whose results were already delivered by the fully-consumed first round trip. The error now stays on the retried SETs; the batch neither retries nor fails wholesale, and the connection, which may hold unread replies, is marked for removal on release (MarkCloseOnPut). Regression-tested by dropping the connection on the replayed PREPARE and asserting the delivered plain SET keeps its OK result. - The single-command recovery unmarked the prepared flag from process(), after withConn had already released the connection, mutating flags on a connection another goroutine may own. The invalidation now happens inside _process while the connection is still held; himportRecover became himportShouldRetrySet, a pure retry decision. - A connection's discard-all epoch could in principle be overwritten backwards by a mark carrying an older registry snapshot. The epoch is now forward-only on marks; a stale snapshot can no longer regress a session wiped at a newer epoch. The working tree also mirrors the master-merge changes to _process (assertUnstableCommand removal) so the merge below resolves cleanly. * fix(himport): survive mass session loss, honor close-on-put in sticky pools Deep-review findings from three independent passes (fresh-eyes diff, behavioral simulation against a session-faithful mock, regression safety), plus one final-review nit: - Mass session loss (failover, cross-region switch, reset storms) could beat the single-command retry-once budget: the retry landed on a different connection whose session was wiped by the same event but whose prepared flag still looked current, so the PREPARE was not injected and "no such fieldset" surfaced for a managed fieldset (deterministic repro: 2-3 surfaced errors per 1200 SETs in a reset storm). A "no such fieldset" failure now bumps the fieldset's registry version (fields kept, guarded against concurrent re-registrations, once per batch), staling every connection's mark so the retry re-prepares wherever it lands. - StickyConnPool.Put ignored MarkCloseOnPut, so a Conn/Tx whose HIMPORT re-issue round trip failed on a transport error got the same connection back with the re-issued replies still unread - the next command read a leftover reply as its own (live repro: HImportDiscard parsing "OK" as an integer; a status-typed command would accept the wrong reply silently). Sticky Put now records the mark as a bad-conn error, Get refuses the connection, and the parent pool removes it on unwind. - The standalone pipeline and tx-pipeline paths stamped no per-command errors when the injected-reply read died on a transport error: the outer retry loop stamps only on its exit branch, not when the attempt budget runs out mid-loop, so Exec surfaced the error while every command still reported Err() == nil. Both paths now stamp the batch like a write failure, matching the cluster pipeline; the regression test drives it through the mock (connection dropped on the injected PREPARE, MaxRetries -1). - The "no such fieldset" check after every single-command reply ran before any type check; its errors.As chain allocates, costing every non-HIMPORT command 2 heap allocations (+32 B, ~21 ns, +12.5% allocs/op for a simple SET). The HImportSet type assertion now runs first: ~1 ns and zero allocations for everything else. Doc hardening: cross-node PREPARE+SET pipelining limitation on ClusterClient, tombstone growth bound, the recovery attempt granted under MaxRetries -1, and the ASKING re-issue comment corrected. * docs(himport): mark the HIMPORT API as experimental The server command family ships behind the hinted-hash-templates feature of Redis 8.10 pre-releases and its surface may still move; the typed client API (methods, HImport*Cmd types, constructors) follows it. Same wording and placement as the vectorset commands. --------- Co-authored-by: ofekshenawa <104765379+ofekshenawa@users.noreply.github.com> |
||
|
Nedyalko Dyakov
|
3dd9675caf |
fix(proto): peek push notification name without demanding 36 bytes (#3936)
PeekPushNotificationName started with bufio.Peek(36), which blocks until 36 bytes are buffered. A complete subscribe confirmation for a channel name of six or fewer characters is shorter than that, and on an otherwise idle connection the peek waited forever, hanging PubSub.Receive (regression from #3842). Parse from the bytes already buffered instead, and when the frame prefix is valid but incomplete block for exactly one more byte before re-parsing. An incomplete prefix means the server is still committed to sending the rest of the frame, so that byte always arrives. Fixes #3935 |
||
|
|
d11bf62381 |
chore(tests): run the Redis Enterprise suite against an external cluster (#3905)
* test: run the Redis Enterprise suite against an external cluster Read the target database connection (host, port, username, password, TLS) from REDIS_ENDPOINTS_CONFIG_PATH when RE_CLUSTER=true, and apply it to the clients the suite builds (redisOptions, the per-suite setupRedisClient closures, the RediSearch constructions, and the universal-client spec). RE_DB_NAME selects a named database; when the env var is unset behaviour is unchanged (localhost:6379, no auth). Label specs that cannot meaningfully run against a remote, authenticated, managed Redis Enterprise deployment NonRedisEnterprise, each with an explanatory comment (client-internal/local-only, OSS-vs-Enterprise behaviour, or impractical over network latency). * test: skip the local-TLS tests when running against Redis Enterprise TestTLSStandalone and TestTLSRedissURL are plain Go tests, so the NonRedisEnterprise Ginkgo label filter cannot exclude them, and they t.Fatalf when the docker-compose TLS server (localhost:6666) is absent. Skip them explicitly on RE_CLUSTER=true runs; local runs keep failing loudly when the TLS server is expected but missing. * test: harden Redis Enterprise endpoint resolution Fall through to the endpoint URLs when a raw endpoint carries no DNS name instead of producing a ":port" address that dials localhost, and share the relaxed timeout values between the Options and UniversalOptions overlays so the two cannot drift. * test: run the Auth spec against password-protected servers A failed AUTH is reported as "ERR AUTH ..." by a passwordless OSS server but as "WRONGPASS ..." by a password-protected one (e.g. Redis Enterprise), so accept either and drop the NonRedisEnterprise label. The two failing AUTH commands now go through separate pipelines: on a password-protected Redis 8.x a second failed AUTH in the same pipeline is throttled by the server's brute-force protection and never answered, which stalls the read until the client times out. * fix(test): bracket IPv6 endpoints with net.JoinHostPort url.Hostname() strips the brackets from an IPv6 literal, so joining host and port with "%s:%d" produced an undialable "::1:6379". JoinHostPort restores the bracketed form; add a regression test pinning the bracketed Addr. --------- Co-authored-by: kiryazovi-redis <ivaylo.kiryazov@redis.com> Co-authored-by: ofekshenawa <104765379+ofekshenawa@users.noreply.github.com> |
||
|
Nedyalko Dyakovandofekshenawa
|
8ba2e2a20f |
feat(search): add COLLECT reducer for FT.AGGREGATE (#3886)
* feat(search): add COLLECT reducer builder + response decode for FT.AGGREGATE Add FTAggregateCollect options, NewCollectReducer constructor, and AggregateBuilder.Collect that render the REDUCE COLLECT clause (FIELDS/ DISTINCT/SORTBY/LIMIT) with automatic @-prefixing and arg-count. Add AggregateRow.Collect to decode the sparse RESP2/RESP3 entry array into a uniform CollectColumn. Integration-tested against Redis 8.8. Additive: no existing type or serializer changed. Requires Redis 8.8+ with search-enable-unstable-features. * fix(search): validate COLLECT args, harden integration test - reject empty field names and ASC+DESC combinations client-side, matching the FT.AGGREGATE pipeline SORTBY error wording - gate the integration test on the live server version via INFO; the float REDIS_VERSION check parses 8.10 as 8.1 and rots - restore search-enable-unstable-features, drop the index, and Del only the test's own keys instead of FlushDB; remove broken itoa * docs(example): add FT.AGGREGATE COLLECT reducer example Runnable example for the COLLECT reducer, mirroring the search-aggregate-steps layout: - top-2 products per category via NewCollectReducer (explicit FIELDS with SORTBY rating DESC and LIMIT), next to a plain COUNT reducer - whole documents per brand via AggregateBuilder.Collect with LOAD * and FIELDS * - replies decoded with AggregateRow.Collect (RESP2/RESP3 agnostic) Verified live against redislabs/client-libs-test:8.8.0. Notes in code and README: on current unstable builds, SORTBY combined with FIELDS * drops the sort field from the entries and does not sort - use an explicit FIELDS list when sorting. * chore(git): remove binary --------- Co-authored-by: ofekshenawa <104765379+ofekshenawa@users.noreply.github.com> |
||
|
Nedyalko Dyakovandofekshenawa
|
ce8d3b053f |
feat(timeseries): add TS.QUERYLABELS support (#3926)
* feat(timeseries): add TS.QUERYLABELS support (Redis 8.10)
- TSQueryLabels lists label names, TSQueryLabelValues the values of
one label, across series matching optional TS.QUERYINDEX-style
filters
- the FILTER token is emitted only when expressions are given (the
server rejects a bare FILTER); expressions pass through verbatim
and the VALUES label is matched byte-exactly
- replies are unordered, server-deduplicated string sets under both
RESP2 and RESP3; empty replies are success, not errors
- wire-format unit tests plus a RESP2/RESP3 integration test that
gates on the live server supporting the command
- runnable example under example/ts-querylabels
* fix(tests): use skip for tests
* test(timeseries): version-gate TS.QUERYLABELS
Replace the standalone probe test (command probing plus the
REDIS_QUERYLABELS_TEST_ADDR env var) with Ginkgo specs in the main
suite, gated on SkipBeforeRedisVersion("8.10"). The plain go test also
called the Ginkgo-based SkipBeforeRedisVersion helper, which panics
outside the suite when it actually skips; moving the specs into the
suite fixes that. Covers both command forms under RESP2 and RESP3.
* chore(deps): bump grpc to v1.82.1 in otel examples
govulncheck fails every PR on GO-2026-6061 in google.golang.org/grpc
v1.80.0 (indirect via the OTLP exporters); v1.82.1 carries the fix.
---------
Co-authored-by: ofekshenawa <104765379+ofekshenawa@users.noreply.github.com>
|
||
|
Nedyalko Dyakovandofekshenawa
|
279615082b |
feat(search): add FT.ALIASLIST support (#3925)
* feat(search): add FT.ALIASLIST support (Redis 8.10)
- FTAliasList(ctx, index) returns the aliases of an index as an
unordered, server-deduplicated collection; an alias-free index
yields an empty result, not an error
- server errors (SEARCH_INDEX_NOT_FOUND, NOPERM) propagate unchanged;
the argument must be an index name, aliases are not resolved
- wire-format unit test plus a RESP2/RESP3 integration test that gates
on the live server supporting the command (the REDIS_VERSION float
cannot tell 8.10 from 8.1)
- runnable example under example/ft-aliaslist
* fix(search): route FT.ALIASLIST as keyless in cluster
The static FT policy table covers aliasadd/aliasupdate/aliasdel but
not aliaslist, so the default cluster resolver returned nil and a cold
command-info cache fell back to treating the index name as a key.
Register it keyless and read-only like the other alias commands.
* test(search): version-gate FT.ALIASLIST specs
Replace the standalone probe test (unknown-command sniffing plus the
REDIS_ALIASLIST_TEST_ADDR env var) with Ginkgo specs in the main suite,
gated on SkipBeforeRedisVersion("8.10"), which compares major.minor as
integers and so distinguishes 8.10 from 8.1. The specs cover RESP2 and
RESP3 and follow the suite conventions (FlushDB per spec, :6379).
* chore(deps): bump grpc to v1.82.1 in otel examples
govulncheck fails every PR on GO-2026-6061 in google.golang.org/grpc
v1.80.0 (indirect via the OTLP exporters); v1.82.1 carries the fix.
* fix(search): route policy-table keyless commands on cold cache
ClusterClient.process computes the slot before routeAndRun consults
the policy table, and with a cold command-info cache
cmdFirstKeyPosWithInfo fell back to treating argument 1 as a key — so
FT.ALIASLIST (and the other keyless FT commands) hashed the index name
into a slot for the first calls of a process. Consult the static
policy table there: entries with default request routing and a keyless
response policy resolve to keyless. Key-carrying (ft.suglen) and
specially-routed (ft.cursor) commands keep the old fallback.
---------
Co-authored-by: ofekshenawa <104765379+ofekshenawa@users.noreply.github.com>
|
||
|
Nedyalko Dyakov
|
5c82f57a47 |
chore(options): align default config values with cross-SDK proposal (#3918)
* feat: align default config values with cross-SDK proposal Read/Write timeout 3s -> 5s, retry backoff 8ms/512ms -> 10ms/1s, cluster state reload 10s -> 60s, TCP keep-alive 5min blanket -> 30s idle / 5s interval / 3 probes via net.KeepAliveConfig. * test(options): pin new defaults, share keep-alive config - extract defaultKeepAliveConfig and use it from both default dialers so options.go and sentinel.go cannot drift apart - pin the 10ms/1s retry backoff defaults across Options, ClusterOptions and RingOptions, plus the -1 escape hatch, the 60s ClusterStateReloadInterval default, and the keep-alive values - drop the go1.7 build tag from options_test.go: since Go 1.21 such a tag downgrades the file's language version, which rejects net.KeepAliveConfig |
||
|
Nedyalko Dyakovandofekshenawa
|
76322e0bac |
chore(ai): add claude skills (#3855)
* chore(ai): add claude skills * add readme for claude configs * chore(ai): add /check-ci slash command * chore(ai): share repo guidance via AGENTS.md Move the layout/commands/architecture/conventions guidance into AGENTS.md so non-Claude agents (Codex, Cursor, Aider, ...) read the same source. CLAUDE.md now imports it via @AGENTS.md and keeps only Claude-specific notes (skills auto-trigger, slash commands). Commands are inlined rather than delegated to skills, since other tools cannot read skills. * chore(ai): add prepare-release skill Playbook to stage a release locally (semver pick, PR gathering, RELEASE-NOTES.md per template, scripts/release.sh bump, scripts/tag.sh dry-run) without ever tagging, pushing, or committing. List it in AGENTS.md; drop CLAUDE.md's duplicate skill enumeration so names live only in AGENTS.md. * update add command skill. * chore(ai): grant allowed-tools to skills add-command: WebFetch (fetch spec + docs). prepare-release: Bash(git describe:*), Bash(gh pr list:*). * chore(ai): update settings.json * docs(ai): fix test commands and stale references - ginkgo focus example used ./..., which fails every non-ginkgo package with 'flag provided but not defined' - TestPoolNew does not exist; point at TestConnStateMachine - use real CI check names in /check-ci and lead with test-redis-ce - correct keyless-command example, backoff wording, file pointers; swap drifting line refs for symbol anchors - normalize settings.json permission style; drop personal caveman block from .gitignore and the /loop reference --------- Co-authored-by: ofekshenawa <104765379+ofekshenawa@users.noreply.github.com> |
||
|
Nedyalko Dyakov
|
5c4977368b |
fix(redisotel): avoid data race on shared attrs during MinIdleConns warmup (#3881)
attribute.NewSet sorts and de-duplicates its input slice in place. In poolStatsAttrs we spread conf.attrs directly into NewSet, so the shared conf.attrs backing array is mutated on every call. That same array is aliased by every metricsHook.attrs (metricsHook is added with attrs: conf.attrs) and read concurrently in the Dial/Process hooks via append(..., mh.attrs...). With a ClusterClient or Ring, conf is shared across all nodes, so one node's registerClient -> poolStatsAttrs sorts the array while another node's background MinIdleConns dial reads it -- the data race reported in #3880. Copy conf.attrs before handing it to NewSet so the shared array is never mutated at runtime. This is lock-free and removes the unnecessary shared mutation rather than serializing around it. Fixes #3880 |
||
|
Nedyalko DyakovandCopilot Autofix powered by AI
|
663bfb384f |
fix(maintnotif): tight race condition windown on handoff (#3872)
* fix(maintnotif): tight race condition windown on handoff * chore(tests): add pool hook test * address pr comment * chore(tests): add tests for race condition * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
|
Nedyalko Dyakov
|
909c611a50 | drop test (#3863) | ||
|
Nedyalko DyakovandCopilot Autofix powered by AI
|
15f526015a |
feat(search): add SHARD_K_RATIO to FT.HYBRID KNN clause (Redis 8.8+) (#3841)
* feat(search): add SHARD_K_RATIO to FT.HYBRID KNN clause (Redis 8.8+)
Redis 8.8 added the SHARD_K_RATIO argument to the FT.HYBRID KNN clause
so callers can request fewer candidates per shard in cluster mode,
trading recall for latency. Valid range is 0.1 - 1.0; has no effect on
standalone Redis.
Expose it as a typed float64 field on FTHybridVectorExpression. The
zero value means "unset" and falls back to the server default of 1.0
(no per-shard reduction), which matches the way every other numeric
option in FTHybridVectorExpression behaves and keeps the struct literal
syntax clean for callers that do not need to tune the ratio.
The token is emitted after the KNN method block and before FILTER on
the VSIM clause, matching the documented syntax:
VSIM @field $vec KNN 2 K 10 SHARD_K_RATIO 0.6 FILTER ...
Users could already pass SHARD_K_RATIO through MethodParams, but a
typed field documents the new option, keeps the value out of an
interface{} slice, and matches the shape of the other typed KNN args.
Unit tests verify the token ordering (KNN < SHARD_K_RATIO < FILTER),
the value type, and that the token is omitted when ShardKRatio is 0.
See: https://redis.io/docs/latest/develop/whats-new/8-8/
* add additional guard
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
|
||
|
Nedyalko Dyakov
|
1551837b70 |
chore(release): 9.21.0 (#3857)
* bump version to 9.21.0 * add release notes |
||
|
Nedyalko Dyakov
|
74d9bb03f8 |
feat(command): add zero-copy GetToBuffer and SetFromBuffer (#3834)
* feat(command): add zero-copy GetToBuffer and SetFromBuffer
Adds two new methods on the StringCmdable interface that let callers read
and write Redis string values directly into/from pre-allocated byte
buffers, eliminating the per-call payload allocation that the regular
Get/Set methods incur.
Public API:
GetToBuffer(ctx, key, buf) *ZeroCopyStringCmd
SetFromBuffer(ctx, key, buf) *StatusCmd
ZeroCopyStringCmd { Val() int; Bytes() []byte; Result() (int, error) }
Available on *Client, *ClusterClient, *Ring, *Conn and Pipeliner via
* fix(proto): avoid stale line[0] after bufio refill in ReadStringInto
ReadLine() returns a slice into bufio.Reader's internal buffer (the
ReadSlice fast path). The follow-up io.ReadFull / Discard calls on
r.rd can trigger bufio.fill(), which copies new source bytes over the
slot line[0] used to point at. The post-read
if line[0] == RespVerbatim { ... }
check on line 520 of reader.go could therefore observe a refilled byte
rather than the original response type. When that refilled byte
happened to be '=' (0x3D, RespVerbatim) — for example the head of a
bulk-string payload whose tail forced the refill — a regular `$<n>...`
Fix: capture line[0] into a local `isVerbatim` boolean before any
further read on r.rd, and use that local for both the pre-read length
check and the post-read prefix-stripping branch.
Regression test (internal/proto/reader_test.go): a payload sized to
exactly DefaultBufferSize, all '=' bytes. After ReadLine consumes the
header, bufio holds (bufsize - headerLen) payload bytes; the final
headerLen bytes io.ReadFull asks for force bufio.fill(), which places
payload-tail + trailing \r\n at position 0 of bufio.buf — i.e.
'=' at the slot line[0] points at. Against the buggy code the test
fails with n=32764 wanted 32768; with the fix all 10 ReadStringInto
tests pass.
* chore(ci): bump bench test timeout to 11m
* chore(bench): cap CI bench wrappers at 64 KiB; cover 10 MiB via test
* fix(bench): keep proto Reader benches O(1) in b.N via loopFrameReader
* fix(proto): drain payload on buffer-too-small; drop verbatim handling
Three related cleanups to ReadStringInto and the cmd it backs:
1. Buffer-too-small now drains the unread payload + trailing CRLF before
returning the error, so the underlying connection stays aligned for
the next reply. Without this drain, a too-small caller buffer left
the bulk payload sitting in the bufio stream — the connection would
then be returned to the pool in a corrupted state, and the next user
of that connection would parse the leftover bytes as a malformed
response. The existing test only checked the first call's error and
missed this; it now feeds two consecutive bulk-string replies and
asserts the second parses cleanly. Verified failure-mode by reverting
just the drain: the follow-up read fails with
`can't parse reply="hello" reading string into buffer`.
2. Drop RespVerbatim handling from ReadStringInto. The only public
caller is GetToBuffer, which issues GET, and GET never returns a
verbatim string — RESP3 verbatim replies come from server-info
commands like CLIENT INFO / DEBUG OBJECT / LOLWUT. Keeping the
branch re-introduces a hazard class where the response-type byte
read from a stale line[0] after a bufio refill can be misinterpreted
as the verbatim format tag (the bug fixed in
"fix(proto): avoid stale line[0] after bufio refill in ReadStringInto").
Removing the branch eliminates the class structurally and drops an
O(n) memmove on the verbatim path (`copy(buf, buf[4:n])`).
The unit test that previously asserted verbatim parsing is reshaped
to assert verbatim now surfaces as an error, and the
BulkStringStartingWithEqRefill regression test is kept (and its
comment updated) as a guard against re-introducing verbatim support
without re-introducing the bug.
3. Add a godoc paragraph to ZeroCopyStringCmd.Clone explaining the
deep-copy semantics (clone gets a freshly allocated buffer, not a
slice of the caller's memory) and noting that Clone is unreachable
through normal client flows: it is only called from cluster fan-out
routing in osscluster_router.go for multi-shard commands like
DBSIZE / KEYS / FLUSHDB, and ZeroCopyStringCmd is only produced by
GetToBuffer which issues GET — a single-key command never fanned
out. Combined with NoRetry()=true blocking the retry path, Clone
exists only to satisfy the Cmder interface and to keep the cmd usable
if a future caller does fan out.
* fix(zerocopy): clone fails explicitly; reset cmd.n; update doc
Three small follow-ups to ZeroCopyStringCmd identified during a full
review of the zero-copy buffer code path:
1. Clone() now returns a marked clone whose readReply drains the
network reply and surfaces an explicit error rather than producing
silently-wrong results. Cloning a ZeroCopyStringCmd has no
well-defined semantics: the cmd writes into caller-owned memory
(the buf passed to GetToBuffer), and a clone can neither share that
buf (sibling clones would race, last-writer wins) nor allocate its
own buf (the result would be invisible to whoever asked for the
original cmd's reply). The previous implementation chose the second
path and lost the result; this commit chooses to fail loudly.
The Cmder interface requires Clone, so we still return a clone.
The clone is flagged via a new private field, and readReply checks
that flag, calls rd.DiscardNext() to keep the connection aligned,
and returns a clear error ("redis: ZeroCopyStringCmd cannot be
cloned (cmd writes into caller-owned memory)").
Pre-setting cmd.err in Clone would NOT work because pipelineReadCmds
overwrites it with whatever readReply returns — the drain-and-error
path is what actually survives the framework.
In practice this path is unreachable through normal client flows:
Clone is only called from cluster fan-out routing in
osscluster_router.go for multi-shard commands (DBSIZE / KEYS /
FLUSHDB), and ZeroCopyStringCmd is only produced by GetToBuffer
which issues GET — a single-key command routed to one shard, never
fanned out. Combined with NoRetry()=true blocking the retry path,
the clone path is exercised only by explicit caller code.
A new Ginkgo spec asserts both that processing a clone returns the
expected error and that a follow-up GetToBuffer on the same client
succeeds (the drain keeps the connection usable).
2. readReply now resets cmd.n = 0 on entry so that an error after a
previous successful run can't leak the old data through Bytes().
Cheap defensive belt-and-braces — the caller is supposed to check
Err() first, but this removes a stale-state failure mode entirely.
3. The ZeroCopyStringCmd type godoc previously claimed "no data is
consumed beyond the header line" on the buffer-too-small path.
That stopped being true after the previous commit added the drain
to ReadStringInto. Updated to match reality.
Verified end-to-end against Redis 8.8 in Docker:
orig GetToBuffer: OK
cloned cmd processing: errored as expected
follow-up GetToBuffer: OK — connection stays aligned
* docs(string): clarify that SetFromBuffer is API symmetry, not a perf path
SetFromBuffer and Set(ctx, key, buf, 0) dispatch to the same `case []byte`
branch in proto.Writer and produce byte-identical output on the wire. The
"zero-copy on send" property comes from bufio.Writer.Write bypassing its
internal buffer for payloads larger than that buffer — a property of
bufio, not of SetFromBuffer. Set([]byte) gets the same behaviour for
free.
Without this clarification, callers reading the godoc could reasonably
infer that SetFromBuffer is a performance optimisation over Set, choose
it for that reason, and be disappointed when the bench numbers come back
indistinguishable from Set([]byte). The added paragraph spells out the
actual rationale (API symmetry with GetToBuffer for code that uses the
buffer-based pattern on both sides) and tells callers who don't care
about that symmetry that Set(ctx, key, buf, 0) is equally efficient.
No behaviour change.
|
||
|
Nedyalko Dyakov
|
a13416bf1c |
chore(release): 9.20.1 (#3847)
* bump version to 9.20.1 * Add release notes |
||
|
Nedyalko Dyakov
|
10dc44f424 |
fix(push): fix peeking when push name is truncated (#3842)
* fix(push): Fix peeking when push name is truncated * fix(push): fix peeking malformated push * protect agains potential panic |
||
|
Nedyalko DyakovandCopilot Autofix powered by AI
|
e1a2d68b27 |
fix(ft.hybrid): Always generate vector param names if they are not provided by the user (#3844)
* feat(search): always pass FT.HYBRID vector data via PARAMS Redis no longer supports inline vector blobs in FT.HYBRID. Always pass vector data through the PARAMS mechanism, generating a unique parameter name when VectorParamName is not provided. Update the related comments and drop the now-obsolete 8.6+ skip guards in the integration tests. * chore(search): add tests for generating vector param names * doc the limitations * use local params * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * fix build --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
|
Nedyalko Dyakov
|
7d05dd3b7c |
chore(release): v9.20.0 (#3832)
* chore(release): add release notes * Update readme and release notes. Deprecate UnstableResp3 everywhere * chore(release): bump version v9.20.0 |
||
|
Nedyalko Dyakovandofekshenawa
|
9756882712 |
fix(test): make waitForSentinelClusterStable robust to disconnected r… (#3830)
* fix(test): make waitForSentinelClusterStable robust to disconnected replicas
CI on master sporadically fails the universal_test.go spec
"should connect to failover servers on slaves when readonly Options is ok"
with `Expected master to equal slave`. The flake reproduces only when the
random spec order puts the sentinel-failover Describe block before the
universal_test spec.
Root cause is in waitForSentinelClusterStable's post-conditions, not in
production code:
1. It only checks `len(replicas) >= 2`. Production
(parseReplicaAddrs in sentinel.go) filters out replicas flagged
`s_down`, `o_down`, or `disconnected`. Post-failover, replicas
often linger as `disconnected` for several seconds after Sentinel
agrees on the master/replica counts, so the stabilization check
can pass while production sees zero usable replicas.
2. It then waits a fixed 10s. With no link between that sleep and
the actual precondition (a FailoverClient(ReadOnly:true) finding
a routable replica), the test gambles that 10s is enough on every
CI host. When it isn't, RandomReplicaAddr silently returns the
master and the spec asserts on the wrong role.
Two changes:
- Add countUsableReplicas, mirroring parseReplicaAddrs's filter, and
require each sentinel to report at least 2 usable (non-down,
non-disconnected) replicas with consistent counts across sentinels.
- Replace the 10s sleep with a bounded Eventually that opens the same
kind of client the failing spec uses (UniversalClient + MasterName +
ReadOnly:true) and confirms ROLE returns "slave". This is the
deterministic equivalent of waiting "long enough" — it returns as
soon as the actual precondition holds, and bounds at 30s if the
cluster never stabilizes.
No production code changes; the flake is entirely in test harness
post-conditions. The new probe uses only public API and existing
package-level fixtures (sentinelName, sentinelAddrs).
* fix(test): harden sentinel ReadOnly spec against silent master fallback
CI on master sporadically fails the universal_test.go spec
"should connect to failover servers on slaves when readonly Options is ok"
with `Expected master to equal slave`. Two random-spec-ordering bug
paths combine to produce this:
1. sentinel_test.go has three failover-causing specs across three
Describe blocks ("Sentinel", "NewFailoverClusterClient", and
"Sentinel Failover with Conns"). Only the third runs Ordered/Serial
with an AfterAll that calls waitForSentinelClusterStable. The
first two can land right before the universal_test spec under any
random spec ordering and leave the cluster mid-recovery.
2. FailoverClient(ReadOnly:true) routes via RandomReplicaAddr, which
silently falls back to the master when Sentinel reports zero
usable replicas. parseReplicaAddrs in sentinel.go filters out
replicas flagged s_down, o_down, or disconnected — and freshly
promoted/demoted replicas often linger as "disconnected" for
several seconds after Sentinel agrees on master/replica counts.
Two test-only changes (no production code touched):
a) waitForSentinelClusterStable: previously only checked
len(replicas) >= 2, which can pass while every replica is
filtered out by production code. Now mirrors parseReplicaAddrs's
filter via countUsableReplicas and requires each sentinel to
report >= 2 usable replicas with consistent counts across
sentinels. The fixed 10s sleep is replaced with a bounded
Eventually that opens the same UniversalClient(ReadOnly:true)
the failing spec uses and confirms ROLE returns "slave" — the
deterministic equivalent of "wait long enough".
b) universal_test.go spec: defense in depth. Since the failover
specs in (1) don't all share an AfterAll, the spec itself
retries client creation + ROLE in a bounded Eventually. Each
iteration closes the old client and creates a new one so
RandomReplicaAddr is re-evaluated against current Sentinel
state. Bounded at 30s so a genuinely broken setup still fails.
The (a) fix keeps the post-failover state clean for any downstream
test; the (b) fix keeps this specific spec correct regardless of
ordering. Together they make the spec deterministic across all 1414!
possible random spec orderings.
* fix(test): make ReadOnly SET assertion atomic with the ROLE probe
Previous attempt fixed the ROLE check ("Expected master to equal slave")
by retrying client creation inside Eventually until ROLE returned
"slave". CI surfaced a second failure mode in the same spec:
[FAILED] Expected an error to have occurred. Got: <nil>: nil
universal_test.go:110
The spec's intent was: ReadOnly client lands on a replica, and a write
against that replica returns READONLY. The new failure means the write
succeeded — there's a second path back to the master:
SET on replica → READONLY error
→ shouldRetry returns true for READONLY (error.go)
→ conn discarded, retried via masterReplicaDialer
→ RandomReplicaAddr called again, may silently fall back to master
when no usable replica is reported (same root cause as the original
flake — Sentinel sees 2 replicas but both are "disconnected")
→ SET hits the master, succeeds, err is nil
→ Expect(err).To(HaveOccurred()) fails
Two changes to make the spec atomic and bypass-proof:
- Set MaxRetries: -1 on the test client. A READONLY reply now
surfaces as the first/only SET error instead of triggering a
retry-dial that may pick the master. This isolates what the test
is actually asserting: a single round-trip to whatever node the
dialer picked.
- Move the SET assertion inside the Eventually closure together
with the ROLE check. If either step lands on the master (because
RandomReplicaAddr fell back at dial time), the closure returns an
error, Eventually closes the client and retries with a fresh one
so the dialer re-queries Sentinel. The full assertion is therefore
atomic — both ROLE and SET observe the same dial.
Error messages from the closure name the precise failure mode so a
genuinely broken setup (timeout after 30s) reports something more
useful than "expected slave, got master".
No production code changes; the retry-on-READONLY behaviour is correct
for normal failover scenarios and stays untouched.
---------
Co-authored-by: ofekshenawa <104765379+ofekshenawa@users.noreply.github.com>
|
||
|
Nedyalko Dyakov
|
d8407df30a |
fix(pubsub): include shard channels in newConn routing list (#3829)
In ClusterClient.SSubscribe, the per-PubSub newConn closure picks the target node by hash slot of channels[0]. PubSub.conn() was only passing c.channels (regular SUBSCRIBE) into that closure — sharded SSubscribe channels stored in c.schannels were silently dropped. For an SSubscribe-only PubSub (the common ClusterClient.SSubscribe case) this meant the channel list was empty on reconnect, so the closure fell through to nodes.Random(). The reconnected SSUBSCRIBE went to the wrong shard, the MOVED reply was never read by the write-only resubscribe path, and the PubSub looked healthy (Ping succeeds, Receive returns) while delivering no messages. On an N-shard cluster the failure mode hit (N-1)/N reconnects. Add c.schannels to the channel list built in PubSub.conn(). c.patterns is intentionally NOT added: patterns are not slot-addressable, and including them would pin PSubscribe-only PubSubs to a single pattern-hash node and lose the existing random-node behaviour. Adds TestPubSubConn_PassesPersistedChannelsToNewConn covering five cases (subscribe-only reconnect, ssubscribe-only reconnect, initial ssubscribe, mixed, and ordering of newChannels) via a stub newConn that captures the channel list. The cluster routing layer itself is not under test — the regression is whether the channel list reaches the closure intact. Fixes #3806 |
||
|
Nedyalko Dyakov
|
6af9bdc84b |
fix(cluster): fall back to origin port when CLUSTER SLOTS reports port 0 (#3828)
In TLS-only clusters started with `--port 0 --tls-port <port>`, CLUSTER SLOTS reports the regular port as 0. newClusterState used to pass that value verbatim into clusterNode.Client.opt.Addr, which the PubSub path dials directly with no MOVED/seed fallback, producing `dial tcp <ip>:0: connection refused`. Add a replaceZeroPort helper symmetric to replaceLoopbackHost: when a slot node's port is "0", substitute the origin endpoint's port — by definition reachable, since it is the port we used to fetch the slot map. NodeAddress is left untouched so maintenance-notification matching continues to see the server-reported value. Adds TestNewClusterState_PortZero covering the bug scenario plus the non-zero passthrough and the loopback+:0 composition. Fixes #3726 |
||
|
Nedyalko Dyakov
|
fa5aa8c4d2 |
chore(doc): Update README and CI image. (#3822)
* chore(doc): Update README and CI image. * Use latest image |
||
|
Nedyalko Dyakovandofekshenawa
|
14a9a85363 |
feat(search): parser resp3 responses for search (#3741)
* parse resp3 responses * backwards compatibility * doctests and format * fix parsing * remove unused functions * debug test * debug test x2 peek directly * debug test * remove debug and skip re build * fix typo * add tests and fix parsers for resp3 * add comment for the unreliable Total in RESP3 for aggregate * fix build * remove shadowing rawVal * fix strange redis search float,int,string behaviour handling * mark UnstableResp3 as Deprecated --------- Co-authored-by: ofekshenawa <104765379+ofekshenawa@users.noreply.github.com> |
||
|
Nedyalko Dyakov
|
c5f08a5847 |
feat(command): add support for INCREX (#3816)
* feat(command): add increx * add one additional test * update increx * use custom 8.8 image with new increx * address pr comments * always use by in float |
||
|
Nedyalko Dyakov
|
b88fd9b372 |
chore(extra): Add LICENSE file to extra packages (#3817)
This way the LICENSE will be shipped with those extra packages as well. |
||
|
Nedyalko Dyakov
|
5305816c42 | fix(pool): CloseConn to trigger hooks if available #3812 (#3818) | ||
|
Nedyalko Dyakov
|
ad7fabb336 | fix(test): flaky tests (#3815) | ||
|
Nedyalko Dyakov
|
3dfa4b663a |
feat(json): Add JSON.SET fpha support (#3797)
* add json.set fpha support * fix json set test |
||
|
Nedyalko Dyakov
|
85bca80945 | fix(pool): Fix Close. Do healthchecks only on idle conns (#3805) | ||
|
Nedyalko Dyakov
|
e7e9866e54 |
chore(release): v9.19.0 (#3796)
- Add release notes - Add release checklist - Fix release script to update deps |
||
|
Nedyalko DyakovandElena Kolevska
|
22b26f4b3e |
feat(ft.aggregate): Add Steps for query building (#3782)
* ft.aggregate with steps * ignore deprecated fields when steps are present * add example app * address pr comments --------- Co-authored-by: Elena Kolevska <elena-kolevska@users.noreply.github.com> |
||
|
Nedyalko DyakovandCopilot
|
44e8b7309c |
fix(sch): auto hostname type detection (#3789)
* fix hostname detection * Update maintnotifications/config.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update maintnotifications/config.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix log level check * fix ip when tls is off * handle empty address --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
Nedyalko Dyakov
|
ad21622ce8 | fix(hello): do not send maintnotifications handshake when hello fails (#3788) | ||
|
Nedyalko Dyakov
|
00a551b3b4 |
fix(credentials): leak in wrappedOnClose (#3785)
* fix leak in wrappedOnClose * address linter warning |
||
|
Nedyalko Dyakov
|
462843a275 |
chore(deps): Update otel sdk (#3770)
* update otel deps * update otel deps to 1.43 * go mod tidy * bump ci to use go 1.25 |
||
|
Nedyalko Dyakov
|
8a4fed48a9 | fix(setnx): use set with nx instead setnx (#3723) | ||
|
Nedyalko Dyakov
|
dcf53a6190 | chore(example): remove throughput binary (#3733) | ||
|
Nedyalko Dyakov
|
3b155e9443 |
chore(go): update go version to 1.24 (#3714)
* chore(go): update go version to 1.24 * update readmes and ci workflows * update dockerfile for proxy-fi-server * address changed test behaviour * address changed test behaviour |
||
|
Nedyalko Dyakov
|
90faf06923 | chore(release): update versions in deps (#3712) | ||
|
Nedyalko Dyakov
|
bf8e8e3b16 |
chore(release): v9.18.0 (#3711)
* chore(release): add release notes * chore(release): update version |
||
|
|
b53f2b0c9d |
feat(sch): MaintNotifications for ClusterClient (#3643)
* [maintnotif] Cluster specific handlers (#3613) * maint notification handlers for cluster messages * unrelax all conns * trigger ci on feature branches * feat(maintnotif): lazy cluster topology reload (#3614) * lazy cluster topology reload * fix discrepancies between options structs * Update osscluster_lazy_reload_test.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update osscluster.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * feat(e2e): mock maintnotif e2e tests (#3639) * lazy cluster topology reload * fix discrepancies between options structs * Update osscluster_lazy_reload_test.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update osscluster.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * wip fault with mock proxy * make lint happy * fix linter issues * faster tests with mocks * linter once again * add complex node test * add ci e2e * use correct redis container * e2e fix --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix(retry): Add retry mechanism for NOREPLICAS error (#3647) * fix(queuedNewConn): protect against nil context (#3649) * fix(maintnotif): fix smigrated parser and add cluster state reload interval option (#3663) * fix SMIGRATED parsing * fix smigrated parser * add ClusterStateReloadInterval to ClusterOptions * fix tests * set default cluster reload interval to 10s * chore(lint): format * feat(smigrated): new format & remember original host:port (#3697) * lazy cluster topology reload * fix discrepancies between options structs * Update osscluster_lazy_reload_test.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update osscluster.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * wip fault with mock proxy * make lint happy * fix linter issues * faster tests with mocks * linter once again * add complex node test * add ci e2e * use correct redis container * e2e fix * additional e2e tests * fix data race * fix random shard picker * fix e2e tests * fix for empty endpoint * fix case when semaphore is full, but still need to check idle * scenario tests * create database from config * wip * feat(client): store original addrs for later use * fix(notif): change smigrated notification * fix(lint): fix linter * fix(smigrated): use array * fix(e2e): wip * Update options.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update redis.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix(notif): if the conn has no original addr, trigger reload with first target * chore(e2e): wip cluster e2e * chore(e2e): fix nil pointer from e2e tests * chore(e2e): fix tests and reports * chore(e2e): proper logging in e2e * chore(e2e): add pubsub in the tests as well * chore(e2e): mockproxy fixes * chore(e2e): mockproxy fixes * chore(e2e): mockproxy fixes * chore(e2e): mockproxy fixes v3 * stop background routines * fix(e2e): tests --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * chore(docs): add example app (#3651) * chore(lint): fmt * chore(docs): improve docs * chore(docs): update features.md * chore(github): remove example * chore(maintnotif): rename option and address pr comments * fix(e2e): command runner should use client timeout * chore(e2e): refactor tests * fix(e2e): set default timeout to 90m * fix(e2e): skip tests if proxy cannot be started --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: ofekshenawa <104765379+ofekshenawa@users.noreply.github.com> Co-authored-by: Elena Kolevska <elena-kolevska@users.noreply.github.com> |
||
|
Nedyalko DyakovandElena Kolevska
|
f25343d281 |
chore(tests): Add comprehensive TLS tests and example (#3681)
* chore(tests): add tls tests * fix: Use ShardPicker for keyless commands instead of random slot - cmdSlot now returns -1 for keyless commands instead of RandomSlot() - This allows cmdNodeWithShardPicker to use the configured ShardPicker (default: RoundRobinPicker) for keyless command routing - Updated mapCmdsByNode to handle slot == -1 by using ShardPicker - Updated test to verify round robin distribution works correctly * chore(image): update clients-test-lib image * chore(tests): enable tls user auth test * chore(docker): use proper envs for test image * chore(image): use latest build * chore(image): use latest build * fix(tests): use the correct root cert * feat(tls): add example app * chore(docker): address pr comments * chore(tests): address pr comments * chore(doc): add README.md in the example app * fix(tests): do not skip tests * fix(tests): skip only because of redis version * Extracts default image name in a var (including osscluster-tls) Signed-off-by: Elena Kolevska <elena@kolevska.com> * fix(tlstests): remove redundent test --------- Signed-off-by: Elena Kolevska <elena@kolevska.com> Co-authored-by: Elena Kolevska <elena@kolevska.com> |
||
|
|
b0cb6c4330 |
chore(ci): use 8.6-pre (#3685)
* chore(ci): use 8.6-pre
* chore(ci): add 8.6 in ci matrix
* fix(xinfostream): added new fields in the response
* fix(doctests): fix expected output for XInfoStream
* fix(xinfostreamfull): same as xinfostream
* fix(doctests): fix expected output for XInfoStream
* Support hybrid search params vectors
* Support xinfo stream change in tests
* Revert "Support xinfo stream change in tests"
This reverts commit
|
||
|
Nedyalko Dyakov
|
06c7fe36cf |
fix(xadd,xtrim): when approx is false, should have = (#3684)
|
||
|
Nedyalko Dyakov
|
0c344e2f69 | chore(cmd): add deprecation warnings (#3673) | ||
|
Nedyalko Dyakov
|
452eb3d15f | fix(queuedNewConn): protect against nil context (#3649) | ||
|
Nedyalko Dyakov
|
e4965ea90b |
chore(release) v9.18.0-beta.2 (#3644)
* add release notes * update version number * update date |
||
|
Nedyalko DyakovandccoVeille
|
674c7b8695 |
chore(go): update go version to 1.21 (#3640)
* chore(go): update go version to 1.21 * chore(aggregators): make aggregators work with 1.21 * fix doctests * address copilot comments * use atomic bool for logic and/or aggregators * Update .github/workflows/build.yml Co-authored-by: ccoVeille <3875889+ccoVeille@users.noreply.github.com> * use stable/oldstable, 1.23 and 1.21 * fix versions in README * add oldstable in wordlist --------- Co-authored-by: ccoVeille <3875889+ccoVeille@users.noreply.github.com> |
||
|
Nedyalko Dyakov
|
981f5db9a2 |
chore(release): 9.18.0-beta.1 (#3628)
* bump version to 9.18.0-beta.1 * add release notes |
||
|
Nedyalko Dyakov
|
68d8c59557 |
chore(release): v9.17.1 (#3617)
* add release notes * update version to 9.17.1 |
||
|
Nedyalko Dyakov
|
63908223ca | fix(time): remove cached time optimization (#3611) | ||
|
Nedyalko Dyakov
|
1bb9e0d130 | chore(scripts: fix help call in tag.sh (#3606) | ||
|
Nedyalko Dyakov
|
5b0b228a37 |
chore(release): v9.17.0 (#3604)
* update release notes * update readme * update version number * cleanup release notes * fix checkMinIdle |