mirrors/go-redis
0
0 Fork 0
mirror of https://github.com/redis/go-redis.git synced 2026-10-28 08:24:54 +08:00
Code Activity

feat(himport): HIMPORT command with lazy per-connection prepare (#3919)

Browse Source 
* feat(himport): HIMPORT command family with lazy per-connection prepare replay

Fieldsets are server-side session state scoped to one physical
connection, so pooled HIMPORT SET needs the fieldset prepared on
whichever connection executes it. Instead of replaying PREPARE in the
handshake, the client keeps a versioned fieldset registry and a
per-connection prepared flag, and injects the PREPARE into the same
round trip right before a SET that needs it - at most once per
connection session. A 'no such fieldset' reply for a registered
fieldset invalidates the flag and grants one retry that re-prepares.

* fix(himport): drain push notifications between injected PREPARE and command replies

proto.Reader.ReadReply consumes a RESP3 push frame as a normal reply, so
a notification arriving between the injected PREPARE's reply and the
command's reply was read as the command's reply and desynced the
connection. Drain pending notifications again after reading the PREPARE
replies, mirroring what the pipeline read path does before every reply.

The regression test drives a RESP3 mock server that emits a push frame
between the two replies; without the drain the SET fails with "can't
parse reply=\">2\"".

* fix(himport): wire registry into failover and tx clients, invalidate stale pipeline flags

Two review fixes for the lazy prepare-replay machinery:

- NewFailoverClient built its baseClient without the HIMPORT registry,
  so HImportPrepare appeared to succeed but never registered and pooled
  HImportSet replay never engaged. Tx (client.Watch) now shares the
  parent client's registry the same way Client.Conn() does, since its
  sticky pool borrows connections from the parent pool.

- A pipelined HImportSet failing with "no such fieldset" left the
  connection's prepared flag stale, so even a caller-initiated retry of
  the pipeline failed forever on that connection. The failed exec now
  invalidates the flag and the retried pipeline replays the PREPARE.
  Pipelines are deliberately not auto-retried: re-running the batch
  would repeat side effects of commands that already succeeded.

* test(himport): skip integration specs when the server lacks HIMPORT

The 8.10.x CI image is a pre-release build without the HIMPORT patch
series, so the version gate alone let the specs run into "ERR unknown
command 'himport'". Probe with HIMPORT DISCARDALL and skip when the
command family is absent.

* docs(himport): add example app

Walks through prepare/set basics, pipelined bulk ingestion timed
against plain HSET, concurrent pooled writers relying on lazy replay,
the template encoding surfaced by OBJECT ENCODING and INFO memory, and
fieldset discard semantics. Exits gracefully when the server does not
support HIMPORT.

* feat(himport): propagate discards lazily to all pooled connections

HIMPORT DISCARD/DISCARDALL execute on one borrowed connection, so other
pooled sessions kept their fieldset copies: a subsequent HImportSet was
nondeterministic (it succeeded against a session that still held the
fieldset and failed elsewhere), and the leftover copies held server
memory until the connections closed.

Discards now propagate the same way prepares do. A discarded name stays
in the client registry as a tombstone and DISCARDALL bumps a registry
epoch; before any HIMPORT command on a connection, the client injects
HIMPORT DISCARD for tombstoned fieldsets that session still holds, or a
single HIMPORT DISCARDALL when the session predates the current epoch.
Re-registering a name clears its tombstone - the replayed PREPARE
already replaces the fieldset server-side.

Post-discard behavior is now deterministic: HImportSet fails with "no
such fieldset" on every pooled connection, exactly as on a single
connection. Non-HIMPORT traffic never pays for the cleanup; sessions
that stop running HIMPORT commands are cleaned up by connection close,
as before.

* fix(himport): guard root-cause swap, discard stale fieldset

- the single-command path replaced the SET reply with an injected
  PREPARE failure even when the SET succeeded or failed with an
  unrelated error; gate it on "no such fieldset" like the batch path
- an injected re-prepare that fails leaves the session holding the
  older fieldset version, and the pipelined SET behind it then writes
  the old field names silently; discard the held version first so the
  SET answers "no such fieldset" and the root cause is surfaced
- move the test-only registry accessors epoch/empty into the test
  file: the linter runs with tests excluded and flags them as unused

* fix(himport): align recovery and discard semantics with HLD v21

- Pipelines: HIMPORT SETs of a registered fieldset that fail with "no
  such fieldset" are re-prepared and re-issued once on the same
  connection - and only those SETs; HIMPORT SET is a full replace, so
  the re-execution is idempotent and no other command of the batch runs
  again. The error therefore never surfaces for managed fieldsets
  (NF.4). Transactions keep surfacing it - an executed MULTI/EXEC
  cannot be partially re-run - and the invalidated flags make the
  caller's transaction retry succeed.

- HImportDiscard/HImportDiscardAll report the registry lifecycle per
  the HLD method shapes: 1 when the fieldset was registered on this
  client and removed, and the count of registered fieldsets removed.
  The executing session's reply passes through only for names never
  registered through the managed API.

- Document that HIMPORT commands sent through generic interfaces (Do,
  raw Cmd values) bypass the registry and carry none of the managed
  guarantees; the root README generalizes the caveat to every command
  that alters connection session state (SELECT, CLIENT SETNAME, RESET,
  CLIENT TRACKING, ...) and points session-scoped work at Conn().

The working tree also carries the remote "guard root-cause swap,
discard stale fieldset" changes verbatim, so the merge below resolves
cleanly.

* feat(himport): cluster and ring support via shared registry and fan-out

Every ClusterClient node client and Ring shard client now shares one
HIMPORT fieldset registry, adopted at client creation - replicas
included, since roles change with the topology: a promoted replica's
connections carry no prepared flags, so their first HIMPORT SET
replays the PREPARE lazily, and MOVED/ASK redirects re-prepare on the
target node through the existing per-connection machinery with no
cluster-specific code.

On top of the shared registry, user-issued PREPARE, DISCARD, and
DISCARDALL fan out eagerly to all masters (R.4) or all ring shards:
one connection per node is prepared or cleaned up front, server-side
validation surfaces immediately, and leftover session state stays
bounded. Fan-out copies carry a pre-assigned registry version/epoch so
the registry is touched exactly once per operation; a deterministic
server rejection (e.g. duplicate field name) withdraws the
registration, while transport failures keep it and lazy replay covers
the connections the fan-out missed (all-succeeded reporting).

* chore(ci): add fieldset to the spellcheck wordlist

* fix(himport): wire cluster pipelines into the managed layer, close review findings

Review findings from three independent passes (wire protocol,
concurrency/state, HLD compliance):

- ClusterClient pipelines and tx pipelines wrote commands directly to
  node connections, bypassing HIMPORT entirely: no PREPARE replay (the
  fan-out prepares one connection per master, so pooled pipelines hit
  "no such fieldset" nondeterministically), no registry updates for
  pipelined PREPARE/DISCARD (a pipelined DISCARD was later resurrected
  by lazy replay), no re-prepare after MOVED/ASK re-queues, and no
  stale-version defense. processPipelineNodeConn and
  processTxPipelineNodeConn now inject and consume the same HIMPORT
  bookkeeping commands as the standalone paths; failed SETs of
  registered fieldsets re-queue through the pipeline's attempt budget,
  which re-prepares lazily on the next attempt.

- A transport failure inside the standalone pipeline's HIMPORT retry
  round trip signalled canRetry after the first round trip had been
  fully consumed, so the outer loop re-executed the whole batch and
  double-applied non-idempotent commands whose results were already
  delivered. The retry now reports the error without requesting a
  batch retry; the connection is still discarded.

- The handshake Conn handed to OnConnect (and the re-auth Conn) had a
  private HIMPORT registry whose versions restart at 1: an OnConnect
  prepare could mark a pooled connection at a version number colliding
  with the client registry's, silently skipping the replay of a
  different fieldset definition. Both now share the client's registry.

- A fan-out PREPARE rejected by one master withdrew the registration
  without a tombstone, leaving unaddressable fieldsets alive on the
  masters where the fan-out had already succeeded. The withdrawal now
  tombstones, so those sessions are cleaned on their next HIMPORT use.

- NewRing stored the ring-wide registry in the caller-owned
  RingOptions, so two rings built from one options struct shared and
  clobbered each other's registry. NewRing now copies the options.

- Rewrite the stale pre-cluster godoc on HImportPrepare that claimed
  ClusterClient/Ring do not fan out, and document the best-effort
  fan-out failure semantics on the cluster/ring methods.

- newConn now takes the HIMPORT registry as an explicit parameter
  instead of allocating a throwaway one that every call site
  overwrote.

* fix(himport): skip set re-queue when the batch retry already carries it

* fix(himport): scope re-issue failures, unmark while holding the conn

Three review findings:

- A transport error inside the pipeline set re-issue round trip was
  returned to generalProcessPipeline, which stamped it onto every
  command with a nil error - including commands whose results were
  already delivered by the fully-consumed first round trip. The error
  now stays on the retried SETs; the batch neither retries nor fails
  wholesale, and the connection, which may hold unread replies, is
  marked for removal on release (MarkCloseOnPut). Regression-tested by
  dropping the connection on the replayed PREPARE and asserting the
  delivered plain SET keeps its OK result.

- The single-command recovery unmarked the prepared flag from
  process(), after withConn had already released the connection,
  mutating flags on a connection another goroutine may own. The
  invalidation now happens inside _process while the connection is
  still held; himportRecover became himportShouldRetrySet, a pure
  retry decision.

- A connection's discard-all epoch could in principle be overwritten
  backwards by a mark carrying an older registry snapshot. The epoch
  is now forward-only on marks; a stale snapshot can no longer regress
  a session wiped at a newer epoch.

The working tree also mirrors the master-merge changes to _process
(assertUnstableCommand removal) so the merge below resolves cleanly.

* fix(himport): survive mass session loss, honor close-on-put in sticky pools

Deep-review findings from three independent passes (fresh-eyes diff,
behavioral simulation against a session-faithful mock, regression
safety), plus one final-review nit:

- Mass session loss (failover, cross-region switch, reset storms)
  could beat the single-command retry-once budget: the retry landed on
  a different connection whose session was wiped by the same event but
  whose prepared flag still looked current, so the PREPARE was not
  injected and "no such fieldset" surfaced for a managed fieldset
  (deterministic repro: 2-3 surfaced errors per 1200 SETs in a reset
  storm). A "no such fieldset" failure now bumps the fieldset's
  registry version (fields kept, guarded against concurrent
  re-registrations, once per batch), staling every connection's mark
  so the retry re-prepares wherever it lands.

- StickyConnPool.Put ignored MarkCloseOnPut, so a Conn/Tx whose
  HIMPORT re-issue round trip failed on a transport error got the same
  connection back with the re-issued replies still unread - the next
  command read a leftover reply as its own (live repro: HImportDiscard
  parsing "OK" as an integer; a status-typed command would accept the
  wrong reply silently). Sticky Put now records the mark as a bad-conn
  error, Get refuses the connection, and the parent pool removes it on
  unwind.

- The standalone pipeline and tx-pipeline paths stamped no per-command
  errors when the injected-reply read died on a transport error: the
  outer retry loop stamps only on its exit branch, not when the
  attempt budget runs out mid-loop, so Exec surfaced the error while
  every command still reported Err() == nil. Both paths now stamp the
  batch like a write failure, matching the cluster pipeline; the
  regression test drives it through the mock (connection dropped on
  the injected PREPARE, MaxRetries -1).

- The "no such fieldset" check after every single-command reply ran
  before any type check; its errors.As chain allocates, costing every
  non-HIMPORT command 2 heap allocations (+32 B, ~21 ns, +12.5%
  allocs/op for a simple SET). The HImportSet type assertion now runs
  first: ~1 ns and zero allocations for everything else.

Doc hardening: cross-node PREPARE+SET pipelining limitation on
ClusterClient, tombstone growth bound, the recovery attempt granted
under MaxRetries -1, and the ASKING re-issue comment corrected.

* docs(himport): mark the HIMPORT API as experimental

The server command family ships behind the hinted-hash-templates
feature of Redis 8.10 pre-releases and its surface may still move;
the typed client API (methods, HImport*Cmd types, constructors)
follows it. Same wording and placement as the vectorset commands.

---------

Co-authored-by: ofekshenawa <104765379+ofekshenawa@users.noreply.github.com>
This commit is contained in:
Nedyalko Dyakovandofekshenawa authored and GitHub committed 2026-07-29 14:52:32 +03:00
1 parent 3dd9675caf
commit 49e004104c
21 files changed
+3512 -16
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.github/wordlist.txt
+2
View File
@@ -18,6 +18,8 @@ de
DisableIdentity
ElastiCache
extensibility
fieldset
fieldsets
FPM
Golang
IANA
README.md
+18
View File
@@ -457,6 +457,24 @@ vals, err := rdb.Eval(ctx, "return {KEYS[1],ARGV[1]}", []string{"key"}, "hello")
res, err := rdb.Do(ctx, "set", "key", "value").Result()
```
### Raw commands and connection state
`Do` sends the command verbatim on whichever pooled connection happens to be
free. For keyspace commands that is all you need. It is the wrong tool for
any command that alters **connection session state** — `SELECT`,
`CLIENT SETNAME`, `CLIENT TRACKING`, `RESET`, `HIMPORT PREPARE`/`DISCARD`,
and similar: the state lands on (or is wiped from) a single arbitrary
connection, later commands are served by other connections that don't share
it, and the affected connection eventually returns to the pool and serves
unrelated callers. The result is nondeterministic behavior that typed APIs
manage for you — for example, the typed `HImport*` methods keep a
client-side registry and replay fieldsets onto every connection that needs
them, while a raw `Do(ctx, "himport", "prepare", ...)` bypasses that
entirely, with no replay, recovery, or discard propagation.
For session-scoped work without a typed API, hold a dedicated connection
(`client.Conn()`) for its whole lifetime and close it afterwards.
## Typed Errors
go-redis provides typed error checking functions for common Redis errors:
example/himport/README.md
+51
View File
@@ -0,0 +1,51 @@
# HIMPORT example
Fast hash ingestion with the HIMPORT command family (hinted hash templates,
Redis 8.10+).
> **Experimental.** The HIMPORT commands and the typed go-redis API around
> them are experimental and may change in a future release.
`HIMPORT PREPARE` registers an ordered list of field names (a *fieldset*)
inside a connection's session; `HIMPORT SET` then creates hashes by sending
only the values, paired positionally with the prepared fields. The server
stores such hashes in a memory-efficient template encoding where the shared
field names are kept only once.
Fieldsets are scoped to a single physical connection on the server. go-redis
hides that from pooled callers: fieldsets registered with `HImportPrepare`
are remembered client-side and the `PREPARE` is replayed lazily — at most
once per connection session — on whichever pooled connection executes an
`HImportSet`, including inside pipelines and transactions. A connection that
lost its session state (`RESET`, reconnect) is re-prepared transparently.
These guarantees apply only to the typed methods (`HImportPrepare`,
`HImportSet`, `HImportDiscard`, `HImportDiscardAll`). Any HIMPORT command
sent through generic command interfaces — `Do`, raw `Cmd` values, or similar
low-level access — bypasses the client-side registry entirely: nothing is
replayed onto other connections, `no such fieldset` is not recovered, and
discards do not propagate. Such commands act on the session of whichever
connection happens to execute them, so they are only meaningful on a
dedicated connection (`client.Conn()`) managed explicitly by the caller.
The example walks through:
- basic prepare/set and reading the result back with regular hash commands
- bulk ingestion through pipelines, timed against plain `HSET` pipelines
- 100 concurrent goroutines writing through the pool with no explicit
connection management
- the observable server side: `OBJECT ENCODING` template encodings and the
`INFO memory` template counters
- discarding a fieldset and the resulting server error for further sets
## Run
Requires a Redis 8.10+ server built with the hinted hash templates feature
(the example exits with a message otherwise). The address defaults to
`localhost:6379` and can be overridden with `REDIS_ADDR`:
```sh
go run .
# or
REDIS_ADDR=localhost:6390 go run .
```
example/himport/go.mod
+12
View File
@@ -0,0 +1,12 @@
module github.com/redis/go-redis/example/himport
go 1.24
replace github.com/redis/go-redis/v9 => ../..
require github.com/redis/go-redis/v9 v9.21.0
require (
github.com/cespare/xxhash/v2 v2.3.0 // indirect
go.uber.org/atomic v1.11.0 // indirect
)
example/himport/go.sum
+20
View File
@@ -0,0 +1,20 @@
github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs=
github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c=
github.com/bsm/gomega v1.27.10 h1:yeMWxP2pV2fG3FgAODIY8EiRE3dy0aeFYt4l7wh6yKA=
github.com/bsm/gomega v1.27.10/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0H+O0=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/klauspost/cpuid/v2 v2.2.10 h1:tBs3QSyvjDyFTq3uoc/9xFpCuOsJQFNPiAhYdw2skhE=
github.com/klauspost/cpuid/v2 v2.2.10/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0Q=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/zeebo/xxh3 v1.1.0 h1:s7DLGDK45Dyfg7++yxI0khrfwq9661w9EN78eP/UZVs=
github.com/zeebo/xxh3 v1.1.0/go.mod h1:IisAie1LELR4xhVinxWS5+zf1lA4p0MW4T+w+W07F5s=
go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE=
go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=
golang.org/x/sys v0.30.0 h1:QjkSwP/36a20jFYWkSue1YwXzLmsV5Gfq7Eiy72C1uc=
golang.org/x/sys v0.30.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
example/himport/main.go
+196
View File
@@ -0,0 +1,196 @@
// Example: fast hash ingestion with the HIMPORT command family (Redis 8.10+).
//
// HIMPORT PREPARE registers a fieldset (an ordered list of hash field names)
// once per connection session; HIMPORT SET then creates hashes by sending
// only the values. go-redis remembers fieldsets registered through
// HImportPrepare and lazily replays the PREPARE on whichever pooled
// connection runs an HImportSet, so the API works transparently on the
// pooled client, pipelines, and transactions.
package main
import (
"context"
"fmt"
"log"
"os"
"strings"
"sync"
"time"
"github.com/redis/go-redis/v9"
)
const (
fieldsetName = "user-fields"
bulkKeys = 10_000
batchSize = 500
)
func main() {
ctx := context.Background()
addr := os.Getenv("REDIS_ADDR")
if addr == "" {
addr = "localhost:6379"
}
rdb := redis.NewClient(&redis.Options{
Addr: addr,
PoolSize: 8,
})
defer rdb.Close()
// HIMPORT requires Redis 8.10+ built with the hinted hash templates
// feature; bail out gracefully on older servers.
if err := rdb.HImportDiscardAll(ctx).Err(); err != nil {
if strings.Contains(strings.ToLower(err.Error()), "unknown command") {
log.Fatalf("this server does not support HIMPORT (requires Redis 8.10+): %v", err)
}
log.Fatalf("connection failed: %v", err)
}
basicUsage(ctx, rdb)
bulkIngestion(ctx, rdb)
concurrentWriters(ctx, rdb)
inspectEncoding(ctx, rdb)
discard(ctx, rdb)
}
// basicUsage registers a fieldset once and creates hashes from values only.
func basicUsage(ctx context.Context, rdb *redis.Client) {
fmt.Println("== basic usage ==")
// Register the field names once. The client remembers the fieldset and
// prepares it on every pooled connection that needs it.
if err := rdb.HImportPrepare(ctx, fieldsetName, "name", "email", "age").Err(); err != nil {
log.Fatalf("prepare: %v", err)
}
// Values map positionally to the prepared fields.
if err := rdb.HImportSet(ctx, "user:1", fieldsetName, "alice", "alice@example.com", 25).Err(); err != nil {
log.Fatalf("set: %v", err)
}
if err := rdb.HImportSet(ctx, "user:2", fieldsetName, "bob", "bob@example.com", 30).Err(); err != nil {
log.Fatalf("set: %v", err)
}
// Keys created through HIMPORT SET are regular hashes.
user, err := rdb.HGetAll(ctx, "user:1").Result()
if err != nil {
log.Fatalf("hgetall: %v", err)
}
fmt.Printf("user:1 = %v\n\n", user)
}
// bulkIngestion loads many hashes through pipelines — the intended
// high-throughput pattern — and compares it with plain HSET pipelines.
func bulkIngestion(ctx context.Context, rdb *redis.Client) {
fmt.Println("== bulk ingestion: HIMPORT SET vs HSET ==")
start := time.Now()
for base := 0; base < bulkKeys; base += batchSize {
pipe := rdb.Pipeline()
for i := base; i < base+batchSize && i < bulkKeys; i++ {
key := fmt.Sprintf("himport:user:%d", i)
name := fmt.Sprintf("user-%d", i)
email := fmt.Sprintf("user-%d@example.com", i)
pipe.HImportSet(ctx, key, fieldsetName, name, email, 20+i%50)
}
if _, err := pipe.Exec(ctx); err != nil {
log.Fatalf("himport pipeline: %v", err)
}
}
himportTook := time.Since(start)
start = time.Now()
for base := 0; base < bulkKeys; base += batchSize {
pipe := rdb.Pipeline()
for i := base; i < base+batchSize && i < bulkKeys; i++ {
key := fmt.Sprintf("hset:user:%d", i)
name := fmt.Sprintf("user-%d", i)
email := fmt.Sprintf("user-%d@example.com", i)
pipe.HSet(ctx, key, "name", name, "email", email, "age", 20+i%50)
}
if _, err := pipe.Exec(ctx); err != nil {
log.Fatalf("hset pipeline: %v", err)
}
}
hsetTook := time.Since(start)
fmt.Printf("%d hashes via HIMPORT SET: %v\n", bulkKeys, himportTook)
fmt.Printf("%d hashes via HSET: %v\n\n", bulkKeys, hsetTook)
}
// concurrentWriters shows that the pooled client needs no connection
// management: each pooled connection is prepared lazily, at most once.
func concurrentWriters(ctx context.Context, rdb *redis.Client) {
fmt.Println("== concurrent pooled writers ==")
var wg sync.WaitGroup
errCh := make(chan error, 100)
for w := 0; w < 100; w++ {
wg.Add(1)
go func(w int) {
defer wg.Done()
key := fmt.Sprintf("concurrent:user:%d", w)
name := fmt.Sprintf("worker-%d", w)
email := fmt.Sprintf("worker-%d@example.com", w)
if err := rdb.HImportSet(ctx, key, fieldsetName, name, email, w).Err(); err != nil {
errCh <- err
}
}(w)
}
wg.Wait()
close(errCh)
for err := range errCh {
log.Fatalf("concurrent set: %v", err)
}
fmt.Printf("100 goroutines wrote through the pool without explicit PREPARE calls\n\n")
}
// inspectEncoding surfaces the server-side effect: hashes created through a
// shared template report a template encoding, and INFO tracks template use.
func inspectEncoding(ctx context.Context, rdb *redis.Client) {
fmt.Println("== server-side encoding ==")
encoding, err := rdb.ObjectEncoding(ctx, "himport:user:0").Result()
if err != nil {
log.Fatalf("object encoding: %v", err)
}
fmt.Printf("OBJECT ENCODING himport:user:0 = %s\n", encoding)
info, err := rdb.Info(ctx, "memory").Result()
if err != nil {
log.Fatalf("info: %v", err)
}
for _, line := range strings.Split(info, "\r\n") {
if strings.HasPrefix(line, "used_memory_hash_templates") ||
strings.HasPrefix(line, "hash_templates") ||
strings.HasPrefix(line, "hash_template_keys") {
fmt.Println(line)
}
}
fmt.Println()
}
// discard removes the fieldset; hashes created through it are unaffected.
func discard(ctx context.Context, rdb *redis.Client) {
fmt.Println("== discard ==")
removed, err := rdb.HImportDiscard(ctx, fieldsetName).Result()
if err != nil {
log.Fatalf("discard: %v", err)
}
fmt.Printf("discarded fieldset (removed=%d)\n", removed)
// The fieldset is gone from the client registry and this connection's
// session: further sets surface the server error unchanged.
err = rdb.HImportSet(ctx, "user:3", fieldsetName, "carol", "carol@example.com", 35).Err()
fmt.Printf("set after discard: %v\n", err)
// Existing hashes survive.
name, err := rdb.HGet(ctx, "user:1", "name").Result()
if err != nil {
log.Fatalf("hget: %v", err)
}
fmt.Printf("user:1 name still readable: %s\n", name)
}
hash_commands.go
+5
View File
@@ -44,6 +44,11 @@ type HashCmdable interface {
HPExpireTime(ctx context.Context, key string, fields ...string) *IntSliceCmd
HTTL(ctx context.Context, key string, fields ...string) *IntSliceCmd
HPTTL(ctx context.Context, key string, fields ...string) *IntSliceCmd
// note: the HIMPORT API is experimental and may be subject to change.
HImportPrepare(ctx context.Context, fieldsetName string, fields ...string) *StatusCmd
HImportSet(ctx context.Context, key, fieldsetName string, values ...interface{}) *StatusCmd
HImportDiscard(ctx context.Context, fieldsetName string) *IntCmd
HImportDiscardAll(ctx context.Context) *IntCmd
}
func (c cmdable) HDel(ctx context.Context, key string, fields ...string) *IntCmd {
himport.go
+468
View File
@@ -0,0 +1,468 @@
package redis
import (
"context"
"strings"
"sync"
"github.com/redis/go-redis/v9/internal"
"github.com/redis/go-redis/v9/internal/pool"
"github.com/redis/go-redis/v9/internal/proto"
)
// himportFieldset is the client-side record of a fieldset registered with
// HImportPrepare.
type himportFieldset struct {
fields []string
version uint64
}
// himportRegistry remembers fieldsets registered through a client so HIMPORT
// SET can lazily prepare them on whichever pooled connection it executes.
// Versions increase monotonically and start at 1; re-registering a name under
// a new version invalidates every connection's prepared flag for it, so a
// replaced fieldset is re-prepared before its next use.
//
// Discards propagate lazily as well: a discarded name is kept as a tombstone
// and the discard-all counter as an epoch, and connections whose sessions
// still hold discarded fieldsets replay HIMPORT DISCARD/DISCARDALL before
// their next HIMPORT command (see baseClient.himportInjectedCmds).
type himportRegistry struct {
mu sync.RWMutex
nextVersion uint64
fieldsets map[string]himportFieldset
// tombstones holds names discarded through this client whose server-side
// copies may survive on pooled connections that prepared them. An entry
// is removed when the name is registered again (the new version replaces
// the fieldset on the server, so no discard is needed) or by discardAll.
// Known limitation: a workload discarding many uniquely-named fieldsets
// grows this map for the client's lifetime and pays an O(tombstones)
// snapshot per HIMPORT round trip; HImportDiscardAll resets it.
tombstones map[string]struct{}
// discardAllEpoch increments on every successful HImportDiscardAll.
discardAllEpoch uint64
}
func newHImportRegistry() *himportRegistry {
return &himportRegistry{}
}
// register stores the fieldset and returns its new version together with the
// current discard-all epoch.
func (r *himportRegistry) register(name string, fields []string) (version, epoch uint64) {
r.mu.Lock()
defer r.mu.Unlock()
if r.fieldsets == nil {
r.fieldsets = make(map[string]himportFieldset)
}
delete(r.tombstones, name)
r.nextVersion++
r.fieldsets[name] = himportFieldset{
fields: append([]string(nil), fields...),
version: r.nextVersion,
}
return r.nextVersion, r.discardAllEpoch
}
func (r *himportRegistry) lookup(name string) (himportFieldset, bool) {
if r == nil {
return himportFieldset{}, false
}
r.mu.RLock()
fs, ok := r.fieldsets[name]
r.mu.RUnlock()
return fs, ok
}
// discard removes the fieldset and leaves a tombstone so connections whose
// sessions still hold it replay the DISCARD before their next HIMPORT
// command. It reports whether the fieldset was registered.
func (r *himportRegistry) discard(name string) bool {
r.mu.Lock()
defer r.mu.Unlock()
if _, ok := r.fieldsets[name]; !ok {
return false
}
delete(r.fieldsets, name)
if r.tombstones == nil {
r.tombstones = make(map[string]struct{})
}
r.tombstones[name] = struct{}{}
return true
}
// discardAll drops every fieldset and tombstone and moves to a new epoch;
// connections prepared under an older epoch replay HIMPORT DISCARDALL before
// their next HIMPORT command. It returns the new epoch and the number of
// fieldsets that were registered.
func (r *himportRegistry) discardAll() (epoch uint64, removed int) {
r.mu.Lock()
removed = len(r.fieldsets)
r.fieldsets = nil
r.tombstones = nil
r.discardAllEpoch++
epoch = r.discardAllEpoch
r.mu.Unlock()
return epoch, removed
}
// discardVersion withdraws a registration whose fan-out PREPARE was rejected
// by a server — but only while the entry is still at that version, so a
// concurrent re-registration is not clobbered. A tombstone is left: the
// fan-out may have succeeded on some masters before another rejected it
// (per-node ACLs, rolling upgrades), and those sessions hold the withdrawn
// fieldset; the tombstone makes their next HIMPORT command discard it
// instead of leaving a fieldset the client can no longer address.
func (r *himportRegistry) discardVersion(name string, version uint64) {
r.mu.Lock()
if fs, ok := r.fieldsets[name]; ok && fs.version == version {
delete(r.fieldsets, name)
if r.tombstones == nil {
r.tombstones = make(map[string]struct{})
}
r.tombstones[name] = struct{}{}
}
r.mu.Unlock()
}
// refreshVersion bumps a registered fieldset to a new version, keeping its
// fields — but only while the entry is still at the given version, so a
// concurrent re-registration is not disturbed. Every connection's prepared
// flag becomes stale, forcing a re-prepare before the fieldset's next use on
// each of them. Used when a "no such fieldset" reply signals session loss
// that may have hit more connections than the one that reported it (failover,
// cross-region switch, reset storms).
func (r *himportRegistry) refreshVersion(name string, version uint64) {
r.mu.Lock()
if fs, ok := r.fieldsets[name]; ok && fs.version == version {
r.nextVersion++
fs.version = r.nextVersion
r.fieldsets[name] = fs
}
r.mu.Unlock()
}
// idle reports whether the registry implies no injection work at all: no
// fieldsets to replay, no tombstones to discard, and no discard-all epoch a
// session could be behind.
func (r *himportRegistry) idle() bool {
if r == nil {
return true
}
r.mu.RLock()
idle := len(r.fieldsets) == 0 && len(r.tombstones) == 0 && r.discardAllEpoch == 0
r.mu.RUnlock()
return idle
}
// cleanupSnapshot returns the current epoch and the tombstoned names.
func (r *himportRegistry) cleanupSnapshot() (epoch uint64, tombstones []string) {
r.mu.RLock()
epoch = r.discardAllEpoch
if len(r.tombstones) > 0 {
tombstones = make([]string, 0, len(r.tombstones))
for name := range r.tombstones {
tombstones = append(tombstones, name)
}
}
r.mu.RUnlock()
return epoch, tombstones
}
// himportNoSuchFieldset reports whether err is the server's "no such
// fieldset" reply, i.e. an HIMPORT SET executed on a connection whose session
// does not hold the referenced fieldset.
func himportNoSuchFieldset(err error) bool {
return isRedisError(err) && strings.Contains(err.Error(), "no such fieldset")
}
// himportInjectedCmds returns the HIMPORT commands to write to cn ahead of a
// batch, in order:
//
// 1. HIMPORT DISCARDALL when cn's session was prepared under an older
// discard-all epoch;
// 2. HIMPORT DISCARD for each discarded fieldset the session still holds;
// 3. HIMPORT PREPARE for each registered fieldset referenced by an HIMPORT
// SET in the batch that the session lacks at the current version.
//
// A fieldset covered by a user-issued PREPARE earlier in the batch needs no
// injection — the server session holds it by the time the SET runs. Returns
// nil when the batch contains no HIMPORT commands: sessions holding only
// discarded fieldsets are cleaned up on their next HIMPORT use, not on
// unrelated traffic.
func (c *baseClient) himportInjectedCmds(ctx context.Context, cn *pool.Conn, cmds []Cmder) []Cmder {
if c.himport.idle() {
return nil
}
hasHImport := false
for _, cmd := range cmds {
if _, ok := cmd.(himportCmder); ok {
hasHImport = true
break
}
}
if !hasHImport {
return nil
}
var injected []Cmder
// Discards first: a session behind the discard-all epoch is wiped
// entirely; otherwise individual tombstoned fieldsets it still holds are
// discarded.
epoch, tombstones := c.himport.cleanupSnapshot()
sessionWiped := false
if cn.HasPreparedFieldsets() && cn.FieldsetEpoch() != epoch {
da := NewHImportDiscardAllCmd(ctx)
da.registryEpoch = epoch
injected = append(injected, da)
sessionWiped = true
} else {
for _, name := range tombstones {
if cn.FieldsetPreparedVersion(name) != 0 {
injected = append(injected, NewHImportDiscardCmd(ctx, name))
}
}
}
// Prepares for registered fieldsets the batch's SETs reference.
var covered map[string]struct{}
cover := func(name string) {
if covered == nil {
covered = make(map[string]struct{})
}
covered[name] = struct{}{}
}
for _, cmd := range cmds {
switch hc := cmd.(type) {
case *HImportPrepareCmd:
cover(hc.fieldsetName)
case *HImportSetCmd:
if _, ok := covered[hc.fieldsetName]; ok {
continue
}
fs, ok := c.himport.lookup(hc.fieldsetName)
if !ok {
continue
}
if !sessionWiped && cn.FieldsetPreparedVersion(hc.fieldsetName) == fs.version {
continue
}
// The session holds an older version. Discard it before the
// re-prepare: the SET behind it is already on the wire, and if
// the re-prepare fails the SET must answer "no such fieldset"
// rather than silently writing the old version's field names.
if !sessionWiped && cn.FieldsetPreparedVersion(hc.fieldsetName) != 0 {
injected = append(injected, NewHImportDiscardCmd(ctx, hc.fieldsetName))
}
prep := NewHImportPrepareCmd(ctx, hc.fieldsetName, fs.fields...)
prep.registryVersion = fs.version
prep.registryEpoch = epoch
injected = append(injected, prep)
cover(hc.fieldsetName)
}
}
return injected
}
// himportReadInjectedReplies consumes the replies of injected HIMPORT
// commands. Server errors are recorded on the command and the connection is
// left readable; transport errors are returned. Successful commands apply
// their prepared-flag bookkeeping on cn.
func (c *baseClient) himportReadInjectedReplies(ctx context.Context, cn *pool.Conn, rd *proto.Reader, injected []Cmder) error {
for _, cmd := range injected {
if err := c.processPendingPushNotificationWithReader(ctx, cn, rd); err != nil {
internal.Logger.Printf(ctx, "push: error processing pending notifications before reading reply: %v", err)
}
err := cmd.readReply(rd)
cmd.SetErr(err)
if err != nil {
if !isRedisError(err) {
return err
}
// A failed injected PREPARE becomes the root cause of the
// dependent SETs' errors downstream; a failed injected discard
// only delays cleanup until the next HIMPORT command.
internal.Logger.Printf(ctx, "himport: injected %s failed: %v", cmd.Name(), err)
continue
}
switch hc := cmd.(type) {
case *HImportPrepareCmd:
cn.MarkFieldsetPrepared(hc.fieldsetName, hc.registryVersion, hc.registryEpoch)
case *HImportDiscardCmd:
cn.UnmarkFieldsetPrepared(hc.fieldsetName)
case *HImportDiscardAllCmd:
cn.ClearPreparedFieldsets(hc.registryEpoch)
}
}
return nil
}
// himportAfterCmd applies registry and prepared-flag updates after a
// user-issued HIMPORT command completed successfully on cn.
func (c *baseClient) himportAfterCmd(cn *pool.Conn, hc himportCmder) {
if c.himport == nil {
return
}
switch cmd := hc.(type) {
case *HImportPrepareCmd:
version, epoch := cmd.registryVersion, cmd.registryEpoch
if version == 0 {
version, epoch = c.himport.register(cmd.fieldsetName, cmd.fields)
}
// A pre-assigned version marks a fan-out copy: the fieldset was
// registered once at the cluster/ring level; only mark the
// executing connection.
cn.MarkFieldsetPrepared(cmd.fieldsetName, version, epoch)
case *HImportDiscardCmd:
registered := c.himport.discard(cmd.fieldsetName)
cn.UnmarkFieldsetPrepared(cmd.fieldsetName)
// The managed API reports the registry lifecycle: 1 when the
// fieldset was registered on this client and is now removed. The
// executing connection's session count stands only for fieldsets
// the registry never knew (raw usage).
if registered {
cmd.SetVal(1)
}
case *HImportDiscardAllCmd:
// A pre-assigned epoch marks a fan-out copy: the registry was
// already wiped at the cluster/ring level; only move the executing
// connection to that epoch.
if cmd.registryEpoch != 0 {
cn.ClearPreparedFieldsets(cmd.registryEpoch)
return
}
epoch, removed := c.himport.discardAll()
cn.ClearPreparedFieldsets(epoch)
// Same registry semantics: report how many registered fieldsets
// were removed, not how many the executing session happened to
// hold.
if removed > 0 {
cmd.SetVal(int64(removed))
}
}
}
// himportAfterBatch runs after all replies of a batch were read: it surfaces
// an injected PREPARE failure as the root cause on the HIMPORT SET commands
// that depended on it (their own reply is the secondary "no such fieldset"
// error), invalidates stale prepared flags for SETs that found their
// registered fieldset missing server-side, and applies registry updates for
// user-issued HIMPORT commands that succeeded in the batch.
func (c *baseClient) himportAfterBatch(cn *pool.Conn, injected []Cmder, cmds []Cmder) {
var failed map[string]error
var refreshed map[string]struct{}
for _, cmd := range injected {
if prep, ok := cmd.(*HImportPrepareCmd); ok {
if err := prep.Err(); err != nil {
if failed == nil {
failed = make(map[string]error)
}
failed[prep.fieldsetName] = err
}
}
}
for _, cmd := range cmds {
hc, ok := cmd.(himportCmder)
if !ok {
continue
}
if set, ok := hc.(*HImportSetCmd); ok {
if rootCause, ok := failed[set.fieldsetName]; ok && himportNoSuchFieldset(set.Err()) {
set.SetErr(rootCause)
continue
}
// The session lost a fieldset the flags claim is prepared (e.g.
// RESET) — and the same event may have wiped other sessions
// whose flags also still look current. Bump the fieldset
// version once so the SET's re-issue, the cluster re-queue on
// whichever connection it lands, or the caller's transaction
// retry replays the PREPARE.
if himportNoSuchFieldset(set.Err()) {
if _, done := refreshed[set.fieldsetName]; !done {
if refreshed == nil {
refreshed = make(map[string]struct{})
}
refreshed[set.fieldsetName] = struct{}{}
if fs, registered := c.himport.lookup(set.fieldsetName); registered {
c.himport.refreshVersion(set.fieldsetName, fs.version)
}
}
}
continue
}
if hc.Err() == nil {
c.himportAfterCmd(cn, hc)
}
}
}
// himportRetryFailedSets re-issues, once, the HIMPORT SET commands of a
// pipeline batch that failed with "no such fieldset" while their fieldset is
// registered — the error must not surface for managed fieldsets (NF.4). Only
// the SETs are re-sent: HIMPORT SET is a full replace, so re-execution is
// idempotent, and no other command of the batch runs again. Their prepared
// flags were invalidated by himportAfterBatch, so himportInjectedCmds
// regenerates the PREPAREs for this connection. Transport errors are
// returned; server errors stay recorded on the commands.
// (The retry does not carry an ASKING prefix. A redirected [ASKING, SET]
// pair whose injected PREPARE failed is excluded by the root-cause swap in
// himportAfterBatch; one that lost its session without an injection can be
// re-issued here, and the bare SET then draws a fresh MOVED/ASK that the
// outer cluster redirect handling resolves.)
func (c *baseClient) himportRetryFailedSets(ctx context.Context, cn *pool.Conn, cmds []Cmder) error {
if c.himport.idle() {
return nil
}
var retry []Cmder
for _, cmd := range cmds {
if set, ok := cmd.(*HImportSetCmd); ok && himportNoSuchFieldset(set.Err()) {
if _, registered := c.himport.lookup(set.fieldsetName); registered {
retry = append(retry, set)
}
}
}
if len(retry) == 0 {
return nil
}
injected := c.himportInjectedCmds(ctx, cn, retry)
if err := cn.WithWriter(c.context(ctx), c.opt.WriteTimeout, func(wr *proto.Writer) error {
for _, ic := range injected {
if err := writeCmd(wr, ic); err != nil {
return err
}
}
return writeCmds(wr, retry)
}); err != nil {
return err
}
return cn.WithReader(c.context(ctx), c.opt.ReadTimeout, func(rd *proto.Reader) error {
if err := c.himportReadInjectedReplies(ctx, cn, rd, injected); err != nil {
return err
}
err := c.pipelineReadCmds(ctx, cn, rd, retry)
if err != nil && !isRedisError(err) {
return err
}
// Server errors (including a repeated failure) stay on the
// individual commands; the batch as a whole is done.
c.himportAfterBatch(cn, injected, retry)
return nil
})
}
// himportShouldRetrySet reports whether a retry of cmd may succeed after it
// failed with "no such fieldset": true when the fieldset is registered
// client-side — the executing connection lost its server session state (for
// example a RESET, or a concurrent discard). The connection's prepared flag
// was already invalidated inside _process, while that goroutine still owned
// the connection, so the retry re-prepares lazily wherever it lands.
func (c *baseClient) himportShouldRetrySet(cmd Cmder, err error) bool {
set, ok := cmd.(*HImportSetCmd)
if !ok || !himportNoSuchFieldset(err) {
return false
}
_, registered := c.himport.lookup(set.fieldsetName)
return registered
}
himport_cluster.go
+204
View File
@@ -0,0 +1,204 @@
package redis
import "context"
// Cluster and ring support for the HIMPORT command family.
//
// Correctness comes from the shared registry: every node/shard client holds
// the same himportRegistry (wired at client construction), so any connection
// executing an HIMPORT SET lazily replays the PREPARE, MOVED/ASK redirects
// re-prepare on the target node, and discards propagate through tombstones
// and the discard-all epoch. Replicas share the registry too — roles change
// with the topology, and a promoted replica's connections simply carry no
// prepared flags, so their first SET self-prepares.
//
// On top of that, user-issued PREPARE/DISCARD/DISCARDALL fan out eagerly to
// all masters (R.4): one connection per master is prepared/cleaned up front,
// server-side validation surfaces immediately, and leftover session state is
// bounded. The fan-out is best-effort — any connection it does not reach is
// covered by the lazy replay.
// The fan-out helpers execute the per-node copies through each node client's
// Process, so node-level hooks observe them; the cluster/ring-level
// ProcessHook chain sees only the user's command object, not the fan-out.
//
// Known limitation: an HImportPrepare pipelined together with HImportSets of
// the same new fieldset in one ClusterClient Exec is not ordered across
// nodes — per-node sub-batches run concurrently, and the registration
// happens when the PREPARE's node completes, so SETs routed to other nodes
// can race it and fail with "no such fieldset". Register the fieldset with
// the client-level HImportPrepare before pipelining (the HLD's back-to-back
// PREPARE+SET pattern is a single-connection guarantee).
// himportForEach runs fn on a set of clients (all cluster masters, or all
// ring shards).
type himportForEach func(ctx context.Context, fn func(ctx context.Context, client *Client) error) error
// himportRequeueFailedSets re-queues HIMPORT SETs of registered fieldsets
// that failed with "no such fieldset" — their stale prepared flags were just
// invalidated by himportAfterBatch, so the next pipeline attempt re-prepares
// lazily and re-executes only those SETs (a full replace, so idempotent).
// Bounded by the cluster pipeline's attempt budget.
func (c *ClusterClient) himportRequeueFailedSets(ctx context.Context, cmds []Cmder, failedCmds *cmdsMap) {
for _, cmd := range cmds {
if set, ok := cmd.(*HImportSetCmd); ok && himportNoSuchFieldset(set.Err()) {
if _, registered := c.himport.lookup(set.fieldsetName); registered {
_ = c.mapCmdsByNode(ctx, failedCmds, []Cmder{set})
}
}
}
}
// himportFanOutPrepare registers the fieldset once in the shared registry
// and executes a pre-versioned PREPARE copy on every client; each copy marks
// its executing connection without registering again. A deterministic server
// rejection (e.g. duplicate field name) withdraws the registration; a
// transport failure keeps it, and lazy replay covers the connections the
// fan-out missed (all-succeeded semantics: the first error is reported).
func himportFanOutPrepare(ctx context.Context, registry *himportRegistry, forEach himportForEach, cmd *HImportPrepareCmd) {
version, epoch := registry.register(cmd.fieldsetName, cmd.fields)
err := forEach(ctx, func(ctx context.Context, client *Client) error {
fanCmd := NewHImportPrepareCmd(ctx, cmd.fieldsetName, cmd.fields...)
fanCmd.registryVersion = version
fanCmd.registryEpoch = epoch
return client.Process(ctx, fanCmd)
})
if err != nil {
if isRedisError(err) {
// Withdraw the registration; the tombstone cleans the sessions
// on which the fan-out succeeded before the rejection.
registry.discardVersion(cmd.fieldsetName, version)
}
cmd.SetErr(err)
return
}
cmd.SetVal("OK")
}
// himportFanOutDiscard removes the fieldset from the shared registry
// (leaving the tombstone that lazily cleans the connections the fan-out does
// not reach) and discards it on one connection of every client.
func himportFanOutDiscard(ctx context.Context, registry *himportRegistry, forEach himportForEach, cmd *HImportDiscardCmd) {
registered := registry.discard(cmd.fieldsetName)
err := forEach(ctx, func(ctx context.Context, client *Client) error {
return client.Process(ctx, NewHImportDiscardCmd(ctx, cmd.fieldsetName))
})
if err != nil {
cmd.SetErr(err)
return
}
if registered {
cmd.SetVal(1)
} else {
cmd.SetVal(0)
}
}
// himportFanOutDiscardAll wipes the shared registry once and executes a
// pre-epoch DISCARDALL copy on every client; each copy moves its executing
// connection to the new epoch without bumping the registry again.
func himportFanOutDiscardAll(ctx context.Context, registry *himportRegistry, forEach himportForEach, cmd *HImportDiscardAllCmd) {
epoch, removed := registry.discardAll()
err := forEach(ctx, func(ctx context.Context, client *Client) error {
fanCmd := NewHImportDiscardAllCmd(ctx)
fanCmd.registryEpoch = epoch
return client.Process(ctx, fanCmd)
})
if err != nil {
cmd.SetErr(err)
return
}
cmd.SetVal(int64(removed))
}
// HImportPrepare registers the fieldset in the cluster-wide registry and
// eagerly prepares one connection on every master; all other connections —
// including those of replicas promoted later and masters added by
// resharding — are prepared lazily before their first HImportSet. See
// HashCmdable.HImportPrepare (cmdable) for the fieldset semantics.
//
// The fan-out is best-effort and reports the first error: on a server
// rejection (e.g. duplicate field name) the registration is withdrawn and
// any sessions the fan-out already prepared are cleaned lazily; on a
// transport failure the registration is kept and lazy replay covers the
// connections the fan-out missed.
//
// Requires Redis 8.10 or newer.
//
// note: the API is experimental and may be subject to change.
func (c *ClusterClient) HImportPrepare(ctx context.Context, fieldsetName string, fields ...string) *StatusCmd {
cmd := NewHImportPrepareCmd(ctx, fieldsetName, fields...)
himportFanOutPrepare(ctx, c.himport, c.ForEachMaster, cmd)
return &cmd.StatusCmd
}
// HImportDiscard removes the fieldset from the cluster-wide registry and
// discards it on every master; connections the fan-out does not reach
// replay the discard before their next HIMPORT command. It returns 1 if the
// fieldset was registered on this client and is now removed.
//
// Requires Redis 8.10 or newer.
//
// note: the API is experimental and may be subject to change.
func (c *ClusterClient) HImportDiscard(ctx context.Context, fieldsetName string) *IntCmd {
cmd := NewHImportDiscardCmd(ctx, fieldsetName)
himportFanOutDiscard(ctx, c.himport, c.ForEachMaster, cmd)
return &cmd.IntCmd
}
// HImportDiscardAll removes all fieldsets from the cluster-wide registry and
// wipes them on every master; connections the fan-out does not reach replay
// the wipe before their next HIMPORT command. It returns the number of
// fieldsets removed from the registry.
//
// Requires Redis 8.10 or newer.
//
// note: the API is experimental and may be subject to change.
func (c *ClusterClient) HImportDiscardAll(ctx context.Context) *IntCmd {
cmd := NewHImportDiscardAllCmd(ctx)
himportFanOutDiscardAll(ctx, c.himport, c.ForEachMaster, cmd)
return &cmd.IntCmd
}
// HImportPrepare registers the fieldset in the ring-wide registry and
// eagerly prepares one connection on every shard; all other connections are
// prepared lazily before their first HImportSet. The fan-out is best-effort
// with the same failure semantics as ClusterClient.HImportPrepare. See
// HashCmdable.HImportPrepare (cmdable) for the fieldset semantics.
//
// Requires Redis 8.10 or newer.
//
// note: the API is experimental and may be subject to change.
func (c *Ring) HImportPrepare(ctx context.Context, fieldsetName string, fields ...string) *StatusCmd {
cmd := NewHImportPrepareCmd(ctx, fieldsetName, fields...)
himportFanOutPrepare(ctx, c.opt.himport, c.ForEachShard, cmd)
return &cmd.StatusCmd
}
// HImportDiscard removes the fieldset from the ring-wide registry and
// discards it on every shard; connections the fan-out does not reach replay
// the discard before their next HIMPORT command. It returns 1 if the
// fieldset was registered on this client and is now removed.
//
// Requires Redis 8.10 or newer.
//
// note: the API is experimental and may be subject to change.
func (c *Ring) HImportDiscard(ctx context.Context, fieldsetName string) *IntCmd {
cmd := NewHImportDiscardCmd(ctx, fieldsetName)
himportFanOutDiscard(ctx, c.opt.himport, c.ForEachShard, cmd)
return &cmd.IntCmd
}
// HImportDiscardAll removes all fieldsets from the ring-wide registry and
// wipes them on every shard; connections the fan-out does not reach replay
// the wipe before their next HIMPORT command. It returns the number of
// fieldsets removed from the registry.
//
// Requires Redis 8.10 or newer.
//
// note: the API is experimental and may be subject to change.
func (c *Ring) HImportDiscardAll(ctx context.Context) *IntCmd {
cmd := NewHImportDiscardAllCmd(ctx)
himportFanOutDiscardAll(ctx, c.opt.himport, c.ForEachShard, cmd)
return &cmd.IntCmd
}
himport_commands.go
+253
View File
@@ -0,0 +1,253 @@
package redis
import "context"
// The HIMPORT command family (Redis 8.10+, "hinted hash templates") provides
// fast ingestion of many hashes sharing the same field names. HIMPORT PREPARE
// registers the field names once under a fieldset name, then HIMPORT SET
// creates hashes by sending only the values.
//
// The server scopes a fieldset to the physical connection that prepared it.
// Because go-redis pools connections, the client additionally keeps a
// client-side registry of fieldsets registered through HImportPrepare and
// lazily replays the PREPARE (at most once per connection session) on any
// pooled connection about to execute an HImportSet that references it. See
// himport.go.
//
// The whole HIMPORT surface — the typed methods, the HImport*Cmd types and
// their constructors — is experimental and may be subject to change.
// himportCmder marks HIMPORT commands that participate in client-side
// fieldset tracking. Process paths do a single interface assertion on the
// hot path and inspect the concrete type only for HIMPORT commands.
type himportCmder interface {
Cmder
himportCmd()
}
var (
_ himportCmder = (*HImportPrepareCmd)(nil)
_ himportCmder = (*HImportSetCmd)(nil)
_ himportCmder = (*HImportDiscardCmd)(nil)
_ himportCmder = (*HImportDiscardAllCmd)(nil)
)
// HImportPrepareCmd represents an HIMPORT PREPARE command.
type HImportPrepareCmd struct {
StatusCmd
fieldsetName string
fields []string
// registryVersion and registryEpoch are set only on commands injected by
// the client to replay a registered fieldset onto a connection; on
// success the connection is marked as prepared at this version under
// this discard-all epoch.
registryVersion uint64
registryEpoch uint64
}
func (cmd *HImportPrepareCmd) himportCmd() {}
// NewHImportPrepareCmd returns an HIMPORT PREPARE command.
func NewHImportPrepareCmd(ctx context.Context, fieldsetName string, fields ...string) *HImportPrepareCmd {
args := make([]interface{}, 3+len(fields))
args[0] = "himport"
args[1] = "prepare"
args[2] = fieldsetName
for i, field := range fields {
args[3+i] = field
}
return &HImportPrepareCmd{
StatusCmd: StatusCmd{
baseCmd: baseCmd{
ctx: ctx,
args: args,
cmdType: CmdTypeStatus,
},
},
fieldsetName: fieldsetName,
fields: append([]string(nil), fields...),
}
}
// HImportSetCmd represents an HIMPORT SET command.
type HImportSetCmd struct {
StatusCmd
fieldsetName string
}
func (cmd *HImportSetCmd) himportCmd() {}
// NewHImportSetCmd returns an HIMPORT SET command.
func NewHImportSetCmd(ctx context.Context, key, fieldsetName string, values ...interface{}) *HImportSetCmd {
args := make([]interface{}, 4+len(values))
args[0] = "himport"
args[1] = "set"
args[2] = key
args[3] = fieldsetName
copy(args[4:], values)
cmd := &HImportSetCmd{
StatusCmd: StatusCmd{
baseCmd: baseCmd{
ctx: ctx,
args: args,
cmdType: CmdTypeStatus,
},
},
fieldsetName: fieldsetName,
}
cmd.SetFirstKeyPos(2)
return cmd
}
// HImportDiscardCmd represents an HIMPORT DISCARD command.
type HImportDiscardCmd struct {
IntCmd
fieldsetName string
}
func (cmd *HImportDiscardCmd) himportCmd() {}
// NewHImportDiscardCmd returns an HIMPORT DISCARD command.
func NewHImportDiscardCmd(ctx context.Context, fieldsetName string) *HImportDiscardCmd {
return &HImportDiscardCmd{
IntCmd: IntCmd{
baseCmd: baseCmd{
ctx: ctx,
args: []interface{}{"himport", "discard", fieldsetName},
cmdType: CmdTypeInt,
},
},
fieldsetName: fieldsetName,
}
}
// HImportDiscardAllCmd represents an HIMPORT DISCARDALL command.
type HImportDiscardAllCmd struct {
IntCmd
// registryEpoch is set only on commands injected by the client to wipe a
// session that predates the registry's discard-all epoch; on success the
// connection adopts this epoch.
registryEpoch uint64
}
func (cmd *HImportDiscardAllCmd) himportCmd() {}
// NewHImportDiscardAllCmd returns an HIMPORT DISCARDALL command.
func NewHImportDiscardAllCmd(ctx context.Context) *HImportDiscardAllCmd {
return &HImportDiscardAllCmd{
IntCmd: IntCmd{
baseCmd: baseCmd{
ctx: ctx,
args: []interface{}{"himport", "discardall"},
cmdType: CmdTypeInt,
},
},
}
}
// HImportPrepare registers an ordered list of hash field names under
// fieldsetName for use by subsequent HImportSet calls:
//
// HIMPORT PREPARE fieldset_name field [field ...]
//
// The server keeps the fieldset in the session of the connection that
// executed the command. On pooled clients (Client, Conn, Pipeline, Tx) the
// fieldset is also remembered client-side and the PREPARE is replayed
// lazily — at most once per connection session — on any pooled connection
// about to execute an HImportSet referencing it, so HImportSet works
// transparently across the pool. Preparing an existing fieldset name again
// silently replaces it.
//
// ClusterClient and Ring override this method (see himport_cluster.go): the
// fieldset registers in a registry shared by every node/shard client and the
// PREPARE additionally fans out eagerly to all masters/shards.
//
// Requires Redis 8.10 or newer.
//
// note: the API is experimental and may be subject to change.
func (c cmdable) HImportPrepare(ctx context.Context, fieldsetName string, fields ...string) *StatusCmd {
cmd := NewHImportPrepareCmd(ctx, fieldsetName, fields...)
_ = c(ctx, cmd)
return &cmd.StatusCmd
}
// HImportSet creates or fully replaces the hash at key using the field list
// registered under fieldsetName, pairing values positionally with the
// prepared fields:
//
// HIMPORT SET key fieldset_name value [value ...]
//
// The number of values must equal the fieldset's field count. The resulting
// key is a regular hash readable and writable by all hash commands. If the
// fieldset was registered through HImportPrepare on this client, it is
// prepared automatically on whichever pooled connection executes the command;
// otherwise the fieldset must have been prepared on the executing connection
// or the server replies "ERR no such fieldset".
//
// "no such fieldset" never surfaces for a registered fieldset: a
// single-command HImportSet whose connection lost its session state (e.g.
// RESET) is transparently re-prepared and retried once — the failure also
// stales every other connection's prepared flag, so the retry re-prepares
// wherever it lands, and this recovery attempt is granted even when retries
// are disabled (MaxRetries -1). In pipelines the failed HImportSets — and
// only those — are re-prepared and re-issued once on the same connection
// (HIMPORT SET is a full replace, so the re-execution is idempotent and no
// other command of the batch runs again). Inside transactions the error does
// surface after EXEC — an executed transaction cannot be partially re-run —
// but the prepared flags are invalidated, so retrying the transaction
// succeeds.
//
// Requires Redis 8.10 or newer.
//
// note: the API is experimental and may be subject to change.
func (c cmdable) HImportSet(ctx context.Context, key, fieldsetName string, values ...interface{}) *StatusCmd {
cmd := NewHImportSetCmd(ctx, key, fieldsetName, values...)
_ = c(ctx, cmd)
return &cmd.StatusCmd
}
// HImportDiscard removes fieldsetName from the executing connection's session
// and from the client-side registry, stopping further automatic replay:
//
// HIMPORT DISCARD fieldset_name
//
// It returns 1 if the fieldset was registered on this client and is now
// removed, 0 otherwise (for names never registered through the managed API,
// the executing connection's session reply passes through unchanged). Pooled
// connections whose sessions still hold the fieldset replay the DISCARD
// before their next HIMPORT command, so a subsequent HImportSet fails with
// "no such fieldset" on every connection, exactly as on a single connection.
// Hashes already created through the fieldset are not affected.
//
// Requires Redis 8.10 or newer.
//
// note: the API is experimental and may be subject to change.
func (c cmdable) HImportDiscard(ctx context.Context, fieldsetName string) *IntCmd {
cmd := NewHImportDiscardCmd(ctx, fieldsetName)
_ = c(ctx, cmd)
return &cmd.IntCmd
}
// HImportDiscardAll removes all fieldsets from the executing connection's
// session and clears the client-side registry:
//
// HIMPORT DISCARDALL
//
// It returns the number of fieldsets removed from the client-side registry
// (when none were registered, the executing connection's session count
// passes through). Other pooled connections whose sessions were prepared
// earlier replay HIMPORT DISCARDALL before their next HIMPORT command.
//
// Requires Redis 8.10 or newer.
//
// note: the API is experimental and may be subject to change.
func (c cmdable) HImportDiscardAll(ctx context.Context) *IntCmd {
cmd := NewHImportDiscardAllCmd(ctx)
_ = c(ctx, cmd)
return &cmd.IntCmd
}
himport_commands_test.go
+219
View File
@@ -0,0 +1,219 @@
package redis_test
import (
"context"
"fmt"
"strings"
"sync"
. "github.com/bsm/ginkgo/v2"
. "github.com/bsm/gomega"
"github.com/redis/go-redis/v9"
)
var _ = Describe("HIMPORT commands", func() {
ctx := context.TODO()
var client *redis.Client
BeforeEach(func() {
SkipBeforeRedisVersion("8.10", "HIMPORT requires Redis 8.10")
client = redis.NewClient(redisOptions())
// Version alone is not enough: 8.10 pre-release builds may lack the
// HIMPORT command family. Probe for it and skip when absent.
if err := client.HImportDiscardAll(ctx).Err(); err != nil &&
strings.Contains(strings.ToLower(err.Error()), "unknown command") {
Expect(client.Close()).NotTo(HaveOccurred())
client = nil
Skip("server does not support HIMPORT")
}
Expect(client.FlushDB(ctx).Err()).NotTo(HaveOccurred())
})
AfterEach(func() {
if client != nil {
Expect(client.Close()).NotTo(HaveOccurred())
}
})
It("prepares a fieldset and creates hashes from values", func() {
prepare := client.HImportPrepare(ctx, "shared", "name", "email", "age")
Expect(prepare.Err()).NotTo(HaveOccurred())
Expect(prepare.Val()).To(Equal("OK"))
set := client.HImportSet(ctx, "shared:1", "shared", "alice", "alice@example.com", "25")
Expect(set.Err()).NotTo(HaveOccurred())
Expect(set.Val()).To(Equal("OK"))
Expect(client.HGet(ctx, "shared:1", "name").Val()).To(Equal("alice"))
Expect(client.HGet(ctx, "shared:1", "email").Val()).To(Equal("alice@example.com"))
Expect(client.HGet(ctx, "shared:1", "age").Val()).To(Equal("25"))
Expect(client.HGetAll(ctx, "shared:1").Val()).To(HaveLen(3))
})
It("pairs values positionally with each fieldset's declared order", func() {
Expect(client.HImportPrepare(ctx, "order1", "a", "b", "c").Err()).NotTo(HaveOccurred())
Expect(client.HImportPrepare(ctx, "order2", "c", "b", "a").Err()).NotTo(HaveOccurred())
Expect(client.HImportSet(ctx, "order:key1", "order1", "va1", "vb1", "vc1").Err()).NotTo(HaveOccurred())
Expect(client.HImportSet(ctx, "order:key2", "order2", "vc2", "vb2", "va2").Err()).NotTo(HaveOccurred())
Expect(client.HGet(ctx, "order:key1", "a").Val()).To(Equal("va1"))
Expect(client.HGet(ctx, "order:key2", "a").Val()).To(Equal("va2"))
})
It("replays the prepare on every pooled connection that runs a set", func() {
Expect(client.HImportPrepare(ctx, "pooled", "f1", "f2").Err()).NotTo(HaveOccurred())
// Concurrent writers spread across the pool; every connection must
// have the fieldset replayed onto its session before its first SET.
var wg sync.WaitGroup
errs := make(chan error, 100)
for i := 0; i < 100; i++ {
wg.Add(1)
go func(i int) {
defer wg.Done()
key := fmt.Sprintf("pooled:%d", i)
if err := client.HImportSet(ctx, key, "pooled", "v1", "v2").Err(); err != nil {
errs <- err
}
}(i)
}
wg.Wait()
close(errs)
for err := range errs {
Expect(err).NotTo(HaveOccurred())
}
keys, err := client.Keys(ctx, "pooled:*").Result()
Expect(err).NotTo(HaveOccurred())
Expect(keys).To(HaveLen(100))
})
It("fully replaces an existing hash", func() {
Expect(client.HSet(ctx, "replace:1", "old", "x", "stale", "y").Err()).NotTo(HaveOccurred())
Expect(client.HImportPrepare(ctx, "repl", "fresh").Err()).NotTo(HaveOccurred())
Expect(client.HImportSet(ctx, "replace:1", "repl", "z").Err()).NotTo(HaveOccurred())
all := client.HGetAll(ctx, "replace:1").Val()
Expect(all).To(Equal(map[string]string{"fresh": "z"}))
})
It("re-prepares after a fieldset is replaced", func() {
Expect(client.HImportPrepare(ctx, "evolving", "a").Err()).NotTo(HaveOccurred())
Expect(client.HImportSet(ctx, "evolving:1", "evolving", "1").Err()).NotTo(HaveOccurred())
// Preparing the same name again silently replaces it; subsequent
// sets must use the new field list on every pooled connection.
Expect(client.HImportPrepare(ctx, "evolving", "a", "b").Err()).NotTo(HaveOccurred())
Expect(client.HImportSet(ctx, "evolving:2", "evolving", "1", "2").Err()).NotTo(HaveOccurred())
Expect(client.HGet(ctx, "evolving:2", "b").Val()).To(Equal("2"))
})
It("discards a fieldset and stops replaying it", func() {
Expect(client.HImportPrepare(ctx, "gone", "f").Err()).NotTo(HaveOccurred())
Expect(client.HImportSet(ctx, "gone:1", "gone", "v").Err()).NotTo(HaveOccurred())
discard := client.HImportDiscard(ctx, "gone")
Expect(discard.Err()).NotTo(HaveOccurred())
err := client.HImportSet(ctx, "gone:2", "gone", "v").Err()
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("no such fieldset"))
// Hashes created through the fieldset survive the discard.
Expect(client.HGet(ctx, "gone:1", "f").Val()).To(Equal("v"))
})
It("discards all fieldsets", func() {
Expect(client.HImportPrepare(ctx, "all1", "f").Err()).NotTo(HaveOccurred())
Expect(client.HImportPrepare(ctx, "all2", "g").Err()).NotTo(HaveOccurred())
count, err := client.HImportDiscardAll(ctx).Result()
Expect(err).NotTo(HaveOccurred())
Expect(count).To(BeNumerically(">=", 2))
err = client.HImportSet(ctx, "all:1", "all1", "v").Err()
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("no such fieldset"))
})
It("propagates server errors unchanged", func() {
err := client.HImportSet(ctx, "nofs:1", "never-prepared", "v").Err()
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("no such fieldset"))
err = client.HImportPrepare(ctx, "dup", "f", "f").Err()
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("duplicate field name"))
Expect(client.Set(ctx, "notahash", "s", 0).Err()).NotTo(HaveOccurred())
Expect(client.HImportPrepare(ctx, "wrongtype", "f").Err()).NotTo(HaveOccurred())
err = client.HImportSet(ctx, "notahash", "wrongtype", "v").Err()
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("WRONGTYPE"))
})
It("counts value/field mismatches server-side", func() {
Expect(client.HImportPrepare(ctx, "counted", "a", "b").Err()).NotTo(HaveOccurred())
err := client.HImportSet(ctx, "counted:1", "counted", "only-one").Err()
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("value count"))
})
It("works on a dedicated connection", func() {
conn := client.Conn()
defer conn.Close()
Expect(conn.HImportPrepare(ctx, "dedicated", "f").Err()).NotTo(HaveOccurred())
Expect(conn.HImportSet(ctx, "dedicated:1", "dedicated", "v").Err()).NotTo(HaveOccurred())
count, err := conn.HImportDiscard(ctx, "dedicated").Result()
Expect(err).NotTo(HaveOccurred())
Expect(count).To(Equal(int64(1)))
})
It("prepares and sets within one pipeline", func() {
pipe := client.Pipeline()
prepare := pipe.HImportPrepare(ctx, "piped", "f1", "f2")
set1 := pipe.HImportSet(ctx, "piped:1", "piped", "a", "b")
set2 := pipe.HImportSet(ctx, "piped:2", "piped", "c", "d")
_, err := pipe.Exec(ctx)
Expect(err).NotTo(HaveOccurred())
Expect(prepare.Val()).To(Equal("OK"))
Expect(set1.Val()).To(Equal("OK"))
Expect(set2.Val()).To(Equal("OK"))
Expect(client.HGet(ctx, "piped:2", "f2").Val()).To(Equal("d"))
})
It("replays a registered fieldset into a pipeline", func() {
Expect(client.HImportPrepare(ctx, "pipedreg", "f").Err()).NotTo(HaveOccurred())
pipe := client.Pipeline()
var sets []*redis.StatusCmd
for i := 0; i < 10; i++ {
sets = append(sets, pipe.HImportSet(ctx, fmt.Sprintf("pipedreg:%d", i), "pipedreg", "v"))
}
_, err := pipe.Exec(ctx)
Expect(err).NotTo(HaveOccurred())
for _, set := range sets {
Expect(set.Val()).To(Equal("OK"))
}
})
It("replays a registered fieldset into a transaction", func() {
Expect(client.HImportPrepare(ctx, "txfs", "f").Err()).NotTo(HaveOccurred())
tx := client.TxPipeline()
set1 := tx.HImportSet(ctx, "txfs:1", "txfs", "a")
set2 := tx.HImportSet(ctx, "txfs:2", "txfs", "b")
_, err := tx.Exec(ctx)
Expect(err).NotTo(HaveOccurred())
Expect(set1.Val()).To(Equal("OK"))
Expect(set2.Val()).To(Equal("OK"))
Expect(client.HGet(ctx, "txfs:1", "f").Val()).To(Equal("a"))
Expect(client.HGet(ctx, "txfs:2", "f").Val()).To(Equal("b"))
})
})
himport_mock_test.go
+893
View File
@@ -0,0 +1,893 @@
package redis_test
import (
"bufio"
"context"
"fmt"
"io"
"net"
"strconv"
"strings"
"sync"
"testing"
"time"
"github.com/redis/go-redis/v9"
"github.com/redis/go-redis/v9/maintnotifications"
)
// himportMockServer is a minimal RESP2 server implementing the HIMPORT
// session semantics: fieldsets live per connection, RESET wipes them, and a
// BOOM command drops the connection. It lets the lazy-replay logic be
// exercised end to end without a Redis 8.10 server.
type himportMockServer struct {
ln net.Listener
// resp3 makes HELLO succeed so the client negotiates RESP3.
resp3 bool
mu sync.Mutex
hashes map[string]map[string]string
prepareCount int
// sessions tracks the per-connection fieldset maps of live connections,
// so tests can observe fieldsets surviving on sessions the client is not
// currently using.
sessions map[*himportMockSession]struct{}
// pushBeforeSetReply, when armed, makes the next HIMPORT SET reply be
// preceded by an out-of-band RESP3 push frame.
pushBeforeSetReply bool
// boomBeforeNextPrepare, when armed, drops the connection when the next
// HIMPORT PREPARE arrives, before replying.
boomBeforeNextPrepare bool
}
// himportMockSession is one connection's fieldset state; guarded by the
// server mutex.
type himportMockSession struct {
fieldsets map[string][]string
}
func newHImportMockServer(t *testing.T) *himportMockServer {
t.Helper()
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatalf("listen: %v", err)
}
srv := &himportMockServer{
ln: ln,
hashes: make(map[string]map[string]string),
sessions: make(map[*himportMockSession]struct{}),
}
go func() {
for {
conn, err := ln.Accept()
if err != nil {
return
}
go srv.serve(conn)
}
}()
t.Cleanup(func() { _ = ln.Close() })
return srv
}
func (s *himportMockServer) addr() string { return s.ln.Addr().String() }
func (s *himportMockServer) prepares() int {
s.mu.Lock()
defer s.mu.Unlock()
return s.prepareCount
}
func (s *himportMockServer) hash(key string) map[string]string {
s.mu.Lock()
defer s.mu.Unlock()
return s.hashes[key]
}
// totalSessionFieldsets counts fieldsets across all live connection
// sessions — the server-side footprint the lazy discard replay must clean up.
func (s *himportMockServer) totalSessionFieldsets() int {
s.mu.Lock()
defer s.mu.Unlock()
n := 0
for session := range s.sessions {
n += len(session.fieldsets)
}
return n
}
func (s *himportMockServer) hashKeys() []string {
s.mu.Lock()
defer s.mu.Unlock()
keys := make([]string, 0, len(s.hashes))
for k := range s.hashes {
keys = append(keys, k)
}
return keys
}
func (s *himportMockServer) armPushBeforeSetReply() {
s.mu.Lock()
s.pushBeforeSetReply = true
s.mu.Unlock()
}
func (s *himportMockServer) armBoomBeforeNextPrepare() {
s.mu.Lock()
s.boomBeforeNextPrepare = true
s.mu.Unlock()
}
func (s *himportMockServer) consumeBoomBeforeNextPrepare() bool {
s.mu.Lock()
defer s.mu.Unlock()
armed := s.boomBeforeNextPrepare
s.boomBeforeNextPrepare = false
return armed
}
func (s *himportMockServer) consumePushBeforeSetReply() bool {
s.mu.Lock()
defer s.mu.Unlock()
armed := s.pushBeforeSetReply
s.pushBeforeSetReply = false
return armed
}
// readCommand parses one RESP2 array-of-bulk-strings request.
func readCommand(rd *bufio.Reader) ([]string, error) {
line, err := rd.ReadString('\n')
if err != nil {
return nil, err
}
line = strings.TrimSuffix(strings.TrimSuffix(line, "\n"), "\r")
if len(line) == 0 || line[0] != '*' {
return nil, fmt.Errorf("unexpected request line %q", line)
}
n, err := strconv.Atoi(line[1:])
if err != nil {
return nil, err
}
args := make([]string, 0, n)
for i := 0; i < n; i++ {
sizeLine, err := rd.ReadString('\n')
if err != nil {
return nil, err
}
sizeLine = strings.TrimSuffix(strings.TrimSuffix(sizeLine, "\n"), "\r")
if len(sizeLine) == 0 || sizeLine[0] != '$' {
return nil, fmt.Errorf("unexpected bulk header %q", sizeLine)
}
size, err := strconv.Atoi(sizeLine[1:])
if err != nil {
return nil, err
}
buf := make([]byte, size+2)
if _, err := io.ReadFull(rd, buf); err != nil {
return nil, err
}
args = append(args, string(buf[:size]))
}
return args, nil
}
func (s *himportMockServer) serve(conn net.Conn) {
defer conn.Close()
rd := bufio.NewReader(conn)
// Per-connection session state, visible to tests through the server.
session := &himportMockSession{fieldsets: make(map[string][]string)}
s.mu.Lock()
s.sessions[session] = struct{}{}
s.mu.Unlock()
defer func() {
s.mu.Lock()
delete(s.sessions, session)
s.mu.Unlock()
}()
reply := func(msg string) bool {
_, err := conn.Write([]byte(msg))
return err == nil
}
var (
inMulti bool
queued [][]string
)
for {
args, err := readCommand(rd)
if err != nil {
return
}
switch strings.ToUpper(args[0]) {
case "HELLO":
if s.resp3 {
if !reply("%3\r\n$6\r\nserver\r\n$5\r\nredis\r\n$7\r\nversion\r\n$6\r\n8.10.0\r\n$5\r\nproto\r\n:3\r\n") {
return
}
continue
}
// Pre-RESP3 server: the client falls back to RESP2.
if !reply("-ERR unknown command 'HELLO'\r\n") {
return
}
case "MULTI":
inMulti = true
queued = queued[:0]
if !reply("+OK\r\n") {
return
}
case "EXEC":
inMulti = false
var b strings.Builder
fmt.Fprintf(&b, "*%d\r\n", len(queued))
for _, q := range queued {
b.WriteString(s.commandReply(q, session))
}
queued = queued[:0]
if !reply(b.String()) {
return
}
case "RESET":
s.mu.Lock()
session.fieldsets = make(map[string][]string)
s.mu.Unlock()
if !reply("+RESET\r\n") {
return
}
case "BOOM":
// Simulate a dropped connection: close without replying.
return
case "HOLD":
// Occupy this connection long enough for a concurrent caller to
// be forced onto another pooled connection.
time.Sleep(150 * time.Millisecond)
if !reply("+OK\r\n") {
return
}
default:
if inMulti {
queued = append(queued, args)
if !reply("+QUEUED\r\n") {
return
}
continue
}
resp := s.commandReply(args, session)
if resp == "" {
// Sentinel from an armed connection drop.
return
}
if strings.ToUpper(args[0]) == "HIMPORT" && strings.ToUpper(args[1]) == "SET" &&
strings.HasPrefix(resp, "+OK") && s.consumePushBeforeSetReply() {
// Out-of-band push frame squeezed in right before the SET
// reply (after the injected PREPARE's reply): the client
// must drain it instead of consuming it as the SET reply.
// The payload is sized like a real notification so
// PeekPushNotificationName's initial peek window (36 bytes)
// is satisfied without waiting for the read deadline.
resp = ">2\r\n$8\r\ntestpush\r\n$32\r\n" + strings.Repeat("x", 32) + "\r\n" + resp
}
if !reply(resp) {
return
}
}
}
}
// commandReply produces the RESP reply for one command; used both for direct
// dispatch and for commands queued inside MULTI/EXEC.
func (s *himportMockServer) commandReply(args []string, session *himportMockSession) string {
if strings.ToUpper(args[0]) == "HIMPORT" {
return s.himportReply(args, session)
}
return "+OK\r\n"
}
func (s *himportMockServer) himportReply(args []string, session *himportMockSession) string {
switch strings.ToUpper(args[1]) {
case "PREPARE":
if s.consumeBoomBeforeNextPrepare() {
// Simulate the connection dying mid-replay: the caller closes
// the conn when it sees the empty sentinel.
return ""
}
name, fields := args[2], args[3:]
seen := make(map[string]struct{}, len(fields))
for _, f := range fields {
if _, dup := seen[f]; dup {
return "-ERR duplicate field name in fieldset\r\n"
}
seen[f] = struct{}{}
}
s.mu.Lock()
session.fieldsets[name] = append([]string(nil), fields...)
s.prepareCount++
s.mu.Unlock()
return "+OK\r\n"
case "SET":
key, name, values := args[2], args[3], args[4:]
s.mu.Lock()
fields, ok := session.fieldsets[name]
s.mu.Unlock()
if !ok {
return "-ERR no such fieldset\r\n"
}
if len(values) != len(fields) {
return "-ERR value count does not match fieldset field count\r\n"
}
hash := make(map[string]string, len(fields))
for i, f := range fields {
hash[f] = values[i]
}
s.mu.Lock()
s.hashes[key] = hash
s.mu.Unlock()
return "+OK\r\n"
case "DISCARD":
s.mu.Lock()
_, ok := session.fieldsets[args[2]]
delete(session.fieldsets, args[2])
s.mu.Unlock()
if ok {
return ":1\r\n"
}
return ":0\r\n"
case "DISCARDALL":
s.mu.Lock()
n := len(session.fieldsets)
session.fieldsets = make(map[string][]string)
s.mu.Unlock()
return ":" + strconv.Itoa(n) + "\r\n"
}
return "-ERR Unknown subcommand\r\n"
}
// TestHImportLazyReplay drives the client against the mock server and pins
// the lazy-replay contract: PREPARE runs at most once per connection session
// (NF.2), a session that lost its fieldsets ("no such fieldset" after RESET)
// is transparently re-prepared and retried once, and a brand-new connection
// is prepared on first use.
func TestHImportLazyReplay(t *testing.T) {
srv := newHImportMockServer(t)
ctx := context.Background()
client := redis.NewClient(&redis.Options{
Addr: srv.addr(),
Protocol: 2,
PoolSize: 1, // deterministic: every command runs on the same connection
DisableIdentity: true,
})
defer client.Close()
if err := client.HImportPrepare(ctx, "fs", "a", "b").Err(); err != nil {
t.Fatalf("prepare: %v", err)
}
if err := client.HImportSet(ctx, "k1", "fs", "1", "2").Err(); err != nil {
t.Fatalf("set k1: %v", err)
}
if err := client.HImportSet(ctx, "k2", "fs", "3", "4").Err(); err != nil {
t.Fatalf("set k2: %v", err)
}
if got := srv.prepares(); got != 1 {
t.Errorf("prepares after two sets on one session = %d, want 1 (NF.2: replay at most once)", got)
}
if h := srv.hash("k1"); h["a"] != "1" || h["b"] != "2" {
t.Errorf("k1 = %v, want a=1 b=2", h)
}
// RESET wipes the server session behind the client's back; the next SET
// gets "no such fieldset", which must trigger one re-prepare + retry.
if err := client.Do(ctx, "reset").Err(); err != nil {
t.Fatalf("reset: %v", err)
}
if err := client.HImportSet(ctx, "k3", "fs", "5", "6").Err(); err != nil {
t.Fatalf("set k3 after reset: %v", err)
}
if got := srv.prepares(); got != 2 {
t.Errorf("prepares after reset recovery = %d, want 2", got)
}
if h := srv.hash("k3"); h["a"] != "5" || h["b"] != "6" {
t.Errorf("k3 = %v, want a=5 b=6", h)
}
// BOOM drops the connection; the pool dials a fresh one whose empty
// session must be prepared before its first SET.
if err := client.Do(ctx, "boom").Err(); err == nil {
t.Fatal("boom should surface a connection error")
}
prepBefore := srv.prepares()
if err := client.HImportSet(ctx, "k4", "fs", "7", "8").Err(); err != nil {
t.Fatalf("set k4 on fresh connection: %v", err)
}
if got := srv.prepares(); got != prepBefore+1 {
t.Errorf("prepares after fresh connection = %d, want %d", got, prepBefore+1)
}
// Pipeline on another fresh connection: one injected PREPARE covers all
// SETs of the same fieldset in the batch.
if err := client.Do(ctx, "boom").Err(); err == nil {
t.Fatal("boom should surface a connection error")
}
prepBefore = srv.prepares()
pipe := client.Pipeline()
set5 := pipe.HImportSet(ctx, "k5", "fs", "9", "10")
set6 := pipe.HImportSet(ctx, "k6", "fs", "11", "12")
if _, err := pipe.Exec(ctx); err != nil {
t.Fatalf("pipeline: %v", err)
}
if set5.Err() != nil || set6.Err() != nil {
t.Fatalf("pipeline sets: %v, %v", set5.Err(), set6.Err())
}
if got := srv.prepares(); got != prepBefore+1 {
t.Errorf("prepares after pipeline on fresh connection = %d, want %d", got, prepBefore+1)
}
if h := srv.hash("k6"); h["a"] != "11" || h["b"] != "12" {
t.Errorf("k6 = %v, want a=11 b=12", h)
}
// Discard drops the client-side registry entry: the next SET is a raw
// pass-through and surfaces the server error unchanged.
if err := client.HImportDiscard(ctx, "fs").Err(); err != nil {
t.Fatalf("discard: %v", err)
}
err := client.HImportSet(ctx, "k7", "fs", "13", "14").Err()
if err == nil || !strings.Contains(err.Error(), "no such fieldset") {
t.Errorf("set after discard = %v, want no-such-fieldset pass-through", err)
}
}
// TestHImportPipelineRecoversAfterSessionLoss pins the pipeline recovery
// contract (NF.4): HIMPORT SETs of a registered fieldset that fail with "no
// such fieldset" after the session was wiped (RESET) are re-prepared and
// re-issued once within the same Exec — the error never reaches the caller,
// and only the SETs run again.
func TestHImportPipelineRecoversAfterSessionLoss(t *testing.T) {
srv := newHImportMockServer(t)
ctx := context.Background()
client := redis.NewClient(&redis.Options{
Addr: srv.addr(),
Protocol: 2,
PoolSize: 1,
DisableIdentity: true,
})
defer client.Close()
if err := client.HImportPrepare(ctx, "fs", "a", "b").Err(); err != nil {
t.Fatalf("prepare: %v", err)
}
if err := client.HImportSet(ctx, "k1", "fs", "1", "2").Err(); err != nil {
t.Fatalf("set k1: %v", err)
}
// Wipe the server session behind the client's back; the connection's
// prepared flag is now stale.
if err := client.Do(ctx, "reset").Err(); err != nil {
t.Fatalf("reset: %v", err)
}
prepBefore := srv.prepares()
pipe := client.Pipeline()
other := pipe.Set(ctx, "plain", "x", 0)
set2 := pipe.HImportSet(ctx, "k2", "fs", "3", "4")
set3 := pipe.HImportSet(ctx, "k3", "fs", "5", "6")
if _, err := pipe.Exec(ctx); err != nil {
t.Fatalf("pipeline exec: %v", err)
}
if other.Err() != nil || set2.Err() != nil || set3.Err() != nil {
t.Fatalf("pipeline cmds after transparent recovery: %v, %v, %v",
other.Err(), set2.Err(), set3.Err())
}
if got := srv.prepares(); got != prepBefore+1 {
t.Errorf("prepares after recovery = %d, want %d (single replay)", got, prepBefore+1)
}
if h := srv.hash("k3"); h["a"] != "5" || h["b"] != "6" {
t.Errorf("k3 = %v, want a=5 b=6", h)
}
}
// TestHImportPipelineReissueTransportErrorScoped pins the failure scoping of
// the pipeline re-issue: when the retry round trip itself dies on a
// transport error, the batch is neither re-executed nor failed wholesale —
// commands whose results were delivered in the first round trip keep them,
// the failed SETs keep their errors, and the suspect connection is discarded.
func TestHImportPipelineReissueTransportErrorScoped(t *testing.T) {
srv := newHImportMockServer(t)
ctx := context.Background()
client := redis.NewClient(&redis.Options{
Addr: srv.addr(),
Protocol: 2,
PoolSize: 1,
MaxRetries: -1,
DisableIdentity: true,
})
defer client.Close()
if err := client.HImportPrepare(ctx, "fs", "a").Err(); err != nil {
t.Fatalf("prepare: %v", err)
}
if err := client.HImportSet(ctx, "k1", "fs", "1").Err(); err != nil {
t.Fatalf("set k1: %v", err)
}
if err := client.Do(ctx, "reset").Err(); err != nil {
t.Fatalf("reset: %v", err)
}
// The first round trip delivers: plain SET OK, HIMPORT SET fails with
// "no such fieldset". The re-issue round trip then dies (the mock drops
// the connection on the replayed PREPARE).
srv.armBoomBeforeNextPrepare()
pipe := client.Pipeline()
plain := pipe.Set(ctx, "plain", "x", 0)
set2 := pipe.HImportSet(ctx, "k2", "fs", "2")
_, _ = pipe.Exec(ctx)
if plain.Err() != nil || plain.Val() != "OK" {
t.Errorf("delivered plain SET = %q, %v; must keep its first-round result", plain.Val(), plain.Err())
}
if err := set2.Err(); err == nil || !strings.Contains(err.Error(), "no such fieldset") {
t.Errorf("failed HIMPORT SET = %v, want its own no-such-fieldset error", err)
}
// The half-read connection was discarded; the next command dials fresh
// and the registered fieldset replays.
if err := client.HImportSet(ctx, "k3", "fs", "3").Err(); err != nil {
t.Fatalf("set k3 on fresh connection: %v", err)
}
if h := srv.hash("k3"); h["a"] != "3" {
t.Errorf("k3 = %v, want a=3", h)
}
}
// TestHImportTxSurfacesSessionLoss pins the documented transaction
// limitation: an executed MULTI/EXEC cannot be partially re-run, so the "no
// such fieldset" error surfaces — but the stale flag is invalidated, and the
// caller's retry of the transaction succeeds.
func TestHImportTxSurfacesSessionLoss(t *testing.T) {
srv := newHImportMockServer(t)
ctx := context.Background()
client := redis.NewClient(&redis.Options{
Addr: srv.addr(),
Protocol: 2,
PoolSize: 1,
DisableIdentity: true,
})
defer client.Close()
if err := client.HImportPrepare(ctx, "fs", "a").Err(); err != nil {
t.Fatalf("prepare: %v", err)
}
if err := client.HImportSet(ctx, "k1", "fs", "1").Err(); err != nil {
t.Fatalf("set k1: %v", err)
}
if err := client.Do(ctx, "reset").Err(); err != nil {
t.Fatalf("reset: %v", err)
}
tx := client.TxPipeline()
set := tx.HImportSet(ctx, "k2", "fs", "2")
_, _ = tx.Exec(ctx)
if err := set.Err(); err == nil || !strings.Contains(err.Error(), "no such fieldset") {
t.Fatalf("tx exec after session loss = %v, want no-such-fieldset", err)
}
// The stale flag was invalidated: the retried transaction replays the
// PREPARE and succeeds.
tx = client.TxPipeline()
set = tx.HImportSet(ctx, "k2", "fs", "2")
if _, err := tx.Exec(ctx); err != nil {
t.Fatalf("tx retry: %v", err)
}
if set.Err() != nil {
t.Fatalf("tx retry set: %v", set.Err())
}
if h := srv.hash("k2"); h["a"] != "2" {
t.Errorf("k2 = %v, want a=2", h)
}
}
// TestHImportLazyDiscardPropagation pins the lazy discard contract: a
// DISCARD/DISCARDALL executes on one pooled connection, and other sessions
// still holding the fieldset replay the discard before their next HIMPORT
// command, so post-discard behavior is deterministic and no zombie fieldsets
// survive on connections the client keeps using.
func TestHImportLazyDiscardPropagation(t *testing.T) {
srv := newHImportMockServer(t)
ctx := context.Background()
client := redis.NewClient(&redis.Options{
Addr: srv.addr(),
Protocol: 2,
PoolSize: 2,
DisableIdentity: true,
})
defer client.Close()
// runBoth occupies both pooled connections concurrently (HOLD keeps each
// busy long enough for the other goroutine to be forced onto the second
// connection) and runs one HIMPORT SET on each; it returns the SET
// errors.
runBoth := func(fieldset, keyPrefix string) [2]error {
var wg sync.WaitGroup
var errs [2]error
for i := 0; i < 2; i++ {
wg.Add(1)
go func(i int) {
defer wg.Done()
pipe := client.Pipeline()
pipe.Do(ctx, "hold")
set := pipe.HImportSet(ctx, fmt.Sprintf("%s:%d", keyPrefix, i), fieldset, "v")
_, _ = pipe.Exec(ctx)
errs[i] = set.Err()
}(i)
}
wg.Wait()
return errs
}
// Prepare the fieldset on both pooled connections.
if err := client.HImportPrepare(ctx, "fs", "f").Err(); err != nil {
t.Fatalf("prepare: %v", err)
}
for i, err := range runBoth("fs", "seed") {
if err != nil {
t.Fatalf("seed set %d: %v", i, err)
}
}
if got := srv.totalSessionFieldsets(); got != 2 {
t.Fatalf("sessions holding fs = %d, want 2", got)
}
// DISCARD executes on one borrowed connection; the other session still
// holds the fieldset for now.
if err := client.HImportDiscard(ctx, "fs").Err(); err != nil {
t.Fatalf("discard: %v", err)
}
if got := srv.totalSessionFieldsets(); got != 1 {
t.Fatalf("sessions holding fs right after discard = %d, want 1", got)
}
// The next HIMPORT command on each connection replays the DISCARD: the
// leftover session copy is removed and every SET fails deterministically.
for i, err := range runBoth("fs", "late") {
if err == nil || !strings.Contains(err.Error(), "no such fieldset") {
t.Errorf("set %d after discard = %v, want no-such-fieldset", i, err)
}
}
if got := srv.totalSessionFieldsets(); got != 0 {
t.Errorf("sessions holding fs after replayed discard = %d, want 0", got)
}
// Same for DISCARDALL, via the epoch: prepare on both connections, wipe
// through one, and let the other session replay the wipe.
if err := client.HImportPrepare(ctx, "fs2", "g").Err(); err != nil {
t.Fatalf("prepare fs2: %v", err)
}
for i, err := range runBoth("fs2", "seed2") {
if err != nil {
t.Fatalf("seed2 set %d: %v", i, err)
}
}
if got := srv.totalSessionFieldsets(); got != 2 {
t.Fatalf("sessions holding fs2 = %d, want 2", got)
}
if err := client.HImportDiscardAll(ctx).Err(); err != nil {
t.Fatalf("discardall: %v", err)
}
if got := srv.totalSessionFieldsets(); got != 1 {
t.Fatalf("sessions right after discardall = %d, want 1", got)
}
for i, err := range runBoth("fs2", "late2") {
if err == nil || !strings.Contains(err.Error(), "no such fieldset") {
t.Errorf("set %d after discardall = %v, want no-such-fieldset", i, err)
}
}
if got := srv.totalSessionFieldsets(); got != 0 {
t.Errorf("sessions after replayed discardall = %d, want 0", got)
}
}
// TestHImportRingFanOut drives the ring (whose shards are plain standalone
// servers, so two mock servers suffice) through the fan-out API: PREPARE
// registers once and eagerly prepares one connection per shard, SETs on any
// shard need no further replay, DISCARD/DISCARDALL clean every shard, and a
// deterministic PREPARE rejection withdraws the registration.
func TestHImportRingFanOut(t *testing.T) {
srv1 := newHImportMockServer(t)
srv2 := newHImportMockServer(t)
ctx := context.Background()
ring := redis.NewRing(&redis.RingOptions{
Addrs: map[string]string{
"shard1": srv1.addr(),
"shard2": srv2.addr(),
},
PoolSize: 1,
DisableIdentity: true,
})
defer ring.Close()
// Fan-out PREPARE: exactly one connection per shard is prepared.
if err := ring.HImportPrepare(ctx, "fs", "a", "b").Err(); err != nil {
t.Fatalf("prepare: %v", err)
}
if p1, p2 := srv1.prepares(), srv2.prepares(); p1 != 1 || p2 != 1 {
t.Fatalf("prepares after fan-out = %d/%d, want 1/1", p1, p2)
}
// SETs route by key hash to some shard; the fan-out already prepared
// those connections, so no further PREPARE is sent.
for i := 0; i < 20; i++ {
key := fmt.Sprintf("ring:%d", i)
if err := ring.HImportSet(ctx, key, "fs", "1", "2").Err(); err != nil {
t.Fatalf("set %s: %v", key, err)
}
}
if p1, p2 := srv1.prepares(), srv2.prepares(); p1 != 1 || p2 != 1 {
t.Errorf("prepares after sets = %d/%d, want 1/1 (no replay needed)", p1, p2)
}
if total := len(srv1.hashKeys()) + len(srv2.hashKeys()); total != 20 {
t.Errorf("hashes across shards = %d, want 20", total)
}
// Fan-out DISCARD cleans both shard sessions; the return value reports
// the registry lifecycle.
removed, err := ring.HImportDiscard(ctx, "fs").Result()
if err != nil {
t.Fatalf("discard: %v", err)
}
if removed != 1 {
t.Errorf("discard = %d, want 1", removed)
}
if got := srv1.totalSessionFieldsets() + srv2.totalSessionFieldsets(); got != 0 {
t.Errorf("session fieldsets after fan-out discard = %d, want 0", got)
}
err = ring.HImportSet(ctx, "ring:after", "fs", "1", "2").Err()
if err == nil || !strings.Contains(err.Error(), "no such fieldset") {
t.Errorf("set after discard = %v, want no-such-fieldset pass-through", err)
}
// A deterministic server rejection (duplicate field) withdraws the
// registration: no replay is attempted afterwards.
err = ring.HImportPrepare(ctx, "dup", "f", "f").Err()
if err == nil || !strings.Contains(err.Error(), "duplicate field name") {
t.Fatalf("duplicate prepare = %v, want duplicate-field error", err)
}
err = ring.HImportSet(ctx, "ring:dup", "dup", "v").Err()
if err == nil || !strings.Contains(err.Error(), "no such fieldset") {
t.Errorf("set after rejected prepare = %v, want raw no-such-fieldset", err)
}
// Fan-out DISCARDALL reports the registry count and cleans both shards.
if err := ring.HImportPrepare(ctx, "fs2", "g").Err(); err != nil {
t.Fatalf("prepare fs2: %v", err)
}
count, err := ring.HImportDiscardAll(ctx).Result()
if err != nil {
t.Fatalf("discardall: %v", err)
}
if count != 1 {
t.Errorf("discardall = %d, want 1", count)
}
if got := srv1.totalSessionFieldsets() + srv2.totalSessionFieldsets(); got != 0 {
t.Errorf("session fieldsets after fan-out discardall = %d, want 0", got)
}
}
// TestHImportInjectedPrepareWithPushNotification pins the RESP3 read
// sequence: a push frame arriving between the injected PREPARE's reply and
// the SET's reply must be drained as a notification, not consumed as the
// SET's reply.
func TestHImportInjectedPrepareWithPushNotification(t *testing.T) {
srv := newHImportMockServer(t)
srv.resp3 = true
ctx := context.Background()
client := redis.NewClient(&redis.Options{
Addr: srv.addr(),
Protocol: 3,
PoolSize: 1,
MaxRetries: -1, // fail BOOM fast; the injected PREPARE needs no retries
DisableIdentity: true,
// The mock is not a real cluster; keep maintenance-notification
// machinery out of the connection lifecycle.
MaintNotificationsConfig: &maintnotifications.Config{Mode: maintnotifications.ModeDisabled},
})
defer client.Close()
if err := client.HImportPrepare(ctx, "fs", "a", "b").Err(); err != nil {
t.Fatalf("prepare: %v", err)
}
// Drop the connection so the next SET runs on a fresh session and gets
// the PREPARE injected in front of it.
if err := client.Do(ctx, "boom").Err(); err == nil {
t.Fatal("boom should surface a connection error")
}
prepBefore := srv.prepares()
srv.armPushBeforeSetReply()
if err := client.HImportSet(ctx, "k1", "fs", "1", "2").Err(); err != nil {
t.Fatalf("set with interleaved push notification: %v", err)
}
if got := srv.prepares(); got != prepBefore+1 {
t.Errorf("prepares = %d, want %d (PREPARE must be injected on the fresh connection)", got, prepBefore+1)
}
if h := srv.hash("k1"); h["a"] != "1" || h["b"] != "2" {
t.Errorf("k1 = %v, want a=1 b=2", h)
}
// The connection must stay aligned for subsequent commands.
if err := client.HImportSet(ctx, "k2", "fs", "3", "4").Err(); err != nil {
t.Fatalf("follow-up set: %v", err)
}
if h := srv.hash("k2"); h["a"] != "3" || h["b"] != "4" {
t.Errorf("k2 = %v, want a=3 b=4", h)
}
}
// TestHImportPipelineInjectedReplyFailureStampsBatch pins the error stamping
// when the injected-reply read dies on a transport error before any batch
// reply was consumed: Exec surfaces the error and every command of the batch
// carries it. The outer retry loop stamps errors only on its exit branch,
// not when the attempt budget runs out mid-loop, so the pipeline path must
// stamp before returning — otherwise a batch that keeps dying here reports
// Err() == nil on every command while Exec errors.
func TestHImportPipelineInjectedReplyFailureStampsBatch(t *testing.T) {
srv := newHImportMockServer(t)
ctx := context.Background()
client := redis.NewClient(&redis.Options{
Addr: srv.addr(),
Protocol: 2,
PoolSize: 1,
// Exhaust the budget on the first attempt: stamping must not
// depend on a later attempt reaching the read path.
MaxRetries: -1,
DisableIdentity: true,
})
defer client.Close()
if err := client.HImportPrepare(ctx, "fs", "a").Err(); err != nil {
t.Fatalf("prepare: %v", err)
}
// Drop the connection so the next batch runs on a fresh session and
// gets the PREPARE injected ahead of it.
if err := client.Do(ctx, "boom").Err(); err == nil {
t.Fatal("boom should surface a connection error")
}
// The mock drops the connection when the injected PREPARE arrives: the
// batch was fully written, not one reply arrives.
srv.armBoomBeforeNextPrepare()
pipe := client.Pipeline()
plain := pipe.Set(ctx, "plain", "x", 0)
set := pipe.HImportSet(ctx, "k1", "fs", "1")
if _, err := pipe.Exec(ctx); err == nil {
t.Fatal("exec must surface the transport error")
}
if plain.Err() == nil || set.Err() == nil {
t.Errorf("batch commands must carry the exec error, got %v / %v",
plain.Err(), set.Err())
}
// Same contract on the transaction path.
if err := client.Do(ctx, "boom").Err(); err == nil {
t.Fatal("boom should surface a connection error")
}
srv.armBoomBeforeNextPrepare()
tx := client.TxPipeline()
txSet := tx.HImportSet(ctx, "k2", "fs", "2")
if _, err := tx.Exec(ctx); err == nil {
t.Fatal("tx exec must surface the transport error")
}
if txSet.Err() == nil {
t.Error("tx command must carry the exec error")
}
}
himport_test.go
+731
View File
@@ -0,0 +1,731 @@
package redis
import (
"context"
"errors"
"net"
"reflect"
"sort"
"testing"
"github.com/redis/go-redis/v9/internal/pool"
"github.com/redis/go-redis/v9/internal/proto"
)
// epoch and empty are test-only registry accessors; the linter runs with
// tests excluded, so keeping them in himport.go would flag them as unused.
func (r *himportRegistry) epoch() uint64 {
r.mu.RLock()
e := r.discardAllEpoch
r.mu.RUnlock()
return e
}
func (r *himportRegistry) empty() bool {
if r == nil {
return true
}
r.mu.RLock()
n := len(r.fieldsets)
r.mu.RUnlock()
return n == 0
}
func TestHImportCmdArgs(t *testing.T) {
ctx := context.Background()
prep := NewHImportPrepareCmd(ctx, "fs", "name", "email", "age")
wantPrep := []interface{}{"himport", "prepare", "fs", "name", "email", "age"}
if !reflect.DeepEqual(prep.Args(), wantPrep) {
t.Errorf("prepare args = %v, want %v", prep.Args(), wantPrep)
}
set := NewHImportSetCmd(ctx, "key1", "fs", "alice", "a@example.com", 25)
wantSet := []interface{}{"himport", "set", "key1", "fs", "alice", "a@example.com", 25}
if !reflect.DeepEqual(set.Args(), wantSet) {
t.Errorf("set args = %v, want %v", set.Args(), wantSet)
}
if pos := set.firstKeyPos(); pos != 2 {
t.Errorf("set firstKeyPos = %d, want 2", pos)
}
discard := NewHImportDiscardCmd(ctx, "fs")
wantDiscard := []interface{}{"himport", "discard", "fs"}
if !reflect.DeepEqual(discard.Args(), wantDiscard) {
t.Errorf("discard args = %v, want %v", discard.Args(), wantDiscard)
}
discardAll := NewHImportDiscardAllCmd(ctx)
wantDiscardAll := []interface{}{"himport", "discardall"}
if !reflect.DeepEqual(discardAll.Args(), wantDiscardAll) {
t.Errorf("discardall args = %v, want %v", discardAll.Args(), wantDiscardAll)
}
}
func TestHImportRegistry(t *testing.T) {
var nilRegistry *himportRegistry
if _, ok := nilRegistry.lookup("fs"); ok {
t.Error("nil registry lookup should miss")
}
if !nilRegistry.empty() || !nilRegistry.idle() {
t.Error("nil registry should be empty and idle")
}
r := newHImportRegistry()
if !r.empty() || !r.idle() {
t.Error("new registry should be empty and idle")
}
v1, e1 := r.register("fs", []string{"a", "b"})
if v1 == 0 {
t.Error("versions must start above the 0 sentinel")
}
if e1 != 0 {
t.Errorf("initial epoch = %d, want 0", e1)
}
fs, ok := r.lookup("fs")
if !ok || fs.version != v1 || !reflect.DeepEqual(fs.fields, []string{"a", "b"}) {
t.Errorf("lookup = %+v, %v; want fields [a b] at version %d", fs, ok, v1)
}
// Re-registering the same name must bump the version so stale
// per-connection flags are invalidated.
v2, _ := r.register("fs", []string{"c"})
if v2 <= v1 {
t.Errorf("re-register version = %d, want > %d", v2, v1)
}
// Discard leaves a tombstone; re-registering clears it (the new version
// replaces the fieldset server-side, no discard needed).
r.discard("fs")
if _, ok := r.lookup("fs"); ok {
t.Error("lookup after discard should miss")
}
if _, tombs := r.cleanupSnapshot(); len(tombs) != 1 || tombs[0] != "fs" {
t.Errorf("tombstones after discard = %v, want [fs]", tombs)
}
if r.idle() {
t.Error("registry with tombstones is not idle")
}
r.register("fs", []string{"d"})
if _, tombs := r.cleanupSnapshot(); len(tombs) != 0 {
t.Errorf("tombstones after re-register = %v, want none", tombs)
}
// Discarding an unknown name leaves no tombstone: the client never
// prepared it anywhere.
r.discard("never-registered")
if _, tombs := r.cleanupSnapshot(); len(tombs) != 0 {
t.Errorf("tombstones after unknown discard = %v, want none", tombs)
}
// Discard-all moves to a new epoch, drops fieldsets and tombstones, and
// reports how many fieldsets were registered.
r.discard("fs")
epoch, removed := r.discardAll()
if epoch != 1 || r.epoch() != 1 {
t.Errorf("epoch after discardAll = %d/%d, want 1", epoch, r.epoch())
}
if removed != 0 {
t.Errorf("discardAll removed = %d, want 0 (fs was already discarded)", removed)
}
if !r.empty() {
t.Error("registry should be empty after discardAll")
}
if _, tombs := r.cleanupSnapshot(); len(tombs) != 0 {
t.Errorf("tombstones after discardAll = %v, want none", tombs)
}
if r.idle() {
t.Error("registry with a non-zero epoch is not idle: sessions may predate it")
}
}
func TestHImportConnPreparedFlags(t *testing.T) {
client, server := net.Pipe()
defer client.Close()
defer server.Close()
cn := pool.NewConn(client)
if v := cn.FieldsetPreparedVersion("fs"); v != 0 {
t.Errorf("unprepared fieldset version = %d, want 0", v)
}
if cn.HasPreparedFieldsets() {
t.Error("fresh connection should have no prepared fieldsets")
}
// The first mark stamps the session's epoch; later marks keep it.
cn.MarkFieldsetPrepared("fs", 7, 3)
if v := cn.FieldsetPreparedVersion("fs"); v != 7 {
t.Errorf("prepared fieldset version = %d, want 7", v)
}
if e := cn.FieldsetEpoch(); e != 3 {
t.Errorf("epoch after first mark = %d, want 3", e)
}
cn.MarkFieldsetPrepared("fs2", 8, 9)
if e := cn.FieldsetEpoch(); e != 3 {
t.Errorf("epoch after second mark = %d, want 3 (unchanged)", e)
}
names := cn.PreparedFieldsetNames()
sort.Strings(names)
if !reflect.DeepEqual(names, []string{"fs", "fs2"}) {
t.Errorf("prepared names = %v, want [fs fs2]", names)
}
cn.UnmarkFieldsetPrepared("fs")
if v := cn.FieldsetPreparedVersion("fs"); v != 0 {
t.Errorf("unmarked fieldset version = %d, want 0", v)
}
// Clearing wipes the flags and adopts the given epoch.
cn.ClearPreparedFieldsets(5)
if cn.HasPreparedFieldsets() || cn.FieldsetEpoch() != 5 {
t.Errorf("after clear: has=%v epoch=%d, want none/5",
cn.HasPreparedFieldsets(), cn.FieldsetEpoch())
}
// A session cleared to epoch 5 that prepares again keeps... the epoch of
// that prepare (empty -> non-empty stamps anew).
cn.MarkFieldsetPrepared("fs", 9, 6)
if e := cn.FieldsetEpoch(); e != 6 {
t.Errorf("epoch after re-mark = %d, want 6", e)
}
// Replacing the network connection is a new server session: flags and
// epoch reset.
client2, server2 := net.Pipe()
defer client2.Close()
defer server2.Close()
cn.SetNetConn(client2)
if cn.HasPreparedFieldsets() || cn.FieldsetEpoch() != 0 {
t.Errorf("after SetNetConn: has=%v epoch=%d, want none/0",
cn.HasPreparedFieldsets(), cn.FieldsetEpoch())
}
}
func injectedKinds(cmds []Cmder) []string {
kinds := make([]string, 0, len(cmds))
for _, cmd := range cmds {
switch cmd.(type) {
case *HImportPrepareCmd:
kinds = append(kinds, "prepare")
case *HImportDiscardCmd:
kinds = append(kinds, "discard")
case *HImportDiscardAllCmd:
kinds = append(kinds, "discardall")
default:
kinds = append(kinds, "other")
}
}
return kinds
}
func TestHImportInjectedPrepare(t *testing.T) {
ctx := context.Background()
c := &baseClient{himport: newHImportRegistry()}
cn := pool.NewConn(nil)
set := NewHImportSetCmd(ctx, "k", "fs", "v1", "v2")
// Not an HIMPORT command: nothing to inject.
if inj := c.himportInjectedCmds(ctx, cn, []Cmder{NewStatusCmd(ctx, "ping")}); inj != nil {
t.Errorf("injected for PING = %v, want nil", injectedKinds(inj))
}
// Unregistered fieldset: raw pass-through, nothing to inject.
if inj := c.himportInjectedCmds(ctx, cn, []Cmder{set}); inj != nil {
t.Errorf("injected for unregistered fieldset = %v, want nil", injectedKinds(inj))
}
// Registered but not prepared on this connection: inject PREPARE.
version, _ := c.himport.register("fs", []string{"f1", "f2"})
inj := c.himportInjectedCmds(ctx, cn, []Cmder{set})
if len(inj) != 1 {
t.Fatalf("injected = %v, want one PREPARE", injectedKinds(inj))
}
prep, ok := inj[0].(*HImportPrepareCmd)
if !ok {
t.Fatalf("injected[0] = %T, want *HImportPrepareCmd", inj[0])
}
wantArgs := []interface{}{"himport", "prepare", "fs", "f1", "f2"}
if !reflect.DeepEqual(prep.Args(), wantArgs) {
t.Errorf("injected prepare args = %v, want %v", prep.Args(), wantArgs)
}
if prep.registryVersion != version {
t.Errorf("injected prepare version = %d, want %d", prep.registryVersion, version)
}
// Prepared at the current version: nothing to inject.
cn.MarkFieldsetPrepared("fs", version, 0)
if inj := c.himportInjectedCmds(ctx, cn, []Cmder{set}); len(inj) != 0 {
t.Errorf("injected for prepared connection = %v, want none", injectedKinds(inj))
}
// Fieldset replaced under a new version: the session's old version is
// discarded before the replay, so a failed re-prepare leaves the SET
// answering "no such fieldset" instead of silently writing the old
// version's field names.
c.himport.register("fs", []string{"f3"})
inj = c.himportInjectedCmds(ctx, cn, []Cmder{set})
if kinds := injectedKinds(inj); !reflect.DeepEqual(kinds, []string{"discard", "prepare"}) {
t.Errorf("injected after re-register = %v, want [discard prepare]", kinds)
}
// A connection with no version at all needs no discard.
fresh := pool.NewConn(nil)
inj = c.himportInjectedCmds(ctx, fresh, []Cmder{set})
if kinds := injectedKinds(inj); !reflect.DeepEqual(kinds, []string{"prepare"}) {
t.Errorf("injected for fresh connection = %v, want [prepare]", kinds)
}
}
func TestHImportInjectedBatch(t *testing.T) {
ctx := context.Background()
c := &baseClient{himport: newHImportRegistry()}
cn := pool.NewConn(nil)
c.himport.register("fs", []string{"f1"})
c.himport.register("fs2", []string{"g1"})
// Two SETs on the same fieldset need a single PREPARE; a user-issued
// PREPARE earlier in the batch covers its fieldset.
batch := []Cmder{
NewStatusCmd(ctx, "ping"),
NewHImportPrepareCmd(ctx, "fs2", "g1"),
NewHImportSetCmd(ctx, "k1", "fs", "v"),
NewHImportSetCmd(ctx, "k2", "fs", "v"),
NewHImportSetCmd(ctx, "k3", "fs2", "v"),
}
inj := c.himportInjectedCmds(ctx, cn, batch)
if kinds := injectedKinds(inj); !reflect.DeepEqual(kinds, []string{"prepare"}) {
t.Fatalf("injected = %v, want [prepare]", kinds)
}
if inj[0].(*HImportPrepareCmd).fieldsetName != "fs" {
t.Errorf("injected prepare fieldset = %q, want fs", inj[0].(*HImportPrepareCmd).fieldsetName)
}
// A connection already prepared at the current version needs nothing.
fs, _ := c.himport.lookup("fs")
cn.MarkFieldsetPrepared("fs", fs.version, 0)
if inj := c.himportInjectedCmds(ctx, cn, []Cmder{NewHImportSetCmd(ctx, "k1", "fs", "v")}); len(inj) != 0 {
t.Errorf("injected for prepared connection = %v, want none", injectedKinds(inj))
}
}
func TestHImportInjectedDiscard(t *testing.T) {
ctx := context.Background()
c := &baseClient{himport: newHImportRegistry()}
cn := pool.NewConn(nil)
// Prepare fs and fs2 on this connection, then discard fs through the
// registry (as if the DISCARD executed on another pooled connection).
v1, e1 := c.himport.register("fs", []string{"f"})
cn.MarkFieldsetPrepared("fs", v1, e1)
v2, e2 := c.himport.register("fs2", []string{"g"})
cn.MarkFieldsetPrepared("fs2", v2, e2)
c.himport.discard("fs")
// The session still holds fs: the next HIMPORT command replays the
// DISCARD ahead of itself.
inj := c.himportInjectedCmds(ctx, cn, []Cmder{NewHImportSetCmd(ctx, "k", "fs2", "v")})
if kinds := injectedKinds(inj); !reflect.DeepEqual(kinds, []string{"discard"}) {
t.Fatalf("injected = %v, want [discard]", kinds)
}
if inj[0].(*HImportDiscardCmd).fieldsetName != "fs" {
t.Errorf("injected discard fieldset = %q, want fs", inj[0].(*HImportDiscardCmd).fieldsetName)
}
// A connection that never prepared fs replays nothing.
other := pool.NewConn(nil)
otherFs2, _ := c.himport.lookup("fs2")
other.MarkFieldsetPrepared("fs2", otherFs2.version, c.himport.epoch())
if inj := c.himportInjectedCmds(ctx, other, []Cmder{NewHImportSetCmd(ctx, "k", "fs2", "v")}); len(inj) != 0 {
t.Errorf("injected on unaffected connection = %v, want none", injectedKinds(inj))
}
// Cleanup piggybacks on any HIMPORT command, not only SET.
inj = c.himportInjectedCmds(ctx, cn, []Cmder{NewHImportDiscardCmd(ctx, "fs2")})
if kinds := injectedKinds(inj); !reflect.DeepEqual(kinds, []string{"discard"}) {
t.Errorf("injected before user discard = %v, want [discard]", kinds)
}
// Non-HIMPORT traffic never triggers cleanup.
if inj := c.himportInjectedCmds(ctx, cn, []Cmder{NewStatusCmd(ctx, "ping")}); inj != nil {
t.Errorf("injected for PING = %v, want nil", injectedKinds(inj))
}
// Re-registering the name clears the tombstone. This session still
// holds the old version (its cleanup discard never ran), so the replay
// discards it before the PREPARE — a failed re-prepare must not leave
// the old field names live.
c.himport.register("fs", []string{"f-new"})
inj = c.himportInjectedCmds(ctx, cn, []Cmder{NewHImportSetCmd(ctx, "k", "fs", "v")})
if kinds := injectedKinds(inj); !reflect.DeepEqual(kinds, []string{"discard", "prepare"}) {
t.Errorf("injected after re-register = %v, want [discard prepare]", kinds)
}
}
func TestHImportInjectedDiscardAll(t *testing.T) {
ctx := context.Background()
c := &baseClient{himport: newHImportRegistry()}
cn := pool.NewConn(nil)
v1, e1 := c.himport.register("fs", []string{"f"})
cn.MarkFieldsetPrepared("fs", v1, e1)
c.himport.discardAll()
// Session predates the epoch: one DISCARDALL wipes it. The fieldset is
// gone from the registry, so no PREPARE follows.
inj := c.himportInjectedCmds(ctx, cn, []Cmder{NewHImportSetCmd(ctx, "k", "fs", "v")})
if kinds := injectedKinds(inj); !reflect.DeepEqual(kinds, []string{"discardall"}) {
t.Fatalf("injected = %v, want [discardall]", kinds)
}
if inj[0].(*HImportDiscardAllCmd).registryEpoch != c.himport.epoch() {
t.Errorf("injected discardall epoch = %d, want %d",
inj[0].(*HImportDiscardAllCmd).registryEpoch, c.himport.epoch())
}
// A fieldset registered after the discard-all is replayed alongside the
// session wipe, ignoring the connection's stale prepared flag.
v2, _ := c.himport.register("fs", []string{"f2"})
cn.MarkFieldsetPrepared("fs", v2, 0) // stale epoch: marked before the wipe
inj = c.himportInjectedCmds(ctx, cn, []Cmder{NewHImportSetCmd(ctx, "k", "fs", "v")})
if kinds := injectedKinds(inj); !reflect.DeepEqual(kinds, []string{"discardall", "prepare"}) {
t.Fatalf("injected = %v, want [discardall prepare]", kinds)
}
// A fresh session (no prepared fieldsets) has nothing to wipe.
fresh := pool.NewConn(nil)
inj = c.himportInjectedCmds(ctx, fresh, []Cmder{NewHImportSetCmd(ctx, "k", "fs", "v")})
if kinds := injectedKinds(inj); !reflect.DeepEqual(kinds, []string{"prepare"}) {
t.Errorf("injected on fresh connection = %v, want [prepare]", kinds)
}
// A session prepared under the current epoch is not wiped again.
current := pool.NewConn(nil)
fs, _ := c.himport.lookup("fs")
current.MarkFieldsetPrepared("fs", fs.version, c.himport.epoch())
if inj := c.himportInjectedCmds(ctx, current, []Cmder{NewHImportSetCmd(ctx, "k", "fs", "v")}); len(inj) != 0 {
t.Errorf("injected on current-epoch connection = %v, want none", injectedKinds(inj))
}
}
func TestHImportAfterCmd(t *testing.T) {
ctx := context.Background()
c := &baseClient{himport: newHImportRegistry()}
cn := pool.NewConn(nil)
// A successful user-issued PREPARE registers the fieldset and marks the
// executing connection.
prep := NewHImportPrepareCmd(ctx, "fs", "f1", "f2")
c.himportAfterCmd(cn, prep)
fs, ok := c.himport.lookup("fs")
if !ok || !reflect.DeepEqual(fs.fields, []string{"f1", "f2"}) {
t.Fatalf("registry after prepare = %+v, %v; want fields [f1 f2]", fs, ok)
}
if cn.FieldsetPreparedVersion("fs") != fs.version {
t.Error("executing connection should be marked at the registered version")
}
// DISCARD unregisters, tombstones, unmarks the executing connection,
// and reports the registry lifecycle: 1 for a registered fieldset even
// when the executing session did not hold it (server replied 0).
discard := NewHImportDiscardCmd(ctx, "fs")
discard.SetVal(0)
c.himportAfterCmd(cn, discard)
if _, ok := c.himport.lookup("fs"); ok {
t.Error("fieldset should be unregistered after discard")
}
if cn.FieldsetPreparedVersion("fs") != 0 {
t.Error("connection flag should be gone after discard")
}
if _, tombs := c.himport.cleanupSnapshot(); len(tombs) != 1 {
t.Errorf("tombstones after discard = %v, want [fs]", tombs)
}
if discard.Val() != 1 {
t.Errorf("discard value = %d, want 1 (fieldset was registered)", discard.Val())
}
// An unregistered name keeps the server's session reply.
rawDiscard := NewHImportDiscardCmd(ctx, "raw-only")
rawDiscard.SetVal(1)
c.himportAfterCmd(cn, rawDiscard)
if rawDiscard.Val() != 1 {
t.Errorf("unregistered discard value = %d, want the server reply 1", rawDiscard.Val())
}
// DISCARDALL clears everything, moves the connection to the new epoch,
// and reports the number of registered fieldsets removed.
c.himportAfterCmd(cn, NewHImportPrepareCmd(ctx, "fs1", "a"))
c.himportAfterCmd(cn, NewHImportPrepareCmd(ctx, "fs2", "b"))
discardAll := NewHImportDiscardAllCmd(ctx)
discardAll.SetVal(0)
c.himportAfterCmd(cn, discardAll)
if !c.himport.empty() {
t.Error("registry should be empty after discardall")
}
if discardAll.Val() != 2 {
t.Errorf("discardall value = %d, want 2 (registered fieldsets removed)", discardAll.Val())
}
if cn.HasPreparedFieldsets() {
t.Error("connection flags should be gone after discardall")
}
if cn.FieldsetEpoch() != c.himport.epoch() {
t.Errorf("connection epoch = %d, want %d", cn.FieldsetEpoch(), c.himport.epoch())
}
}
func TestHImportAfterBatch(t *testing.T) {
ctx := context.Background()
c := &baseClient{himport: newHImportRegistry()}
cn := pool.NewConn(nil)
c.himport.register("fs", []string{"f1"})
// An injected PREPARE failure is the root cause of the dependent SET's
// "no such fieldset" reply.
prep := NewHImportPrepareCmd(ctx, "fs", "f1")
prepErr := proto.RedisError("ERR duplicate field name in fieldset")
prep.SetErr(prepErr)
set := NewHImportSetCmd(ctx, "k", "fs", "v")
set.SetErr(proto.RedisError("ERR no such fieldset"))
otherSet := NewHImportSetCmd(ctx, "k2", "other", "v")
otherSet.SetErr(proto.RedisError("ERR no such fieldset"))
c.himportAfterBatch(cn, []Cmder{prep}, []Cmder{set, otherSet})
if !errors.Is(set.Err(), prepErr) {
t.Errorf("dependent set error = %v, want the prepare root cause", set.Err())
}
if errors.Is(otherSet.Err(), prepErr) {
t.Error("unrelated set must keep its own error")
}
// A registered fieldset that came back "no such fieldset" without a
// failed injected PREPARE means the session lost it — and other
// sessions may have been wiped by the same event. The fieldset version
// is bumped once, staling every connection's mark, so any retry
// re-prepares wherever it lands.
fs, _ := c.himport.lookup("fs")
cn.MarkFieldsetPrepared("fs", fs.version, 0)
lost := NewHImportSetCmd(ctx, "k3", "fs", "v")
lost.SetErr(proto.RedisError("ERR no such fieldset"))
lost2 := NewHImportSetCmd(ctx, "k4", "fs", "v")
lost2.SetErr(proto.RedisError("ERR no such fieldset"))
c.himportAfterBatch(cn, nil, []Cmder{lost, lost2})
bumped, _ := c.himport.lookup("fs")
if bumped.version != fs.version+1 {
t.Errorf("fieldset version = %d, want %d (bumped exactly once for two failed sets)",
bumped.version, fs.version+1)
}
if cn.FieldsetPreparedVersion("fs") == bumped.version {
t.Error("the connection's mark must be stale against the bumped version")
}
// Successful user-issued commands in the batch update the registry.
userPrep := NewHImportPrepareCmd(ctx, "fs3", "x")
c.himportAfterBatch(cn, nil, []Cmder{userPrep})
if _, ok := c.himport.lookup("fs3"); !ok {
t.Error("successful in-batch prepare should register the fieldset")
}
failedPrep := NewHImportPrepareCmd(ctx, "fs4", "x")
failedPrep.SetErr(proto.RedisError("ERR duplicate field name in fieldset"))
c.himportAfterBatch(cn, nil, []Cmder{failedPrep})
if _, ok := c.himport.lookup("fs4"); ok {
t.Error("failed in-batch prepare must not register the fieldset")
}
}
// Every pooled client constructor must wire the HIMPORT registry: without it,
// HImportPrepare silently skips registration and pooled HImportSet replay
// never engages.
func TestHImportRegistryWiring(t *testing.T) {
client := NewClient(&Options{Addr: "127.0.0.1:0"})
defer client.Close()
if client.himport == nil {
t.Error("NewClient must initialize the HIMPORT registry")
}
// Conn and Tx borrow connections from the client's pool and return them:
// they must share the parent registry so prepared flags stay coherent.
conn := client.Conn()
defer conn.Close()
if conn.himport != client.himport {
t.Error("Client.Conn() must share the parent HIMPORT registry")
}
tx := client.newTx()
defer tx.Close(context.Background())
if tx.himport != client.himport {
t.Error("newTx must share the parent HIMPORT registry")
}
failover := NewFailoverClient(&FailoverOptions{
MasterName: "mymaster",
SentinelAddrs: []string{"127.0.0.1:0"},
})
defer failover.Close()
if failover.himport == nil {
t.Error("NewFailoverClient must initialize the HIMPORT registry")
}
// Cluster node clients — replicas included, roles change with the
// topology — share the cluster-wide registry.
cluster := NewClusterClient(&ClusterOptions{Addrs: []string{"127.0.0.1:0"}})
defer cluster.Close()
if cluster.himport == nil {
t.Fatal("NewClusterClient must initialize the HIMPORT registry")
}
node, err := cluster.nodes.GetOrCreate("127.0.0.1:0")
if err != nil {
t.Fatalf("GetOrCreate: %v", err)
}
if node.Client.himport != cluster.himport {
t.Error("cluster node clients must share the cluster-wide HIMPORT registry")
}
// Ring shard clients share the ring-wide registry.
ring := NewRing(&RingOptions{Addrs: map[string]string{"shard1": "127.0.0.1:0"}})
defer ring.Close()
if ring.opt.himport == nil {
t.Fatal("NewRing must initialize the HIMPORT registry")
}
shards := ring.sharding.List()
if len(shards) == 0 {
t.Fatal("ring has no shards")
}
if shards[0].Client.himport != ring.opt.himport {
t.Error("ring shard clients must share the ring-wide HIMPORT registry")
}
// NewRing copies the options: rings built from one caller-owned struct
// must not share or clobber each other's registry.
sharedOpts := &RingOptions{Addrs: map[string]string{"shard1": "127.0.0.1:0"}}
ringA := NewRing(sharedOpts)
defer ringA.Close()
ringB := NewRing(sharedOpts)
defer ringB.Close()
if ringA.opt.himport == ringB.opt.himport {
t.Error("rings built from the same options must not share a registry")
}
if sharedOpts.himport != nil {
t.Error("NewRing must not write the registry into the caller's options")
}
}
// Fan-out copies carry a pre-assigned version/epoch: they mark or wipe the
// executing connection but must not touch the shared registry again.
func TestHImportFanOutCopyBookkeeping(t *testing.T) {
ctx := context.Background()
c := &baseClient{himport: newHImportRegistry()}
cn := pool.NewConn(nil)
version, epoch := c.himport.register("fs", []string{"f"})
copyPrep := NewHImportPrepareCmd(ctx, "fs", "f")
copyPrep.registryVersion = version
copyPrep.registryEpoch = epoch
c.himportAfterCmd(cn, copyPrep)
if fs, _ := c.himport.lookup("fs"); fs.version != version {
t.Errorf("fan-out prepare copy bumped the registry version to %d", fs.version)
}
if cn.FieldsetPreparedVersion("fs") != version {
t.Error("fan-out prepare copy must mark the executing connection")
}
newEpoch, _ := c.himport.discardAll()
copyDA := NewHImportDiscardAllCmd(ctx)
copyDA.registryEpoch = newEpoch
c.himportAfterCmd(cn, copyDA)
if got := c.himport.epoch(); got != newEpoch {
t.Errorf("fan-out discardall copy bumped the epoch to %d, want %d", got, newEpoch)
}
if cn.FieldsetEpoch() != newEpoch || cn.HasPreparedFieldsets() {
t.Error("fan-out discardall copy must wipe the executing connection at the given epoch")
}
}
// A deterministic server rejection withdraws the registration, but only at
// the failed version — a concurrent re-registration survives — and leaves a
// tombstone: the fan-out may have prepared some sessions before another
// master rejected it, and those sessions must be cleaned lazily.
func TestHImportRegistryDiscardVersion(t *testing.T) {
r := newHImportRegistry()
v1, _ := r.register("fs", []string{"f"})
r.discardVersion("fs", v1)
if _, ok := r.lookup("fs"); ok {
t.Error("discardVersion must delete the entry at the matching version")
}
if _, tombs := r.cleanupSnapshot(); len(tombs) != 1 || tombs[0] != "fs" {
t.Errorf("discardVersion must leave a tombstone for partial fan-out cleanup, got %v", tombs)
}
v2, _ := r.register("fs", []string{"g"})
r.discardVersion("fs", v2-1) // stale version: no-op
if fs, ok := r.lookup("fs"); !ok || fs.version != v2 {
t.Error("discardVersion with a stale version must not clobber the current entry")
}
if _, tombs := r.cleanupSnapshot(); len(tombs) != 0 {
t.Errorf("stale discardVersion must not tombstone (re-register cleared it), got %v", tombs)
}
}
// refreshVersion invalidates every connection's mark by bumping the version
// while keeping the fields — used on "no such fieldset" so a retry
// re-prepares on whichever connection it lands (mass session loss may have
// staled more marks than the reporting connection's).
func TestHImportRegistryRefreshVersion(t *testing.T) {
r := newHImportRegistry()
v1, _ := r.register("fs", []string{"f1", "f2"})
r.refreshVersion("fs", v1)
fs, ok := r.lookup("fs")
if !ok || fs.version <= v1 {
t.Fatalf("version after refresh = %d, want > %d", fs.version, v1)
}
if !reflect.DeepEqual(fs.fields, []string{"f1", "f2"}) {
t.Errorf("fields after refresh = %v, must be unchanged", fs.fields)
}
// A stale expected version must not disturb a concurrent
// re-registration.
v2, _ := r.register("fs", []string{"g"})
r.refreshVersion("fs", v2-1)
if fs, _ := r.lookup("fs"); fs.version != v2 {
t.Errorf("version = %d, want %d (stale refresh must no-op)", fs.version, v2)
}
// Unregistered names are ignored.
r.refreshVersion("missing", 1)
if _, ok := r.lookup("missing"); ok {
t.Error("refresh must not create entries")
}
}
func TestHImportShouldRetrySet(t *testing.T) {
ctx := context.Background()
c := &baseClient{himport: newHImportRegistry()}
noSuchFieldset := proto.RedisError("ERR no such fieldset")
set := NewHImportSetCmd(ctx, "k", "fs", "v")
// Unregistered fieldset: a retry would fail identically.
if c.himportShouldRetrySet(set, noSuchFieldset) {
t.Error("retry must be refused for unregistered fieldsets")
}
c.himport.register("fs", []string{"f1"})
// Wrong error: nothing to recover.
if c.himportShouldRetrySet(set, proto.RedisError("ERR value count does not match fieldset field count")) {
t.Error("retry must only react to no-such-fieldset errors")
}
// Wrong command type: nothing to recover.
if c.himportShouldRetrySet(NewStatusCmd(ctx, "ping"), noSuchFieldset) {
t.Error("retry must only react to HIMPORT SET")
}
// Registered fieldset: a retry may succeed (the stale flag is
// invalidated inside _process while the connection is held; the RESET
// recovery in TestHImportLazyReplay pins that end to end).
if !c.himportShouldRetrySet(set, noSuchFieldset) {
t.Error("retry should be allowed for registered fieldsets")
}
}
internal/pool/conn.go
+91
View File
@@ -109,6 +109,18 @@ type Conn struct {
expiresAt time.Time
poolName string // Name of the pool this connection belongs to (for metrics)
// preparedFieldsets tracks HIMPORT fieldsets prepared on this
// connection's current server session: fieldset name -> client-side
// registry version. The server drops fieldsets when the session ends,
// so the map is cleared whenever the underlying network connection is
// replaced. preparedFieldsetsEpoch records the registry's discard-all
// epoch the session was prepared under; a session behind the current
// epoch replays HIMPORT DISCARDALL before its next HIMPORT command.
// Guarded by preparedFieldsetsMu; the map is nil until first use.
preparedFieldsetsMu sync.Mutex
preparedFieldsets map[string]uint64
preparedFieldsetsEpoch uint64
// When a goroutine closes a connection, it usually knows the reason, so closeReason is not needed.
// closeReason is only used when an in-use connection is closed by another goroutine,
// to inform the goroutine using the connection why the connection was closed.
@@ -661,6 +673,85 @@ func (cn *Conn) SetNetConn(netConn net.Conn) {
cn.readerMu.Unlock()
cn.bw.Reset(netConn)
// A new socket is a new server session with no HIMPORT fieldsets and
// nothing left to discard.
cn.ClearPreparedFieldsets(0)
}
// FieldsetPreparedVersion returns the client-side registry version at which
// the named HIMPORT fieldset was prepared on this connection's current server
// session, or 0 if it was not prepared on it (registry versions start at 1).
func (cn *Conn) FieldsetPreparedVersion(name string) uint64 {
cn.preparedFieldsetsMu.Lock()
version := cn.preparedFieldsets[name]
cn.preparedFieldsetsMu.Unlock()
return version
}
// MarkFieldsetPrepared records that the named HIMPORT fieldset was prepared
// on this connection's current server session at the given registry version.
// A session acquiring its first fieldset adopts the given discard-all epoch
// (fieldsets prepared after an HIMPORT DISCARDALL are not subject to it);
// the epoch never moves backwards, so a mark carrying an older snapshot
// cannot regress a session already wiped at a newer epoch.
func (cn *Conn) MarkFieldsetPrepared(name string, version, epoch uint64) {
cn.preparedFieldsetsMu.Lock()
if len(cn.preparedFieldsets) == 0 {
cn.preparedFieldsets = make(map[string]uint64)
if epoch > cn.preparedFieldsetsEpoch {
cn.preparedFieldsetsEpoch = epoch
}
}
cn.preparedFieldsets[name] = version
cn.preparedFieldsetsMu.Unlock()
}
// UnmarkFieldsetPrepared forgets that the named HIMPORT fieldset was prepared
// on this connection, forcing a replay before the next HIMPORT SET using it.
func (cn *Conn) UnmarkFieldsetPrepared(name string) {
cn.preparedFieldsetsMu.Lock()
delete(cn.preparedFieldsets, name)
cn.preparedFieldsetsMu.Unlock()
}
// HasPreparedFieldsets reports whether any HIMPORT fieldset is prepared on
// this connection's current server session.
func (cn *Conn) HasPreparedFieldsets() bool {
cn.preparedFieldsetsMu.Lock()
n := len(cn.preparedFieldsets)
cn.preparedFieldsetsMu.Unlock()
return n > 0
}
// PreparedFieldsetNames returns the names of the HIMPORT fieldsets prepared
// on this connection's current server session.
func (cn *Conn) PreparedFieldsetNames() []string {
cn.preparedFieldsetsMu.Lock()
names := make([]string, 0, len(cn.preparedFieldsets))
for name := range cn.preparedFieldsets {
names = append(names, name)
}
cn.preparedFieldsetsMu.Unlock()
return names
}
// FieldsetEpoch returns the discard-all epoch this connection's prepared
// fieldsets belong to (0 when none were ever prepared on the session).
func (cn *Conn) FieldsetEpoch() uint64 {
cn.preparedFieldsetsMu.Lock()
epoch := cn.preparedFieldsetsEpoch
cn.preparedFieldsetsMu.Unlock()
return epoch
}
// ClearPreparedFieldsets forgets all HIMPORT fieldsets prepared on this
// connection and records the discard-all epoch the wipe corresponds to.
func (cn *Conn) ClearPreparedFieldsets(epoch uint64) {
cn.preparedFieldsetsMu.Lock()
cn.preparedFieldsets = nil
cn.preparedFieldsetsEpoch = epoch
cn.preparedFieldsetsMu.Unlock()
}
// GetNetConn safely returns the current network connection using atomic load (lock-free).
internal/pool/pool_sticky.go
+8
View File
@@ -102,6 +102,14 @@ func (p *StickyConnPool) Put(ctx context.Context, cn *Conn) {
p.freeConn(ctx, cn)
}
}()
// A connection marked for removal on release (it may hold unread
// replies) must not be served to the next Get: record it as a bad
// connection — exactly like Remove — so Get refuses and the underlying
// connection is removed from the parent pool when the sticky pool
// unwinds (the parent's Put honors the same mark).
if reason := cn.CloseOnPutReason(); reason != "" {
p._badConnError.Store(BadConnError{wrapped: errors.New(reason)})
}
p.ch <- cn
}
internal/pool/pool_sticky_close_on_put_test.go
+72
View File
@@ -0,0 +1,72 @@
package pool
import (
"context"
"errors"
"net"
"testing"
)
// stubPooler records removals; Get hands out a fixed conn.
type stubPooler struct {
cn *Conn
removed bool
}
func (s *stubPooler) NewConn(context.Context) (*Conn, error) { return s.cn, nil }
func (s *stubPooler) CloseConn(context.Context, *Conn, string, string) error {
return nil
}
func (s *stubPooler) Get(context.Context) (*Conn, error) { return s.cn, nil }
func (s *stubPooler) Put(context.Context, *Conn) {}
func (s *stubPooler) Remove(_ context.Context, _ *Conn, _ error) {
s.removed = true
}
func (s *stubPooler) RemoveWithoutTurn(ctx context.Context, cn *Conn, reason error) {
s.Remove(ctx, cn, reason)
}
func (s *stubPooler) Len() int { return 1 }
func (s *stubPooler) IdleLen() int { return 0 }
func (s *stubPooler) Size() int { return 1 }
func (s *stubPooler) Stats() *Stats { return &Stats{} }
func (s *stubPooler) Close() error { return nil }
func (s *stubPooler) AddPoolHook(PoolHook) {}
func (s *stubPooler) RemovePoolHook(PoolHook) {}
// A connection marked for removal on release must not be served to the next
// sticky Get: it may hold unread replies, and reusing it desyncs the reply
// stream for the Conn/Tx that owns the sticky pool.
func TestStickyConnPoolPutHonorsCloseOnPut(t *testing.T) {
client, server := net.Pipe()
defer client.Close()
defer server.Close()
stub := &stubPooler{cn: NewConn(client)}
p := NewStickyConnPool(stub)
ctx := context.Background()
cn, err := p.Get(ctx)
if err != nil {
t.Fatalf("first get: %v", err)
}
cn.MarkCloseOnPut("unread replies")
p.Put(ctx, cn)
if _, err := p.Get(ctx); err == nil {
t.Fatal("second get must refuse a connection marked close-on-put")
} else {
var badConn BadConnError
if !errors.As(err, &badConn) {
t.Fatalf("second get error = %v, want BadConnError", err)
}
}
// Unwinding the sticky pool removes the connection from the parent.
if err := p.Close(); err != nil {
t.Fatalf("close: %v", err)
}
if !stub.removed {
t.Error("underlying connection must be removed from the parent pool")
}
}
osscluster.go
+73 -3
View File
@@ -1155,6 +1155,12 @@ type ClusterClient struct {
cmdInfoResolver *commandInfoResolver
cmdable
hooksMixin
// himport is the cluster-wide HIMPORT fieldset registry, shared with
// every node client (masters and replicas alike — roles change with the
// topology) so any connection serving an HIMPORT SET can lazily replay
// the PREPARE (see himport.go, himport_cluster.go).
himport *himportRegistry
}
// NewClusterClient returns a Redis Cluster client as described in
@@ -1167,10 +1173,18 @@ func NewClusterClient(opt *ClusterOptions) *ClusterClient {
opt.init()
c := &ClusterClient{
opt: opt,
nodes: newClusterNodes(opt),
opt: opt,
nodes: newClusterNodes(opt),
himport: newHImportRegistry(),
}
// Every node client shares the cluster-wide fieldset registry, replicas
// included: a promoted replica's connections carry no prepared flags, so
// the first HIMPORT SET routed to it replays the PREPARE lazily.
c.nodes.OnNewNode(func(nodeClient *Client) {
nodeClient.himport = c.himport
})
c.cmdsInfoCache = newCmdsInfoCache(c.cmdsInfo)
c.state = newClusterStateHolder(c.loadState, opt.ClusterStateReloadInterval)
@@ -1738,7 +1752,17 @@ func (c *ClusterClient) processPipelineNode(
func (c *ClusterClient) processPipelineNodeConn(
ctx context.Context, node *clusterNode, cn *pool.Conn, cmds []Cmder, failedCmds *cmdsMap,
) error {
// HIMPORT bookkeeping: pending discards for this session and PREPAREs
// for registered fieldsets the batch references get written ahead of
// the batch (see himport.go).
injected := node.Client.himportInjectedCmds(ctx, cn, cmds)
if err := cn.WithWriter(c.context(ctx), c.opt.WriteTimeout, func(wr *proto.Writer) error {
for _, ic := range injected {
if err := writeCmd(wr, ic); err != nil {
return err
}
}
return writeCmds(wr, cmds)
}); err != nil {
if isBadConn(err, false, node.Client.getAddr()) {
@@ -1752,7 +1776,34 @@ func (c *ClusterClient) processPipelineNodeConn(
}
return cn.WithReader(c.context(ctx), c.opt.ReadTimeout, func(rd *proto.Reader) error {
return c.pipelineReadCmds(ctx, node, rd, cmds, failedCmds)
if err := node.Client.himportReadInjectedReplies(ctx, cn, rd, injected); err != nil {
// Transport error with the batch replies unread: same handling
// as a write error — the batch may be retried on a fresh
// connection.
if isBadConn(err, false, node.Client.getAddr()) {
node.MarkAsFailing()
}
if shouldRetry(err, true) && !cmdsContainNoRetry(cmds) {
_ = c.mapCmdsByNode(ctx, failedCmds, cmds)
}
setCmdsErr(cmds, err)
return err
}
err := c.pipelineReadCmds(ctx, node, rd, cmds, failedCmds)
if err == nil || isRedisError(err) {
node.Client.himportAfterBatch(cn, injected, cmds)
// SETs of registered fieldsets that lost their session state
// re-queue for the next attempt, which re-prepares lazily —
// the cluster equivalent of himportRetryFailedSets, bounded by
// the pipeline's attempt budget. A non-nil redis error here
// means pipelineReadCmds already re-queued the whole batch
// (retryable first-command error); adding the SETs again would
// duplicate them in the next attempt.
if err == nil {
c.himportRequeueFailedSets(ctx, cmds, failedCmds)
}
}
return err
})
}
@@ -2109,7 +2160,17 @@ func (c *ClusterClient) processTxPipelineNode(
func (c *ClusterClient) processTxPipelineNodeConn(
ctx context.Context, node *clusterNode, cn *pool.Conn, wire []Cmder, cmds []Cmder, asking bool,
) *txOutcome {
// HIMPORT bookkeeping: pending discards and PREPAREs for registered
// fieldsets the transaction references get written ahead of the wire
// batch (before ASKING/MULTI; the session state is visible at EXEC).
injected := node.Client.himportInjectedCmds(ctx, cn, cmds)
if err := cn.WithWriter(c.context(ctx), c.opt.WriteTimeout, func(wr *proto.Writer) error {
for _, ic := range injected {
if err := writeCmd(wr, ic); err != nil {
return err
}
}
return writeCmds(wr, wire)
}); err != nil {
// Write failure: re-route the whole tx on a fresh connection.
@@ -2121,7 +2182,16 @@ func (c *ClusterClient) processTxPipelineNodeConn(
var outcome *txOutcome
readErr := cn.WithReader(c.context(ctx), c.opt.ReadTimeout, func(rd *proto.Reader) error {
if err := node.Client.himportReadInjectedReplies(ctx, cn, rd, injected); err != nil {
// Transport error with the tx replies unread; the batch was
// written and may have committed — fatal, discard the conn.
outcome = c.txReadFatal(err)
return nil
}
outcome = c.readTxPipelineReplies(ctx, node, cn, rd, cmds, asking)
if outcome != nil && outcome.kind == txSuccess {
node.Client.himportAfterBatch(cn, injected, cmds)
}
return nil
})
redis.go
+170 -12
View File
@@ -366,6 +366,11 @@ type baseClient struct {
// streamingCredentialsManager is used to manage streaming credentials
streamingCredentialsManager *streaming.Manager
// himport is the client-side registry of HIMPORT fieldsets, used to
// lazily replay HIMPORT PREPARE onto pooled connections (see himport.go).
// Shared by clones and by Conn instances derived from the same pool.
himport *himportRegistry
}
func (c *baseClient) clone() *baseClient {
@@ -381,6 +386,7 @@ func (c *baseClient) clone() *baseClient {
pushProcessor: c.pushProcessor,
maintNotificationsManager: maintNotificationsManager,
streamingCredentialsManager: c.streamingCredentialsManager,
himport: c.himport,
}
return clone
}
@@ -472,8 +478,9 @@ func (c *baseClient) reAuthConnection() func(poolCn *pool.Conn, credentials auth
connPool := pool.NewSingleConnPool(c.connPool, poolCn)
// Pass hooks so that reauth commands are recorded/traced
cn := newConn(c.opt, connPool, &c.hooksMixin)
// Pass hooks so that reauth commands are recorded/traced; share the
// HIMPORT registry for the same reason as in initConn.
cn := newConn(c.opt, connPool, &c.hooksMixin, c.himport)
if username != "" {
err = cn.AuthACL(ctx, username, password).Err()
@@ -587,7 +594,12 @@ func (c *baseClient) initConn(ctx context.Context, cn *pool.Conn) error {
// If we fail, we must transition to CLOSED
var initErr error
connPool := pool.NewSingleConnPool(c.connPool, cn)
conn := newConn(c.opt, connPool, &c.hooksMixin)
// The handshake Conn (handed to OnConnect) must share the client's
// HIMPORT registry: a private registry restarts versions at 1, so an
// OnConnect prepare would mark the pooled connection with a version
// number that collides with the client registry's and silently skips
// the replay of a different fieldset definition.
conn := newConn(c.opt, connPool, &c.hooksMixin, c.himport)
username, password := "", ""
if c.opt.StreamingCredentialsProvider != nil {
@@ -874,7 +886,9 @@ func (c *baseClient) process(ctx context.Context, cmd Cmder) error {
var lastErr error
totalAttempts := 0
for attempt := 0; attempt <= c.opt.MaxRetries; attempt++ {
maxRetries := c.opt.MaxRetries
himportRetried := false
for attempt := 0; attempt <= maxRetries; attempt++ {
totalAttempts++
attempt := attempt
@@ -882,6 +896,20 @@ func (c *baseClient) process(ctx context.Context, cmd Cmder) error {
if cn != nil {
lastConn = cn
}
// A "no such fieldset" reply for a registered fieldset means the
// connection lost its server session state (e.g. RESET, concurrent
// discard). The stale prepared flag was invalidated inside _process
// while the connection was still held; grant a single extra attempt
// so the retry re-prepares lazily on whichever connection it lands.
if err != nil && !retry && !himportRetried && !cmd.NoRetry() &&
c.himportShouldRetrySet(cmd, err) {
himportRetried = true
if attempt == maxRetries {
maxRetries++
}
lastErr = err
continue
}
// Don't retry if command explicitly disables retries (e.g., RawWriteToCmd
// which writes directly to an io.Writer and cannot undo partial writes)
if err == nil || !retry || cmd.NoRetry() {
@@ -994,7 +1022,20 @@ func (c *baseClient) _process(ctx context.Context, cmd Cmder, attempt int) (bool
internal.Logger.Printf(ctx, "push: error processing pending notifications before command: %v", err)
}
// HIMPORT bookkeeping: pending discards for this session and the
// PREPARE for an HIMPORT SET's registered fieldset are written in
// the same round trip, right before the command.
var injected []Cmder
if _, ok := cmd.(himportCmder); ok {
injected = c.himportInjectedCmds(ctx, cn, []Cmder{cmd})
}
if err := cn.WithWriter(c.context(ctx), c.opt.WriteTimeout, func(wr *proto.Writer) error {
for _, ic := range injected {
if err := writeCmd(wr, ic); err != nil {
return err
}
}
return writeCmd(wr, cmd)
}); err != nil {
retryTimeout.Store(1)
@@ -1005,7 +1046,44 @@ func (c *baseClient) _process(ctx context.Context, cmd Cmder, attempt int) (bool
if err := c.processPendingPushNotificationWithReader(ctx, cn, rd); err != nil {
internal.Logger.Printf(ctx, "push: error processing pending notifications before reading reply: %v", err)
}
return cmd.readReply(rd)
if len(injected) > 0 {
if err := c.himportReadInjectedReplies(ctx, cn, rd, injected); err != nil {
return err
}
// A push notification can arrive between the injected
// replies and the command reply; drain again so the
// reply read below does not consume it as the command's.
if err := c.processPendingPushNotificationWithReader(ctx, cn, rd); err != nil {
internal.Logger.Printf(ctx, "push: error processing pending notifications before reading reply: %v", err)
}
}
err := cmd.readReply(rd)
// Assert the command type before touching the error: the
// errors.As chain inside himportNoSuchFieldset allocates, and
// this is the per-command hot path.
if set, ok := cmd.(*HImportSetCmd); ok && himportNoSuchFieldset(err) {
// A failed injected PREPARE is the root cause of the
// command's "no such fieldset" reply (drained above).
for _, ic := range injected {
if prep, ok := ic.(*HImportPrepareCmd); ok &&
prep.fieldsetName == set.fieldsetName && prep.Err() != nil {
err = prep.Err()
break
}
}
// The session lost a registered fieldset the flags claim is
// prepared — and the same event (failover, cross-region
// switch, reset storm) may have wiped other sessions whose
// flags also still look current. Bump the fieldset version
// so every connection re-prepares before its next use,
// wherever the retry granted by process() lands.
if himportNoSuchFieldset(err) {
if fs, registered := c.himport.lookup(set.fieldsetName); registered {
c.himport.refreshVersion(set.fieldsetName, fs.version)
}
}
}
return err
}); err != nil {
if cmd.readTimeout() == nil {
retryTimeout.Store(1)
@@ -1015,6 +1093,9 @@ func (c *baseClient) _process(ctx context.Context, cmd Cmder, attempt int) (bool
return err
}
if hc, ok := cmd.(himportCmder); ok {
c.himportAfterCmd(cn, hc)
}
return nil
}); err != nil {
retry := shouldRetry(err, retryTimeout.Load() == 1)
@@ -1229,21 +1310,67 @@ func (c *baseClient) pipelineProcessCmds(
internal.Logger.Printf(ctx, "push: error processing pending notifications before writing pipeline: %v", err)
}
// HIMPORT bookkeeping: pending discards for this session and PREPAREs
// for registered fieldsets the batch references get written ahead of
// the batch.
injected := c.himportInjectedCmds(ctx, cn, cmds)
if err := cn.WithWriter(c.context(ctx), c.opt.WriteTimeout, func(wr *proto.Writer) error {
for _, ic := range injected {
if err := writeCmd(wr, ic); err != nil {
return err
}
}
return writeCmds(wr, cmds)
}); err != nil {
setCmdsErr(cmds, err)
return true, err
}
var readErr error
if err := cn.WithReader(c.context(ctx), c.opt.ReadTimeout, func(rd *proto.Reader) error {
if err := c.himportReadInjectedReplies(ctx, cn, rd, injected); err != nil {
// Transport error with every batch reply unreadXX: stamp the
// batch like a write failure. The outer retry loop stamps only
// on its exit branch, not when attempts run out, so without
// this a batch that keeps dying here would surface an Exec
// error while every command still reports Err() == nil.
setCmdsErr(cmds, err)
return err
}
// read all replies
return c.pipelineReadCmds(ctx, cn, rd, cmds)
readErr = c.pipelineReadCmds(ctx, cn, rd, cmds)
if readErr != nil && !isRedisError(readErr) {
return readErr
}
c.himportAfterBatch(cn, injected, cmds)
return nil
}); err != nil {
return true, err
}
return false, nil
// Registered fieldsets whose SETs came back "no such fieldset" (the
// session was lost between prepare and use) are re-prepared and those
// SETs re-issued once on the same connection; the error must not
// surface for managed fieldsets.
//
// A transport failure here must neither retry nor fail the batch: the
// first round trip was fully consumed and its results delivered, so
// re-executing would double-apply non-idempotent commands and failing
// would stamp a spurious error onto commands that succeeded. The
// re-issue errors stay on the retried SETs; the connection, which may
// hold unread replies, is marked for removal when released.
if err := c.himportRetryFailedSets(ctx, cn, cmds); err != nil {
internal.Logger.Printf(ctx, "himport: pipeline set re-issue failed: %v", err)
cn.MarkCloseOnPut("himport: transport error during set re-issue")
}
// Preserve retryable first-command errors (e.g. LOADING) for the outer
// loop; the re-issue above may have cleared it.
if readErr != nil {
readErr = cmds[0].Err()
}
return readErr != nil, readErr
}
func (c *baseClient) pipelineReadCmds(ctx context.Context, cn *pool.Conn, rd *proto.Reader, cmds []Cmder) error {
@@ -1271,7 +1398,17 @@ func (c *baseClient) txPipelineProcessCmds(
internal.Logger.Printf(ctx, "push: error processing pending notifications before transaction: %v", err)
}
// HIMPORT bookkeeping: pending discards for this session and PREPAREs
// for registered fieldsets the transaction references get written ahead
// of MULTI; the session state is visible inside the transaction.
injected := c.himportInjectedCmds(ctx, cn, cmds)
if err := cn.WithWriter(c.context(ctx), c.opt.WriteTimeout, func(wr *proto.Writer) error {
for _, ic := range injected {
if err := writeCmd(wr, ic); err != nil {
return err
}
}
return writeCmds(wr, cmds)
}); err != nil {
setCmdsErr(cmds, err)
@@ -1279,6 +1416,13 @@ func (c *baseClient) txPipelineProcessCmds(
}
if err := cn.WithReader(c.context(ctx), c.opt.ReadTimeout, func(rd *proto.Reader) error {
if err := c.himportReadInjectedReplies(ctx, cn, rd, injected); err != nil {
// Transport error with every transaction reply unread: stamp
// the batch like a write failure (see pipelineProcessCmds).
setCmdsErr(cmds, err)
return err
}
statusCmd := cmds[0].(*StatusCmd)
// Trim multi and exec.
trimmedCmds := cmds[1 : len(cmds)-1]
@@ -1289,7 +1433,11 @@ func (c *baseClient) txPipelineProcessCmds(
}
// Read replies.
return c.pipelineReadCmds(ctx, cn, rd, trimmedCmds)
err := c.pipelineReadCmds(ctx, cn, rd, trimmedCmds)
if err == nil || isRedisError(err) {
c.himportAfterBatch(cn, injected, trimmedCmds)
}
return err
}); err != nil {
return false, err
}
@@ -1371,6 +1519,7 @@ func NewClient(opt *Options) *Client {
baseClient: &baseClient{
opt: opt,
onClose: &onCloseHooks{},
himport: newHImportRegistry(),
},
}
c.init()
@@ -1448,7 +1597,11 @@ func (c *Client) WithTimeout(timeout time.Duration) *Client {
}
func (c *Client) Conn() *Conn {
return newConn(c.opt, pool.NewStickyConnPool(c.connPool), &c.hooksMixin)
// Share the HIMPORT fieldset registry: the sticky pool borrows
// connections from this client's pool, so fieldsets prepared on them
// stay valid after the connections are returned.
conn := newConn(c.opt, pool.NewStickyConnPool(c.connPool), &c.hooksMixin, c.himport)
return conn
}
func (c *Client) Process(ctx context.Context, cmd Cmder) error {
@@ -1646,14 +1799,19 @@ type Conn struct {
}
// newConn is a helper func to create a new Conn instance.
// the Conn instance is not thread-safe and should not be shared between goroutines.
// the parentHooks will be cloned, no need to clone before passing it.
func newConn(opt *Options, connPool pool.Pooler, parentHooks *hooksMixin) *Conn {
// The Conn instance is not thread-safe and should not be shared between goroutines.
// The parentHooks will be cloned, no need to clone before passing it.
// himport is the HIMPORT fieldset registry the Conn participates in — pass
// the owning client's registry (a private one would restart versions at 1
// and collide with the client's version space on the shared pooled
// connections); nil disables HIMPORT tracking.
func newConn(opt *Options, connPool pool.Pooler, parentHooks *hooksMixin, himport *himportRegistry) *Conn {
c := Conn{
baseClient: baseClient{
opt: opt,
connPool: connPool,
onClose: &onCloseHooks{},
himport: himport,
},
}
ring.go
+20 -1
View File
@@ -48,6 +48,10 @@ type RingOptions struct {
// NewClient creates a shard client with provided options.
NewClient func(opt *Options) *Client
// himport is the ring-wide HIMPORT fieldset registry, set by NewRing and
// shared with every shard client (see himport.go, himport_cluster.go).
himport *himportRegistry
// ClientName will execute the `CLIENT SETNAME ClientName` command for each conn.
ClientName string
@@ -270,10 +274,16 @@ func newRingShard(opt *RingOptions, addr string) *ringShard {
clopt := opt.clientOptions()
clopt.Addr = addr
return &ringShard{
shard := &ringShard{
Client: opt.NewClient(clopt),
addr: addr,
}
// Share the ring-wide HIMPORT fieldset registry so any shard connection
// serving an HIMPORT SET can lazily replay the PREPARE.
if opt.himport != nil {
shard.Client.himport = opt.himport
}
return shard
}
func (shard *ringShard) String() string {
@@ -605,7 +615,16 @@ func NewRing(opt *RingOptions) *Ring {
if opt == nil {
panic("redis: NewRing nil options")
}
// Shallow-copy the options: the ring-wide HIMPORT registry is carried
// through them to shard construction, and reusing one caller-owned
// RingOptions across several rings must not make the rings share (or
// clobber each other's) registry.
optCopy := *opt
opt = &optCopy
opt.init()
// The registry must exist before the first shard is created; shards
// adopt it in newRingShard.
opt.himport = newHImportRegistry()
hbCtx, hbCancel := context.WithCancel(context.Background())
sentinel.go
+1
View File
@@ -538,6 +538,7 @@ func NewFailoverClient(failoverOpt *FailoverOptions) *Client {
baseClient: &baseClient{
opt: opt,
onClose: &onCloseHooks{},
himport: newHImportRegistry(),
},
}
rdb.init()
tx.go
+5
View File
@@ -48,6 +48,11 @@ func (c *Client) newTx() *Tx {
hooksMixin: c.hooksMixin.clone(),
pushProcessor: c.pushProcessor, // Copy push processor from parent client
onClose: &onCloseHooks{},
// Share the HIMPORT fieldset registry: the sticky pool borrows
// connections from the parent client's pool, so fieldsets
// prepared on them stay valid after the connections are
// returned.
himport: c.himport,
},
}
tx.init()