mirror of
https://github.com/dapr/dapr.git
synced 2026-10-28 08:35:25 +08:00
master
7210
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Josh van Leeuwen
|
a8e26cf324 |
CI: use a required gate bc of generated names (#10946)
Signed-off-by: joshvanl <me@joshvanl.dev> |
||
|
Josh van Leeuwen
|
edff789de4 |
ci: cut KinD e2e time by running more packages at once (#10944)
* ci: cut KinD e2e time by running more packages at once The KinD e2e jobs took 25 to 29 min and now set the PR wall time. In run 40507380283 the time went to: - The main pass: 30 min of package time at -p 3 took 10.8 min. Packages start in alphabetical order, so pubsub (5.9 min) started at minute 4 and ran alone at the end. - The scheduler placement pass: 7.2 min at -p 2 took 4 min. - "Free disk space": 2 to 3 min to free 21 GB on a runner that already has 76 GB free. - Six helm installs (Dapr, Redis, Kafka, Zipkin, Postgres, Meilisearch), run one after another. The cases mostly wait on sleeps and redeliveries, not CPU. KinD now runs 5 packages at once in the main pass and 3 in the scheduler placement pass, and the five longest packages are listed first: go test starts packages in the order given. The defaults stay at 3 and 2 because dapr-test.yml runs the same target on AKS. The disk cleanup runs in the background, and the helm installs run in parallel. Running 5 packages at once pairs packages that never overlapped before. The resiliency Kafka bindings shared consumer group "group1" with the bindings tests' components, on different topics. Kafka rebalances a group across all of its topics, so starting either package's consumers would pause the other's. The resiliency bindings get their own group. Signed-off-by: joshvanl <me@joshvanl.dev> * Review comment Signed-off-by: joshvanl <me@joshvanl.dev> * wait for the resiliency Kafka bindings before timing them Signed-off-by: joshvanl <me@joshvanl.dev> --------- Signed-off-by: joshvanl <me@joshvanl.dev> |
||
|
Josh van Leeuwen
|
c5a9d80c33 |
ci: cut integration test time with a dedicated build cache and more parallelism (#10940)
* ci: cut integration test time with a dedicated build cache and more parallelism The integration jobs restored the setup-go cache keyed on go.sum alone, which every setup-go job in the repo shares. Whichever job saves it first wins until go.sum changes, and on 27/10/2026 that was the scheduled dapr-perf run. Its cache does not hold the integration build (CGO off, the integration build tags), so each of the 18 Linux integration jobs spent 6 to 7 minutes rebuilding daprd and the control plane. On master the same build takes 13 to 51 seconds when the cache does hold it, which is why the workflow-mode jobs swung between 9 and 18 minutes. The integration jobs now add tests/integration/framework/binary/binary.go, where those build tags live, to cache-dependency-path, so they get a key of their own. go test runs as many cases at once as the runner has CPUs: 4 on the Linux and Windows runners, 3 on macOS. The test reports show exactly that concurrency for the whole run, and most cases wait on the processes they start rather than use CPU. On 4 pinned CPUs the workflow suite took 7.8 minutes at -parallel=4 and 4.0 minutes at -parallel=8, with no new failures. The jobs now pass -parallel=8, and macOS passes -parallel=6. The workflow-mode matrix also ran the workflow suite with only the scheduler placement authority enabled. The scheduler-placement extended focus job already runs that exact combination, so it is excluded. Signed-off-by: joshvanl <me@joshvanl.dev> * Flakes and account for windows Signed-off-by: joshvanl <me@joshvanl.dev> * Update python version to make go-sdk tests pass Signed-off-by: joshvanl <me@joshvanl.dev> --------- Signed-off-by: joshvanl <me@joshvanl.dev> |
||
|
|
ce93de6d63 |
workflow: host the workflow actor types as fallback while no daprd has an executor (staged operations 1/2) (#10909)
* actors: host types without their reminders, watch remote hosts and drain types Three actor runtime building blocks for the WorkflowsStagedOperations preview, unused until the workflow engine adopts them: - RegisterActorTypeOptions.NoReminderDelivery and SetReminderDelivery host an actor type without advertising it to the scheduler for reminder delivery. - Placement's WatchRemoteHosts tells whether a daprd other than this one hosts an actor type, at the end of each dissemination round. - DrainActorTypes gives up actor types gracefully: they leave the placement report at once and keep their factories until a dissemination round moves them, so calls routed here meanwhile are served. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> * actors: settle remote-host answers per round, reload reminders on re-registration - A remote-host watch refresh, or the completion of one v2 round, no longer publishes the answer for a type whose round is still in progress: its tables are installed at UPDATE but not final until that round's UNLOCK. - Registering an already-advertised type again with a different NoReminderDelivery now broadcasts the new reminder types. That causes no placement round, so the scheduler would otherwise keep the old ones. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> * actors: order type updates under the table lock, keep drains through open rounds Broadcast the advertised types while flagsLock is held, so a concurrent drain or unregistration cannot be overtaken by an older list. Keep a type draining while a round still in progress covers it, and make UnRegisterActorTypes wait out a background halt CompleteDrains started. Also keep the remote-hosts answer per actor type rather than per subscriber, guard halting with flagsLock as a plain map, and merge the two forwarding cases in the placement loop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> * actors: keep the remote-host watch on the placement loop, halt drains inline - The placement loop owns the WatchRemoteHosts subscribers and the answers, with no lock. The disseminator sends it one RoundCompleted per round with the types other daprds host and the types a round still in progress covers. This removes the remotehosts package, RefreshRemoteHosts and its gate, and the last answer now survives stream loss for every subscriber. - noReminders and draining are plain maps guarded by flagsLock, and Types and ReminderTypes take it, so they never see a drain half applied. - CompleteDrains halts the drained factories before it returns, which drops the halting map and the waits on it. - table.New no longer makes a reentrancy store when none is given. - Tests: v2 rounds for CompleteDrains and the reminder reload, and a reload on a new stream waiting for its first round. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> * workflow: host the workflow actor types as fallback while no daprd has an executor Adds the WorkflowsStagedOperations preview feature, off by default. Each daprd hosts the workflow, activity and retention actor types if it has a local executor, or if no other daprd of its app has one for a settle period. A daprd with a local executor also registers the executor actor type, which is how the app's other daprds see that one exists. A daprd hosting the types without a local executor is a fallback host: it does not advertise the types to the scheduler for reminder delivery, a turn on it fails at once so its reminder runs on a daprd with an executor, and it skips the WorkflowsFastPath local drives. It switches in place when a worker connects or the last one leaves, and drains the types once another daprd has had an executor for a damping period. Operations a fallback host serves still wait for the instance to advance; returning from them at once comes in a follow-up. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> * actors: keep re-registered draining types serving while the old factory halts RegisterActorTypes now swaps a draining type's factory for the new one at once and halts the old factory afterwards, so the type is never reported unregistered. Until the old factory holds no actors, the actor IDs it still holds are served by it and the others are created in the new factory, so one actor ID is never live in two factories. RegisterActorTypes also waits for a halt of its types already in progress (CompleteDrains, UnRegisterActorTypes). The create lock is only taken between halts, never across one: a turn being halted may create an actor of its own type. CompleteDrains takes the disseminator's context for its halts. SuspendHosting swaps and broadcasts under flagsLock. Reminder type updates are a signal. WatchRemoteHosts returns a closed channel once placement stopped, and closes the channels the loop no longer can. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> * workflow: run the fallback hosting rule in one loop The connection callbacks register a local executor at once, as without the feature, and hand the rest to runFallback through a kick channel. runFallback owns whether another daprd has an executor and the hosting of a daprd without a local executor: one timer waits out the period the desired hosting must hold (settle to become a fallback host, damping to give the types up). The generation counters, the settled hand-off and the AfterFunc callbacks go away, and actorRegLock only guards the registration. A daprd whose last executor leaves still keeps the types in place as a fallback host, or gives them up at once when another daprd has an executor. A failed registration is still retried. A failed or closed watch now ends runFallback, as it only happens once the actor runtime shuts down, and the callbacks then act alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> * workflow: end a drained factory's background work once the table retires it The workflow, activity, retention and executor factories registered under WorkflowsStagedOperations ran their background goroutines until the next registration replaced them or shutdown, so a daprd which drained the types and then hosted nothing kept them running long after their actors halted. The actor table now retires a factory it has removed for good: once the halt of a factory detached by CompleteDrains, UnRegisterActorTypes or the re-registration of a draining type ends, it calls Retire on a factory implementing targets.Retirer. HaltAll and SuspendHosting keep their factories and do not. The backend wraps the factories sharing a context, and cancels it once the table has retired them all. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> * tests: wait on observable state in the fallback suite, add its missing cases Moves the reusable helpers of the fallback suite into the framework: HostsActorType, WaitForActorTypePlaced and ExecutorActorType on workflow.Workflow, StartConcurrently next to them, and WaitForCompleted beside WaitForAllCompleted. helpers.go keeps newPair and withSettle. flap waits for the fallback host's own log to show it applied each placement round which adds or removes the executor actor type, rather than for the placement service's table and fixed sleeps, so a leave can no longer merge with the next connect into one round on it. transition, inflight and disconnect wait for the load to reach each daprd, and inflight polls the activity's runs over the timer's due time, in place of their fixed sleeps. The suite's polls run every 10ms. New cases: settle (the settle period keeps a daprd a non-host when a worker connects elsewhere within it), placementrestart (a fallback host keeps the types across a placement restart), clustered (staged operations with WorkflowsClusteredDeployment), and forward (instances a fallback host keeps while another daprd has a worker wait, and run once it gives the types up). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> * tests: reconnect a worker during a drain under load Adds the fallback suite's reconnectdrain case: under a load of starts and raises through both daprds of an app, the worker of daprd 0 disconnects, so daprd 0 drains the workflow actor types, and connects again at once, re-registering them while the drain is in progress. It fails on this commit: starts and raises reaching daprd 0 right after the re-registration fail with "operations on actor reminders are only possible on hosted actor types", and an instance whose raise failed never completes. A plain drain, waited out before the reconnect, does not fail. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> * tests: share the fallback workflows' output constant, range the log lines lazily Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> * actors: always halt and retire a replaced factory, halt until empty Registering a type again now replaces its factory through the same swap and halt whether or not the type was draining, so the replaced factory is always halted and retired. Before, re-registering a type that was not draining stored the new factory and left the old one's actors live and its background work running until shutdown. The halt of a retiring factory no longer stops when a pass makes no progress. A call routed to the retiring factory can re-create an actor its halt just deactivated; stopping there retired the factory with that actor live, and the next call for the ID created it again in the new factory. The halt now repeats until the factory is empty or a halt fails. The nretiring counter is gone. retiring only changes with the create lock held for writing, so a create, which holds it for reading, reads its length directly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> * actors: move the executor presence stub into a generic marker factory The executor actor type without WorkflowsClusteredDeployment is registered only so the placement tables show which daprds have a workflow executor. Its factory and actor stubs lived in the workflow backend. They are now pkg/actors/targets/marker, a factory for any actor type registered only to mark the hosts which register it, and the backend registers marker.New(executorActorType). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> * actors: give the remote hosts watch channel one owner, the placement loop WatchRemoteHosts now asks the placement loop for the channel the way Lock does: it enqueues a request with a response channel and waits for the reply or for ctx to end. The loop creates the channel, registers it, arms the unwatch on the request's ctx and replies, so it is the channel's only sender and only closer. A watch enqueued after the loop stopped gets no reply and returns a closed channel once ctx is done. RemoteHostsCh with its once-close and onClose callback, the watches registry, the stopped flag and closeWatches are gone, and loops.go holds plain event structs again. The loop also closes the watch channels when it stops on an error, since it then never handles Shutdown. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> * tests: skip rather than fail the settle case when the runner is slow Daprd 1 may start its settle period a second or more before the test sees daprd 0 hosting, and a loaded runner can take over two seconds to connect the worker and see its executor placed. Either failed the case for timing alone. Skip with the elapsed time instead, and double the settle period so a three second margin separates the skip budget from its end. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> * tests: wait on daprd's unlock count rather than its log in the flap case The flap case keyed on two differently worded log lines, one at debug level, counted against a fixed total. Read the fallback host's count of unlock orders from runtime/actor/table_operation_recv_total before each change, and wait for it to pass that count once the authority shows the change. Both placement protocols record it, so the case needs no debug log or captured output. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> * tests: share the fallback suite's start counter and hosted watcher Three cases copied a counter of started instances and a closure waiting on it, and two copied a goroutine polling whether a daprd still hosts an actor type. StartConcurrently now returns a Load whose WaitStarted waits for each goroutine to record more instances, and Workflow.WatchActorTypeHosted runs the poll and reports whether the type was ever unhosted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> --------- Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Josh van Leeuwen <me@joshvanl.dev> |
||
|
Josh van Leeuwen
|
903a585d26 |
Raise inotify limits for version-skew tests (#10936)
Signed-off-by: joshvanl <me@joshvanl.dev> |
||
|
Josh van Leeuwen
|
24f5c3cfd5 |
ci: raise inotify limits before creating the KinD cluster (#10931)
GitHub moved ubuntu-latest from ubuntu-24.04 to ubuntu-26.04 on 2026-10-26. The 26.04 image leaves fs.inotify.max_user_instances at the kernel default of 128. KinD nodes share the runner's kernel, so the cluster's pods all draw from that one limit. The .NET e2e apps (actordotnet, workflowsapp) set up a file watcher for their config when they start. With the limit used up they crash with "The configured user limit (128) on the number of inotify instances has been reached". This made actor_sdks, workflows, workflow_crossapp, workflow_retention and workflow_accesspolicy fail on every e2e job, on master and on release-1.19. Set max_user_instances to 1280 and max_user_watches to 655360 before creating the cluster, so the job no longer depends on the runner image defaults. Signed-off-by: joshvanl <me@joshvanl.dev> |
||
|
Josh van Leeuwen
|
265324b659 |
tests: wait for forked children before handing out a freed port (#10925)
ports.Free closes the reserved listeners so the process a test starts can bind them. A child forked while those listeners were open (Go forks without CLONE_FILES) holds copies of them until its exec has finished, and the process that got the port then fails with "bind: address already in use". In CI the slow children were the cold `unshare -Umr true` probes of workflow/disk/diskfull and diskfullmidrun, which start alongside workflow/disk/quota: its sentry, placement and scheduler died on their healthz and metrics ports (22/10/2026, three jobs). On Linux, after closing the listeners, Free now takes syscall.ForkLock for reading. forkExec holds it for writing until its vfork child has started to exec, so this waits for every child that may hold copies. While holding it, Free polls each port until it binds, which waits out the rest of those children's exec, and no fork can copy a probe socket meanwhile. darwin releases ForkLock before its child execs and takes it in net.Listen, so the wait is Linux only. Signed-off-by: joshvanl <me@joshvanl.dev> |
||
|
Josh van Leeuwen
|
cbeb47569e |
tests: plant the stalled history the upgrade recovery test relies on (#10929)
* tests: plant the stalled history the upgrade recovery test relies on The workflow in-flight upgrade job runs the upgrade suite against the latest release. v1.19.0 carries the early-result fix (#10590, backported in #10647), so the release daprd no longer stalls the workflow that upgrade/stalledrecovery asked it to stall. The test has failed on master and release-1.19 since that release shipped. The test keeps its coverage without that premise. The release daprd writes the workflow's history, the test inserts the TaskCompleted ahead of the event that gates its activity, which is the shape releases before v1.19.0 persisted, and master must complete the workflow from it. It still fails against a daprd without the fix. Upgrade.Restart is removed, as nothing uses it now. Signed-off-by: joshvanl <me@joshvanl.dev> * Discover correct version for dapr upgrade version Signed-off-by: joshvanl <me@joshvanl.dev> --------- Signed-off-by: joshvanl <me@joshvanl.dev> |
||
|
Josh van LeeuwenandCassie Coyle
|
a24a1981c4 |
ci: make the version-skew run pass against v1.19.0 (#10924)
- integration legs: v1.19.0's suite takes about 70 minutes run serially and hits the 70m test timeout. Run it with test-integration-parallel, as master's own integration jobs do. - integration legs: svidcontext, binarymetadata and binarystore/http/middleware need integration-test-only components (state_spiffeprobe, secretstores_spiffeprobe, bindings_metadataprobe, middleware_http_bodyprobe build tags) that the framework compiles in when it builds daprd itself. The skew run passes prebuilt binaries without them. Build master's binaries with those tags, and in the control-plane-master leg build the release's daprd from the release tag with them instead of using the downloaded binary. The tags go through the environment so the Makefile still adds the allcomponents flavor. - e2e legs: TestSearchVector needs meilisearch. Deploy it when the release's tests/dapr_tests.mk has the target, so older releases still work with /test-version-skew. Signed-off-by: joshvanl <me@joshvanl.dev> Co-authored-by: Cassie Coyle <cassie@diagrid.io> |
||
|
Josh van Leeuwen
|
7f1b4704d4 |
ci: auto-merge clean backport PRs (#10920)
Turn on the `automerge` option of the org backport workflow. Clean backport PRs are now approved by github-actions[bot] and merged by dapr-bot once all required checks on the release branch pass. Backports with conflicts still need a maintainer review. Signed-off-by: joshvanl <me@joshvanl.dev> |
||
|
Nelson ParenteandClaude Opus 5.5
|
49741c24f6 |
docs: add the v1.19.1 and v1.18.10 release notes for the placement connection leak (#10916)
* docs: add the v1.19.1 release note for the placement connection leak Add docs/release_notes/v1.19.1.md with the entry for #10912. That change closes the gRPC connection to Placement when a stream fails to open or ends. It merged without a release note. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Nelson Parente <nelson_parente@live.com.pt> * docs: add the v1.18.10 release note for the placement connection leak Add docs/release_notes/v1.18.10.md with the same entry as v1.19.1.md. release-1.18 has the same tryConnect code, so the leak is there too. The 1.18 note leaves out the goroutine growth rate, because that rate was measured on master. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Nelson Parente <nelson_parente@live.com.pt> --------- Signed-off-by: Nelson Parente <nelson_parente@live.com.pt> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
389422328d |
workflow: stop the executor factory's deactivation goroutine with its context (#10897)
* workflow: stop the executor factory's deactivation goroutine with its context The executor factory started a goroutine that ranged over its deactivation channel, which is never closed, so the goroutine never exited. The workflow actor types build a new executor factory each time they are registered in a clustered deployment, which happens on every workflow worker connect, so a reconnecting worker leaked one goroutine per connect. Exit the goroutine when the factory's context is done, as the orchestrator factory's deactivation workers already do. Actors still queued for deactivation stay in the table until HaltAll or HaltNonHosted reaps them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> * workflow: tie factory background work to the registration, not the stream The workflow actor factories were built with the context of the worker stream whose connect registered the types. When that worker disconnected while another stayed connected, the types stayed registered but their background work stopped: the executor factory no longer deactivated executor actors and the orchestrator reaper no longer reaped idle workflow actors. RegisterActors now builds the factories with a context of their own, which UnRegisterActors cancels once the types are unregistered and Stop cancels when the backend stops. Adds integration tests: repeated worker reconnects leak no goroutines, and executor deactivation and idle reaping keep working after the registering worker leaves while a second one stays connected. Co-Authored-By: Josh van Leeuwen <me@joshvanl.dev> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> * workflow: range over bytes.SplitSeq in factory test Fixes the modernize/stringsseq lint. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> * workflow: end a factory registration that lands after Stop A worker stream can still register the actor types after Stop has run. Nothing cancels that registration's context, so its factories' background work outlived the backend. swapRegCancel now cancels it at once when the backend is stopped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> --------- Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Josh van Leeuwen <me@joshvanl.dev> |
||
|
Nelson ParenteandClaude Opus 5.5
|
6686c0db85 |
actors: close the placement connection when a stream fails to open or ends (#10912)
* Close the placement connection when a stream fails to open or ends daprd retries the placement service about twice a second. Each attempt dials a new gRPC connection. The dial does not block, so it succeeds while placement is down, and only the stream open fails. tryConnect cancelled the stream context but did not close the connection, so each failed attempt left one connection that kept redialing placement. In a 30 s outage, daprd leaked 61 connections. 10 s after placement came back, daprd held 57 TCP connections to it instead of 1. The connection of a stream that ends was also never closed, because the next attempt dials a new one. This leaked one idle connection for each placement stream session. Close the connection when the stream fails to open. Return a stream cancel function that also closes the connection, so the connection closes when the stream ends. The leader connector also closes its previous connection on the next Connect. A second Close has no effect. Add the unit test TestTryConnectClosesConn. It counts the connections that stay open after 5 failed attempts and after a stream ends. Same fix as #10571. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Nelson Parente <nelson_parente@live.com.pt> * tests: add an integration test for the placement connection leak Stop the placement service and check that the daprd goroutine count stays flat while daprd retries it. Without the fix the count grows by about 8 per second and the test fails after about 1 s. The case takes about 3.7 s. Signed-off-by: Nelson Parente <nelson_parente@live.com.pt> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Signed-off-by: Nelson Parente <nelson_parente@live.com.pt> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
c926bf102d |
docs: add the v1.19.0 release notes (#10899)
* docs: add the v1.19.0 release notes The text is the shared release notes draft as of 2026-10-22 10:57 CEST. The list of issues addressed in this release uses patch-equivalence against release-1.18 for dapr, components-contrib, the CLI and the Java and Python SDKs. For the other repositories, it starts at the versions that Dapr 1.18 shipped. Chore, CI, test and dependency PRs are not listed, and backports are listed under their original PR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Nelson Parente <nelson_parente@live.com.pt> * docs: drop the inline PR links from the v1.19.0 highlights The list of issues addressed in this release has every PR. Without the inline citations in Highlights and SDK Improvements, the file is 119,920 characters, inside the 125,000-character limit of a GitHub release body. Links that are part of a sentence stay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Nelson Parente <nelson_parente@live.com.pt> * docs: resync the v1.19.0 release notes with the shared draft The text is the shared release notes draft as of 2026-10-22 13:22 CEST. The JavaScript SDK section comes from its maintainer as a separate file, because it did not fit in the draft. The .NET and JavaScript SDK sections now list their PRs, so the generated list leaves those two SDKs out. components-contrib is listed up to v1.19.0. The file is 137,865 characters, over the 125,000-character limit of a GitHub release body. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Nelson Parente <nelson_parente@live.com.pt> * docs: move the v1.19.0 PR list to its own file The release notes were 137,865 characters. A GitHub release description can have at most 125,000 characters. The release workflow also passes the notes as one environment variable and as one argument, at most 131,072 bytes each. The generated list of pull requests is now in v1.19.0-prs.md, and the notes link to it. The notes are 102,322 characters. The two files keep all 482 pull request links. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Nelson Parente <nelson_parente@live.com.pt> * docs: fix the JavaScript SDK PR count and the health-ping order in the v1.19.0 notes The JavaScript SDK section counted its PRs from the .NET SDK v1.18.1. The previous JavaScript SDK release is v3.18.0. The health-ping paragraph was between two parts about the workflow history cache. It is now after the stateful history table, under its own heading. Its text does not change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Nelson Parente <nelson_parente@live.com.pt> * docs: resolve the review markers and group the 1.18.x changes in the v1.19.0 notes - Remove the CONFIRM markers. proposals#107 is the Search and Vector proposal, and the Responses API text matches components-contrib v1.19.0. - Remove the EDITOR comments. The status sample now shows the 1.19 names dapr-placement-server and dapr-scheduler-server, and 3 Scheduler replicas. - Security: the intro counts five fixes, and Breaking Changes has an entry for the Operator fix (#10890), which also shipped in 1.18.9 and 1.17.18. - Breaking Changes: the entries that also shipped in 1.18 patch releases are in one subsection. - Close the parentheses in the Search and Vector SDK lines. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Nelson Parente <nelson_parente@live.com.pt> * docs: remove three changes that shipped with Dapr 1.18 from the v1.19.0 notes - dapr#10819: a completed workflow does not re-arm for a duplicate result. Its backport #10838 is in v1.18.8. - components-contrib#4413: Kafka connection timeouts and producer settings. Its backport is in components-contrib v1.18.2, which Dapr 1.18.2 and later use. - go-sdk#828: the Go SDK hosts actors over gRPC. Its backport is in go-sdk v1.15.0, the Go SDK for Dapr 1.18. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Nelson Parente <nelson_parente@live.com.pt> * docs: add the .NET 8 and .NET 9 support notice to the v1.19.0 notes Deprecation Notices: .NET 8 and .NET 9 go out of support in November 2026, and the .NET SDK drops both in 1.20. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Nelson Parente <nelson_parente@live.com.pt> --------- Signed-off-by: Nelson Parente <nelson_parente@live.com.pt> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Josh van Leeuwen <me@joshvanl.dev> |
||
|
Nelson ParenteandClaude Opus 5.5
|
d23b217ade |
chore: pin components-contrib v1.19.0 (#10898)
* chore: pin components-contrib v1.19.0 Pin components-contrib v1.19.0, the components release for Dapr 1.19. It adds the dependency updates from components-contrib#4681 to b7184b5f, the commit that master and release-1.19 pin now. go mod tidy raises the shared dependencies to match, for example golang.org/x/crypto v0.58.0, golang.org/x/net v0.61.0 and go.etcd.io/etcd v3.5.34. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Nelson Parente <nelson_parente@live.com.pt> * chore: add the go.sum entries for components-contrib v1.19.0 components-contrib v1.19.0 is tagged at 1d331c389, which is components-contrib#4681 on top of b7184b5f. Its tree is the one that the earlier tidy ran against, so go.mod does not change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Nelson Parente <nelson_parente@live.com.pt> --------- Signed-off-by: Nelson Parente <nelson_parente@live.com.pt> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Josh van Leeuwen
|
a4b118441c |
ci: make the 1.18 version-skew patches apply to v1.18.8 and v1.18.9 (#10895)
Both integration-version-skew legs are red since v1.18.8 was published on 21/10/2026 (run 39662944047). dapr-sidecar-master stops at "Apply patches to latest release": v1.18.8 carries the #10819 backport (#10838), so dedupreassert.go already is the file 008-dedupreassert-running-workflow.patch produces and the patch no longer applies. Remove it. control-plane-master fails 20 tests: the operator (#10890) rejects a GetConfiguration for a configuration none of the app's pods references, and the v1.18.8 tests create configurations without pods. The release-1.18 backport (#10892) changes those tests, so v1.18.9 fixes them, but it also adds the WithNamespacedPodList helper that 004-scheduler-placement-pod-informer.patch inserts into options.go, so the patch would stop applying the moment v1.18.9 is cut. Rewrite it to add the helper in a new file under its own name, WithSkewNamespacedPodList, and to register it first in WithBaseOperatorAPI, where the surrounding lines do not change between releases. The patch applies to v1.18.8 and to the release-1.18 head. Signed-off-by: joshvanl <me@joshvanl.dev> |
||
|
Josh van Leeuwen
|
1efeb205e0 |
workflow: ack a settled completed workflow's wake-ups without a full reload (#10894)
* workflow: ack a settled completed workflow's wake-ups without a full reload On the default path every activity result arms its own wake-up reminder. When the results of a fan-out land before the first wake-up runs, one turn consumes them all and completes the workflow, and the other N-1 wake-ups fire against the completed instance. They queue on the actor lock ahead of the completing turn's deactivation, and handleEmptyInbox reloads a terminal cache unconditionally (since 4,999 loads of 10,009 rows, about 100 ms each. The client's completion wait and every other call to the instance wait behind them: the workflow completed after 46 s and its client saw it after 10m8s. 1.18.7 settles a terminal cache without reloading and answers the same fan-out of 2,000 in 18 s against 1m47s on rc.6, with the same app. An empty wake-up that loaded a completed instance whole, settled it and found no parent notification owed now keeps a record of its metadata row in place of the history. The next wake-up reads that row and the parent-notify row first: an unchanged row (ETag, generation, history length), an empty inbox and nothing owed mean the instance is the one already settled, and the wake-up acks without loading it. Anything else drops the record and takes the existing path, as does a wake-up with a completion held for folding. The actor stays resident with only the record; any actor method drops it, and the idle reaper bounds its residency. The new integration test bounds the history reads that leftover wake-ups and late duplicate results cost after completion, so an O(N) path on a completed instance fails CI. Signed-off-by: joshvanl <me@joshvanl.dev> * Review comment Signed-off-by: joshvanl <me@joshvanl.dev> --------- Signed-off-by: joshvanl <me@joshvanl.dev> |
||
|
Josh van LeeuwenandSathwik Yadlapati
|
d26af14803 |
Operator: enforce app scopes and configuration assignment (#10890)
* Operator: enforce app scopes and configuration assignment The operator's ListHTTPEndpoints and ListMCPServers APIs returned every resource in the caller's namespace, with Kubernetes secret references resolved, without checking each resource's scopes against the caller's app ID. ListResiliency, GetResiliency, ListSubscriptions and ListWorkflowAccessPolicy also ignored scopes, although they return no secrets. GetConfiguration returned any configuration in the namespace by name, including resolved OpenTelemetry header secrets. The list APIs now skip resources not scoped to the calling app ID before resolving any secret, as ListComponents and the update streams already do. GetResiliency reports an out of scope resiliency as not found. GetConfiguration now only serves a configuration named in the dapr.io/config annotation of one of the calling app's Dapr pods, parsed as daprd parses --config (the first entry of a CSV list). Pods are read from the pod metadata cache, and from the API server, selected by the dapr.io/app-id label, only when the cache errors or has no pod for the app. ConfigurationUpdate uses the same lookup, so a pod the cache has not caught up with is still streamed its configuration updates. Remove the unused GetHTTPEndpoint method, which is not part of the Operator gRPC service. Credit to Sathwik Yadlapati Co-authored-by: Sathwik Yadlapati <341710139+yadlapatisathwik@users.noreply.github.com> Signed-off-by: joshvanl <me@joshvanl.dev> * Fix e2e Signed-off-by: joshvanl <me@joshvanl.dev> --------- Signed-off-by: joshvanl <me@joshvanl.dev> Co-authored-by: Sathwik Yadlapati <341710139+yadlapatisathwik@users.noreply.github.com> |
||
|
Josh van Leeuwen
|
02f0b3c3bd |
Updates components-contrib to HEAD (#10887)
Signed-off-by: joshvanl <me@joshvanl.dev> |
||
|
Josh van Leeuwen
|
e407edce6b |
grpc: return DeadlineExceeded for a timed-out actor invocation (#10868)
InvokeActor wrapped every error from the actor call that was not already a gRPC status in ERR_ACTOR_INVOKE_METHOD, which is Internal. That included a deadline: the caller's propagated deadline or the actor's resiliency timeout. A client whose deadline expired therefore saw either DeadlineExceeded (its own timer won) or Internal (daprd's reply won). actors/schedulerplacement/pertype failed on Windows that way (job 116946105556): "code = Internal desc = error invoke actor method: context deadline exceeded". Convert a context.DeadlineExceeded with status.FromContextError before the existing status passthrough, in InvokeActor and in CallActor, the hosting sidecar's side of a remote actor call, so the caller's sidecar passes DeadlineExceeded through as well. context.Canceled is left alone: daprd cancels calls itself (placement dissemination, shutdown, actor lock close), and Canceled would tell the client, and SDKs that do not retry it, that it cancelled a call it did not. A caller's own cancellation never reaches that caller anyway. InvokeService, which already mapped both, is unchanged. Signed-off-by: joshvanl <me@joshvanl.dev> |
||
|
Josh van Leeuwen
|
66f70d0a1b |
tests, ci: fix integration flakes, the 1.18 skew run and push retries (#10869)
Version Skew (dapr-sidecar-master) has failed on every master push since re-arm its wake-up reminder on a duplicate result, which #10819 removed on purpose. Add a 1.18 skew patch that runs the duplicate against a running workflow, as master's test does (jobs 118237499962, 118602178658). ports/internalgrpcshutdown released the app as soon as the internal gRPC port was free. The port is released before daprd closes its components, so the message could complete before the subscription started draining, and the drain then cancelled the stream before the ack was sent. Hold the app until daprd logs that it is shutting down the pubsub component, and a second more, so the drain counts the message as in flight (jobs 117161252491, 118638648849, 118638648507, 118628279831). actors/schedulerplacement/bothalive checked the capability bit before every scheduler's own probe had sighted the placement service; wait for it. Its WatchHosts helper now cancels each stream and reports errors instead of reading them as "not capable" (jobs 116946105596, 118530298187). actors/reminders/scheduler/durable required exactly two firings per reminder, but the one in flight at the interrupt is redelivered to the second daprd; allow one redelivery (job 116938966999). daprd/workflow/loadbalance/haltparked started the rebalance before the completions had parked on the executor actors, so they were routed to the new owner and nothing was handed off; wait for the complete_actor route count first (job 118237503385). Pre-fetch the root module and install gotestsum with retries in test-deps, so unit tests get them too (job 118311894549). Push images with the existing docker-push-retry target in dapr.yml, and back off 30s and 60s between its attempts, as the registry limit is per minute (jobs 118237502871, 118530298592, 116955838537). Signed-off-by: joshvanl <me@joshvanl.dev> |
||
|
Josh van Leeuwen
|
0b63ad6484 |
Sidecar leaked a connection every second while the Scheduler was unreachable (#10878)
Problem While the sidecar could not reach any Scheduler host, it opened a new gRPC connection for each attempt to watch the Scheduler host list, about once a second, and never closed the connection when the attempt failed. Each leaked connection kept its goroutines and memory until the sidecar restarted. Impact You were affected if the Scheduler was unreachable from a sidecar for a sustained period, for example because its replicas were crash looping or their etcd volumes had run out of disk. The goroutine count and memory use of every sidecar grew for as long as the outage lasted. In the reported case, each sidecar accumulated several thousand leaked goroutines over two and a half days. Restarting the sidecar released them. Root Cause The sidecar opens a connection to a Scheduler host and then starts the WatchHosts stream on it. When starting the stream failed, the sidecar returned the error without closing the connection it had just opened, and the retry loop opened another one a second later. Solution The sidecar closes the connection when starting the WatchHosts stream fails, so each failed attempt releases its connection before the next one. Signed-off-by: joshvanl <me@joshvanl.dev> |
||
|
Josh van Leeuwen
|
accd8c83d1 |
Go: update 1.26.9 -> 1.26.10 (#10876)
Signed-off-by: joshvanl <me@joshvanl.dev> |
||
|
Josh van Leeuwen
|
7faabe830a |
Scope scheduler pod RBAC to its own namespace (#10873)
The placement pod informer added in #10416 granted the scheduler get/list/watch on pods through the dapr-scheduler ClusterRole. With the default global.rbac.namespaced=false, that let the scheduler read every pod in the cluster. The informer only watches the placement pods in the scheduler's own namespace, selected by app=dapr-placement-server, so it never needed cluster-wide access. The pods rule moves out of the ClusterRole into a new dapr-scheduler-pods Role and RoleBinding in the release namespace. The ClusterRole keeps namespaces, which the namespace controller still watches cluster-wide. Signed-off-by: joshvanl <me@joshvanl.dev> |
||
|
Cassie Coyle
|
c840236916 |
update docs to match behavior (#10871)
Signed-off-by: Cassandra Coyle <cassie@diagrid.io> |
||
|
Spicy SharkandJosh van Leeuwen
|
6e958f5b7a |
actors: report the TTL of reminders in the reminder list (#10850)
The Scheduler returns the TTL of every reminder job in its list response, but List dropped it when it converted a job to a reminder. ListActorReminders and the HTTP list endpoint therefore left out the ttl of a reminder with a timestamp TTL, which GetActorReminder returns. Parse the TTL in List the same way Get does. A TTL that is not a timestamp, such as the duration 5000s, is left out as in Get, and the reminder is still listed. List logs that parse error at debug level, since it would otherwise log once per such reminder on every call. Signed-off-by: spicysharkco <321699413+spicysharkco@users.noreply.github.com> Co-authored-by: Josh van Leeuwen <me@joshvanl.dev> |
||
|
Josh van LeeuwenandDapr Bot
|
dcb1b1e82b |
Clear error when no workflow worker is connected (#10865)
* Clear error when no workflow worker is connected Workflow actor types are only registered, and advertised to placement, while an app worker holds an open GetWorkItems stream (or an MCP server is registered). With no worker connected, workflow operations failed with actor routing errors that did not explain the cause, such as "did not find address for actor" or "failed to create actor: actor type ... not registered". Starting a workflow retried indefinitely and only returned once the caller's deadline expired. See dapr/docs#5305. The workflow actors backend now wraps these errors with a message stating that no workflow worker is connected for the target app, and that the operation can be retried once it connects. The original error stays wrapped, so existing error codes and errors.Is checks are unchanged. This covers start, raise event, terminate, suspend, resume, purge, rerun and the wait operations, for local and cross-app targets. If no actor state store is configured, the existing state store error is returned instead. CreateWorkflowInstance no longer retries once placement has no host for the workflow actor type. The router has already spent its built-in actor not found retries by then. It still retries when a host from a stale placement table no longer registers the type. The Universal StartWorkflow API no longer retries this error either. Each of its retries repeated the router's full retry budget, which delayed the error by about 20 seconds. Signed-off-by: joshvanl <me@joshvanl.dev> * Review comments Signed-off-by: joshvanl <me@joshvanl.dev> --------- Signed-off-by: joshvanl <me@joshvanl.dev> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> |
||
|
|
04215fd7eb |
feat: search and vector blocks (#9974)
* feat: add search and vector building blocks Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Mike Nguyen <hey@mike.ee> * fix: regen proto, bump deps and update tests Signed-off-by: Mike Nguyen <hey@mike.ee> * fix: linter issues Signed-off-by: Mike Nguyen <hey@mike.ee> * fix: linter issues Signed-off-by: Mike Nguyen <hey@mike.ee> * fix: linter issues Signed-off-by: Mike Nguyen <hey@mike.ee> * fix(tests): disable meilisearch windows integration test Meilisearch does not publish a windos image. Signed-off-by: Mike Nguyen <hey@mike.ee> * feat(searchvector): address review comments - Make optional request fields in search.proto and vector.proto proper fields so an unset value can be told apart from false or 0. Remove the reserved placeholders for alpha and regenerate the Go code. - Leave out the search continuation token when there are no more pages, instead of returning an empty string. - Reject a batch vector query that names a different store or collection. The error goes in that query's slot; queries that leave both unset still use the batch's store and collection. - Stop the HTTP batch query handler from overwriting each query's store and collection, so the runtime can catch a mismatch. - Reject a write's wait_timeout if it isn't shorter than the store's resiliency timeout, and add PolicyDefinition.Timeout() to support this. - Pin the Meilisearch test image to v1.43, the version components-contrib tests against. - Add unit and integration tests for wait-for-completion, wait timeout actions, continuation tokens, missing indexes and collections, dimension mismatches, batch query mismatches and resiliency behaviour. Signed-off-by: Mike Nguyen <hey@mike.ee> * fix: stop retrying writes after an expired FAIL_REQUEST wait Signed-off-by: Mike Nguyen <hey@mike.ee> * chore: remove replace and update dep Signed-off-by: Mike Nguyen <hey@mike.ee> * fix: address various feedback and issues Signed-off-by: Mike Nguyen <hey@mike.ee> * fix: do not treat ALREADY_EXISTS ona retry as a success Signed-off-by: Mike Nguyen <hey@mike.ee> * Use in memory vector/search for int tests- move muilisearch to e2e Signed-off-by: joshvanl <me@joshvanl.dev> --------- Signed-off-by: Mike Nguyen <hey@mike.ee> Signed-off-by: joshvanl <me@joshvanl.dev> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Cassie Coyle <cassie@diagrid.io> Co-authored-by: Nelson Parente <nelson_parente@live.com.pt> Co-authored-by: joshvanl <me@joshvanl.dev> |
||
|
Josh van Leeuwen
|
570245ef5f |
Release notes for redis Entra ID fix (#10861)
Signed-off-by: joshvanl <me@joshvanl.dev> |
||
|
Josh van Leeuwen
|
3d682ba263 |
Fix tracing spans not exported after a SIGHUP restart (#10841)
A HotReload SIGHUP restart replaces the runtime in-process. The old runtime shuts down its SDK tracer provider and the new one registers a replacement with otel.SetTracerProvider. OpenTelemetry delegates the tracers cached at package init, in pkg/diagnostics and in durabletask-go, to the first provider only, so after the first restart every span was started on a shut-down provider. Spans were still created and propagated, but had no span processor left, so nothing was exported for the rest of the process lifetime. Register one forwarding tracer provider as the otel global for the life of the process, and swap the SDK provider behind it on each runtime start. Tracers cached anywhere forward their spans to the provider of the current runtime, and record nothing once a runtime starts with tracing disabled. The SDK tracer is cached per provider so the provider lock is only taken when the provider changes. Fixes #10831 Signed-off-by: joshvanl <me@joshvanl.dev> |
||
|
Javier Aliaga
|
ad5770c86d |
fix: treat a gRPC Canceled under a policy timeout as the timeout (#10787)
* fix: treat a gRPC Canceled under a policy timeout as the timeout A gRPC server whose propagated deadline fires resets the stream with CANCEL, which the caller reads as Canceled. On Windows this can arrive before the caller's clock reads past its own deadline, so timeoutError returned the Canceled as the operation's answer and a retry matching on status codes stopped. Treat Canceled within deadlineClockTolerance the same as DeadlineExceeded. Signed-off-by: Javier Aliaga <javier@diagrid.io> * test: silence gosec G115 on the gRPC code conversion in the retry test Signed-off-by: Javier Aliaga <javier@diagrid.io> --------- Signed-off-by: Javier Aliaga <javier@diagrid.io> |
||
|
|
d3858e4fe5 |
fix: keep the replay buffer when a request body reader is not read (#10805)
A resiliency attempt that hits its timeout is not waited for, so it can call RawData() after the next attempt has read the body. Each RawData() call reset the replay buffer, so if that reader was never read the body was lost and every later retry sent an empty body. Keep everything read so far in the replay buffer and give each reader a snapshot of it followed by the rest of the stream. Signed-off-by: Javier Aliaga <javier@diagrid.io> Co-authored-by: Mike Nguyen <hey@mike.ee> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> |
||
|
Josh van LeeuwenandCassie Coyle
|
854766a40d |
Fix actor lock dispatching queued calls when a queued call is cancelled (#10842)
Each actor keeps a ring of in-flight calls waiting for its lock, with the running turn at the front. The release function is shared by the running turn and by queued calls that give up on their context, and it popped the front of the ring whenever a call's depth reached zero. A call cancelled while queued therefore evicted the running turn and started the next queued call while the running one was still executing. Each cancellation could start one more call, so with cancellations spread across a turn several invocations of the same actor ID ran at once, with reentrancy enabled or disabled. Timers removed while queued behind a turn took the same path. Only the running (front) flight now pops the ring. A flight cancelled while queued stays in place with depth zero, and the running flight skips such entries when it hands over. With reentrancy enabled, a retry carrying the same reentrancy ID revives the entry in its original position. Fixes #10835 Signed-off-by: joshvanl <me@joshvanl.dev> Co-authored-by: Cassie Coyle <cassie@diagrid.io> |
||
|
Mike Nguyen
|
d566b7dd4b |
fix: memory/routine leak - grpc connections unclosed (#10843)
Signed-off-by: Mike Nguyen <hey@mike.ee> |
||
|
Nelson ParenteandClaude Opus 5.5
|
7f85241de7 |
chore: pin dapr/kit v0.19.0 and durabletask-go v0.15.1 (#10845)
Use the tags for 1.19 in place of the main commits that RC3 and RC4 pin. kit v0.19.0 adds dapr/kit#172. durabletask-go v0.15.1 is the commit that is pinned now. Signed-off-by: Nelson Parente <nelson_parente@live.com.pt> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
00d57da050 |
fix: start a root span for pubsub deliveries with no inbound trace context (#10511)
* fix: always start pubsub span and propagate context in subscription runner Signed-off-by: RituGupta23 <ritugupta232004@gmail.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Ritu Gupta <ritugupta232004@gmail.com> * fix: start the pubsub span after the last error path and cover it with tests Extract the cloud event extensions before starting the delivery span, so an extraction failure can no longer return an error alongside a span the caller never gets to end. Add unit tests for GRPCEnvelopeFromSubscriptionMessage covering a cloud event with no trace context (new root span started and a traceparent injected), an inbound traceparent (trace continued), an unparseable traceparent, tracing disabled, and an extensions failure starting no span at all. Signed-off-by: Nelson Parente <nelson_parente@live.com.pt> * fix: start a root span on every pubsub delivery path The fix so far covered GRPCEnvelopeFromSubscriptionMessage, which serves gRPC single and streaming subscriptions. The gRPC bulk, HTTP single and HTTP bulk postmen carry their own copies of the same gate and still skip span creation entirely when the inbound cloud event has no trace context. Replace all four copies with a shared ParentSpanContextFromCloudEvent, which returns the zero SpanContext when the cloud event carries no usable trace context. StartInternalCallbackSpan turns a zero parent into a new root span, so the delivery is traced either way, and is still a no-op when tracing is disabled. Starting a span for every message also makes an existing span leak reachable on every failed delivery, rather than only on the ones that happened to carry a traceparent. The gRPC postman now acquires the app client before starting the span, which removes its only early return between starting the span and ending it. The HTTP postman cannot be re-ordered the same way, because the call that can fail is the app call the span exists to measure, so it defers the end instead. The streaming postman defers it too, though its only early return is unreachable while registerPublishResponse always allocates a channel. Tests cover all four paths: a new root span when there is no inbound trace context, the inbound trace still continued when there is one, unparseable and non-string trace context, tracing disabled, and the error paths that must start no span or leave none unended. The bulk cases mix all three kinds of entry in one batch, and ParentSpanContextFromCloudEvent has a table test covering the legacy traceid field and traceparent taking precedence over it. Three tests cover seeding the delivery runner with the inbound handler context rather than context.Background(): a delivery inherits the handler context's values and deadline, cancelling it cancels an in-flight delivery, and it stops the resiliency runner retrying. An integration test drives a real daprd with tracing on and an OTLP collector attached, and asserts the reported symptom end to end: the app is handed a usable traceparent rather than an empty header, and the matching span reaches the collector as a root. A raw payload subscription stands in for a publisher outside Dapr, since FromRawPayload builds the cloud event with no trace fields and a raw publish puts none in the message metadata either. A shared SpanRecorder in pkg/testing/trace records each span's name, span context and parent as it starts, and which spans have ended, so a test can tell a new root from a continued trace and can see a leaked span. Also scopes the delivery error in TestOnNewPublishedMessage to its subtest. The subtests run in parallel and all assigned to an err declared in the enclosing function, which -race reports on every run of that package. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: MyMirelHub <15373565+MyMirelHub@users.noreply.github.com> * test: cover the root span on gRPC subscriber delivery end to end The existing integration coverage for the fix drives an HTTP subscriber only, while GRPCEnvelopeFromSubscriptionMessage serves the gRPC single and streaming paths. Add the gRPC counterpart: a raw payload subscription stands in for a publisher outside Dapr, and the test asserts the app is handed a usable traceparent in its gRPC metadata rather than none, and that the matching pubsub/<topic> span reaches the collector as a root. Fails against the pre-fix gate with "the app must be given a traceparent, not none". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: MyMirelHub <15373565+MyMirelHub@users.noreply.github.com> * Update pkg/runtime/subscription/postman/http/http.go Co-authored-by: Cassie Coyle <cassie.i.coyle@gmail.com> Signed-off-by: Mirel <15373565+MyMirelHub@users.noreply.github.com> * fix: import otel codes for the failed-delivery span status The review suggestion that sets an error status on a delivery which never reached the app was applied from the web UI, which inserts the code but cannot add the import it needs. otelcodes was undefined, so the postman package did not build and every job that builds daprd failed. Add the import, aliased as pkg/diagnostics already does, and assert the status in TestDeliverEndsSpanOnAppChannelError: the recorder now keeps each ended span's status code, and the test fails if the span ends Unset. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: MyMirelHub <15373565+MyMirelHub@users.noreply.github.com> * test: pass the bulk ingress diagnostics by the pointer it now returns NewBulkSubIngressDiagnostics started returning *BulkSubIngressDiagnostics, so taking its address in the tracing tests yields a double pointer and the postman packages no longer build: cannot use &bulkSubDiag (value of type **todo.BulkSubIngressDiagnostics) as *todo.BulkSubIngressDiagnostics value in struct literal Drop the &, matching how the existing tests in both packages assign it. This is what failed the lint job too, since golangci-lint type checks. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: MyMirelHub <15373565+MyMirelHub@users.noreply.github.com> --------- Signed-off-by: RituGupta23 <ritugupta232004@gmail.com> Signed-off-by: Ritu Gupta <ritugupta232004@gmail.com> Signed-off-by: Nelson Parente <nelson_parente@live.com.pt> Signed-off-by: MyMirelHub <15373565+MyMirelHub@users.noreply.github.com> Signed-off-by: Mirel <15373565+MyMirelHub@users.noreply.github.com> Co-authored-by: RituGupta23 <ritugupta232004@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: Nelson Parente <nelson_parente@live.com.pt> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Cassie Coyle <cassie@diagrid.io> Co-authored-by: Cassie Coyle <cassie.i.coyle@gmail.com> |
||
|
Josh van LeeuwenandDapr Bot
|
6801e6d54e |
workflow: do not re-arm a completed workflow for a duplicate result (#10819)
* workflow: do not re-arm a completed workflow for a duplicate result A completion that duplicates one already in history or the inbox is acked and its wake-up re-driven, so an inbox row whose first wake-up failed to arm is not stranded. A completed workflow never processes its inbox again, yet the re-drive still ran for it: every late redelivery created a new-event reminder, which then fired into an empty inbox. After a fan-out whose activities re-ran, that is thousands of reminder creates, fires and actor turns after the workflow has finished. Ack a duplicate for a completed workflow through the existing drop path, which confirms the cache against the store's metadata row first, as other drops for a completed workflow do. A running workflow and a retry joining a held fast-path completion keep the re-drive. Signed-off-by: joshvanl <me@joshvanl.dev> * Review comment Signed-off-by: joshvanl <me@joshvanl.dev> --------- Signed-off-by: joshvanl <me@joshvanl.dev> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> |
||
|
Josh van LeeuwenandDapr Bot
|
ee35baa8cb |
daprd: bind the internal gRPC port before any outbound connection (#10820)
* daprd: bind the internal gRPC port before any outbound connection daprd binds its internal gRPC port only after it has connected to sentry, the operator and every scheduler. Until then the port is free, and one of those connections can be given it as its ephemeral source port: the default port, 50002, is inside Linux's default ephemeral range. Both binds then fail for 10s each and daprd exits with "failed to start internal gRPC server: could not listen on any endpoint". This happens on a start, and again on every SIGHUP (for example a Configuration hot reload), where the old runtime closes its listener and the new one binds it again late. Bind the port once per process, in Run() before the restart loop and so before any outbound connection, and hold the socket for the life of the process. Each runtime serves its own duplicate (net.FileListener) and closes only that, so the port stays bound between runtimes. The held socket is guarded by a mutex, so closing it at shutdown cannot race a runtime taking its copy, and it is closed when shutdown starts, so the port is released when the runtime closes its server, as before. The process-level bind retries for 10s like the runtime's; if it still fails, each runtime binds the port itself, as before. Signed-off-by: joshvanl <me@joshvanl.dev> * Review comments Signed-off-by: joshvanl <me@joshvanl.dev> --------- Signed-off-by: joshvanl <me@joshvanl.dev> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> |
||
|
Josh van LeeuwenandDapr Bot
|
a0f53a73db |
workflow: reload the state on an empty inbox only when the cache is stale (#10823)
* workflow: reload the state on an empty inbox only when the cache is stale A wake-up that finds the inbox empty dropped the cached state and reloaded every history row before acking, to guard against a cache made stale by a peer host writing to the store. It did so on every such wake-up. A fan-out arms about one wake-up per activity result, and a turn consumes every result already in the inbox, so most of those wake-ups find it drained. At 10,000 activities that is thousands of reloads of 20,000 rows, about 205ms each, back to back under the actor lock: the workflow spends most of its time reloading itself, and status streams queued behind the wake-ups never answer. Reload only when the cache cannot be shown current: a state the turn just loaded is current, and so is a cached state whose metadata row in the store carries the cached ETag, generation and history length and an empty inbox. Checking that is one row read. The ETag alone is not trusted: the post-save refresh can record a peer's newer ETag, and some stores restart a row's version when it is recreated. A missing ETag, any mismatch or a missing row reload as before; a failed read returns a retryable error. Signed-off-by: joshvanl <me@joshvanl.dev> * Review comments Signed-off-by: joshvanl <me@joshvanl.dev> --------- Signed-off-by: joshvanl <me@joshvanl.dev> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> |
||
|
|
906228c8e4 |
security: an interrupt during the identity fetch is a shutdown, not an error (#10792)
* security: an interrupt during the identity fetch is a shutdown, not an error After a SIGHUP, daprd starts a new runtime under a context the next signal cancels. If an interrupt arrived while that runtime was waiting for sentry to sign its identity, the cancelled SignCertificate call failed with a gRPC Canceled status, which does not wrap context.Canceled. The runner managers ignore only context.Canceled, so the error reached the restart loop, which logged "Fatal error from runtime" and exited 1 for an ordinary shutdown. When the identity fetch fails after its context was cancelled, the security provider now returns the context's error. Every other failure, including a failed shutdown, still exits 1. Signed-off-by: joshvanl <me@joshvanl.dev> * Review comments Signed-off-by: joshvanl <me@joshvanl.dev> * Fix windows tests Signed-off-by: joshvanl <me@joshvanl.dev> --------- Signed-off-by: joshvanl <me@joshvanl.dev> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> Co-authored-by: Cassie Coyle <cassie@diagrid.io> |
||
|
|
50b8b6dd71 |
hotreload: do not restart daprd for an unchanged HTTPEndpoint (#10785)
* hotreload: do not restart daprd for an unchanged HTTPEndpoint The SIGHUP reconciler restarts daprd when a Configuration, HTTPEndpoint or Resiliency event differs from what daprd holds. It compared the event against the component store, but the HTTPEndpoint processor stores the endpoint after resolving its secret references, so an endpoint with a secret ref never compared equal. Every operator informer relist then sent an UPDATED event for an unchanged endpoint and restarted daprd. Keep the reconciler's own copy of each resource as the loader served it, seeded from the loader's list when the reconciler starts and updated on every event, and compare against that. The component store is only the fallback for a name the list did not return. Signed-off-by: joshvanl <me@joshvanl.dev> * Adds int test for coverage Signed-off-by: joshvanl <me@joshvanl.dev> * Review comments Signed-off-by: joshvanl <me@joshvanl.dev> * lint Signed-off-by: joshvanl <me@joshvanl.dev> * Fix tests on windows Signed-off-by: joshvanl <me@joshvanl.dev> --------- Signed-off-by: joshvanl <me@joshvanl.dev> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> Co-authored-by: Cassie Coyle <cassie@diagrid.io> |
||
|
Josh van LeeuwenandDapr Bot
|
d221275c3e |
scheduler: exit when the server cannot bind its port (#10817)
Since #10345 runServerLoop recreates the scheduler server in-process after any runtime error. A port the server cannot bind stays unbindable for as long as whatever holds it lives, so every incarnation fails the same way: the replica stays unready until its liveness probe restarts it, 230s with the chart defaults. In 1.18 the error was fatal and the restarted process got new sockets within seconds. The scheduler's port, 50006, lies inside Linux's default ephemeral port range, so one of the process's own outbound connections can hold it, and a connection that outlives the server incarnations (the Kubernetes API or sentry) is never released by the retry. Return a bind failure from runServerLoop so the process exits, as in 1.18. Every other runtime error is still retried in-process. Signed-off-by: joshvanl <me@joshvanl.dev> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> |
||
|
Josh van LeeuwenandDapr Bot
|
180905c68b |
scheduler: keep 10,000 etcd revisions instead of 1,000,000 (#10818)
The scheduler's embedded etcd compacts by revision count and keeps the last 1,000,000 revisions. Nothing is compacted until a cluster passes that count, and every revision of a deleted job keeps the job's payload, so the database holds the payloads of the last million writes. Under workflow load that is gigabytes: 2.7 GB of database and 3.4 GB of scheduler memory after 15 hours of chaos testing, for 19 MB of live jobs. A 1Gi volume, which clusters installed before 1.17.7 keep, fills long before the first compaction, and etcd writes then fail. Keep 10,000 revisions. etcd's revision compactor runs every 5 minutes, so the database holds at most 10,000 revisions plus 5 minutes of writes. At the campaign's average of about 70,000 revisions an hour that is 8 to 9 minutes of history, close to the 10 minute periodic retention used before revision mode; during a burst it is shorter. The retention bounds the number of revisions, not their size. The flag default and the chart value change together, as the chart passes the flag explicitly. Signed-off-by: joshvanl <me@joshvanl.dev> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> |
||
|
Amit HampalandCursor
|
7d381078e0 |
sentry: scope Kubernetes validator pod cache to injected pods (#10802)
* sentry: scope Kubernetes validator pod cache to injected pods Signed-off-by: amit <amit@grafana.com> Co-authored-by: Cursor <cursoragent@cursor.com> * test(sentry): assert pod cache selector Signed-off-by: amit <amit@grafana.com> Co-authored-by: Cursor <cursoragent@cursor.com> * fix: review notes Signed-off-by: amit <amit@grafana.com> Co-authored-by: Cursor <cursoragent@cursor.com> --------- Signed-off-by: amit <amit@grafana.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Josh van LeeuwenandDapr Bot
|
2c9b171a81 |
pubsub: register a streaming subscriber before acknowledging it (#10793)
* pubsub: register a streaming subscriber before acknowledging it
SubscribeTopicEventsAlpha1 started the subscription's delivery and sent
the initial response before the connection was registered in the
streamer. A message delivered in between found no connection in
streamer.Publish ("no streamer subscribed"), was treated as
non-retriable and dropped, although the client had been told it was
subscribed. A client that publishes as soon as it reads the initial
response, as daprd/subscriptions/stream/multi does, could lose it.
Make the streamer own the handshake: Subscribe registers the connection,
then calls start (the processor subscription), sends the initial
response, marks the connection ready and only then receives. Publish
waits for ready, the connection closing or its context, so a message
delivered the moment the subscription starts is sent after the initial
response, a failed start ends the wait, and the wait can be cancelled.
The "Subscribing to pubsub" log line now follows a successful start.
Signed-off-by: joshvanl <me@joshvanl.dev>
* Review comments
Signed-off-by: joshvanl <me@joshvanl.dev>
---------
Signed-off-by: joshvanl <me@joshvanl.dev>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
|
||
|
Javier AliagaandJosh van Leeuwen
|
d77dbcdcf9 |
test(e2e): give the resiliency pubsub publish a 5s timeout (#10806)
The first publish to a topic sets up the broker connection. On Azure Service Bus from the Windows e2e this can take longer than the 500ms fast timeout, and every retry starts the setup again. Signed-off-by: Javier Aliaga <javier@diagrid.io> Co-authored-by: Josh van Leeuwen <me@joshvanl.dev> |
||
|
Josh van LeeuwenandDapr Bot
|
d1aad0c659 |
tests: legacyactivityid allows a re-entered body in clustered mode (#10789)
Signed-off-by: joshvanl <me@joshvanl.dev> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> |
||
|
Josh van Leeuwen
|
ed24151fcf |
tests: stop the MCPServer integration tests dialling example.com (#10790)
Add prochttp.NotFoundURL, a local server answering 404 for the test's lifetime, and use it for every dialled URL. A closed local port is not used: on Windows a refused localhost connection takes seconds. The ftp:// inputs stay (they are validation cases), and mcpserverchurn is unchanged (its URL is never dialled). Signed-off-by: joshvanl <me@joshvanl.dev> |
||
|
Josh van LeeuwenandDapr Bot
|
a2471f596b |
scheduler: reconnect job streams while a job is still running (#10794)
When a scheduler job stream ended, the streamer waited for every job it had started before returning, so the connector could not reconnect until those jobs finished. The jobs ran under the streamer's context, which is only cancelled once the streamer returns. A job that waits on something only a reconnected stream provides never finished: with actor placement hosted by the scheduler, an activity that completed after its scheduler restarted waited to publish its result for a placement leader, and the restarted scheduler only becomes leader once a sidecar connects. Cancel the jobs' context when the stream's context ends as well. While the stream is live the streamer still waits for every result before it closes the send side. Signed-off-by: joshvanl <me@joshvanl.dev> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> |
||
|
Josh van LeeuwenandDapr Bot
|
66eddb5dfa |
grpc proxy: keep each resiliency attempt's state local to that attempt (#10797)
With a resiliency timeout the policy runner starts the next attempt while the timed-out one is still running. The proxy handler's attempt function wrote two variables declared outside it: requestStartedAt and the metadata map md. A timed-out attempt could therefore record the next attempt's start time, or delete the app ID key from the map the next attempt was copying, which can crash the sidecar with a concurrent map write. Declare both inside the attempt. The metadata returned by FromOutgoingContext is already a copy, so it no longer needs another. Signed-off-by: joshvanl <me@joshvanl.dev> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> |