mirrors/dapr
0
0 Fork 0
mirror of https://github.com/dapr/dapr.git synced 2026-10-28 08:35:25 +08:00
Code Activity

Add Workflow and Scheduler Grafana dashboards, extend system services (#10496)

Browse Source 
* Add Workflow and Scheduler Grafana dashboards, extend system services

The three shipped dashboards have not had a functional change since
#7121 in 2023, and two subsystems have no coverage at all.

Workflow: 25 workflow metrics are registered and none were charted.
Adds a dashboard covering execution and activity rates by status,
execution and scheduling latency percentiles by workflow name, client
operations by operation and status, and the two payload size ratio
instruments with their 0.95 graceful-stall threshold. The overview row
includes activity executions per workflow execution, which reads near 1
when work runs inside the orchestrator instead of being fanned out to
activities.

Scheduler: 15 scheduler metrics and the embedded etcd collectors had no
coverage. Adds a dashboard for job lifecycle including undelivered jobs,
trigger latency, trigger concurrency, placement dissemination, and etcd
health: size against quota, fragmentation, WAL fsync and backend commit
latency, leader changes, compaction and defragmentation. The etcd
histogram panels need --etcd-metrics=extensive; the default basic level
omits them. Addresses #9341, which specified these panels.

System services: adds placement and raft leadership status, hosts
against actor hosts, injection rate by outcome, injection failures by
reason, and operator service reconciliation.

All new panels use timeseries rather than the deprecated graph type, so
they add nothing to #9183 and do not overlap #10266, which changes panel
types on the existing files only.

Metric names were checked against the registered view names, and the
etcd series against a live Scheduler metrics endpoint on 1.18.4.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: MyMirelHub <15373565+MyMirelHub@users.noreply.github.com>

* Fix panels found wrong when run against a live cluster

Deployed the three dashboards against Dapr 1.18.4 with a workflow
application driving real traffic, and ran every panel query against
Prometheus. Three corrections:

Workflow payload size ratio panels queried the metric as a gauge with
max(). Both payload size ratio instruments are histograms, so the
queries returned nothing. Changed to histogram_quantile over the
_bucket series at p99, which is also the useful reading given the 0.95
stall threshold.

Scheduler gauges used sum() across replicas. Every Scheduler replica
reports the same cluster-wide value, so sum multiplies by the replica
count, and multiplies again for every Prometheus job scraping the same
target. Sidecars connected read 144 on a cluster with four sidecars.
Changed sidecars connected, placement streams, incapable sidecars and
the concurrency gauges to max.

Noted on the sidecars connected panel that the metric counts Scheduler
connections rather than distinct sidecars; it read 12 against four
daprd containers.

Also noted on the placement hosts panel that
dapr_placement_actor_runtimes_total was not emitted on this cluster
despite actor hosts being connected.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: MyMirelHub <15373565+MyMirelHub@users.noreply.github.com>

* Add reminder, timer and lifecycle panels to the actor dashboard

The actor dashboard charted one actor metric, dapr_runtime_actor_pending_actor_calls.
Its other panels are HTTP latency filtered by an actor path, and the
placement and lifecycle actor metrics sit on the sidecar dashboard.
Five actor metrics were charted nowhere, all of them reminders and
timers.

Adds two rows, leaving every existing panel untouched:

Reminders and timers
  reminders fired by actor type and success
  timers fired by actor type and success
  timers dropped by actor type
  active timers by actor type

Placement and lifecycle
  pending actor calls by actor type
  actor rebalancing
  deactivations, including failures

Reminders matter more than the count of them suggests: Dapr Workflow
runs on actor reminders, so the internal dapr.internal.<ns>.<app>.workflow
and .activity types show up here and this is where a workflow that never
started becomes visible. Timers dropped records a timer that fired for an
actor no longer hosted on that instance, so the work did not run, which
makes it the clearest signal of actors moving faster than their timers
can complete.

Four actor metrics are registered and documented but have no recorder in
1.18: dapr_runtime_actor_status_report_total, _status_report_fail_total,
_table_operation_recv_total and dapr_runtime_actor_reminders. They are
not charted here. The existing panels for three of them on the actor and
sidecar dashboards are left as they are.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: MyMirelHub <15373565+MyMirelHub@users.noreply.github.com>

* Remove sidecar panels charting metrics that no longer record

Three panels in the Actor row of the sidecar dashboard chart metrics
that nothing has recorded since 1.17, so they have rendered empty for
everyone on 1.17 and later:

  Status report status          dapr_runtime_actor_status_report_total
                                dapr_runtime_actor_status_report_fail_total
  Report status error reasons   dapr_runtime_actor_status_report_fail_total
  Placement table operation     dapr_runtime_actor_table_operation_recv_total

The recorder methods are still registered but have no call sites; the
placement reporting path was rewritten in 1.17 and the instrumentation
was not carried across.

Removed rather than annotated, so the shipped dashboards work against
current releases. The two remaining panels in the row, Actor
Deactivation and Actor table rebalancing, both chart metrics that still
record and are reflowed to fill the row.

Also replaces a description on the system services placement hosts panel
that speculated about when dapr_placement_actor_runtimes_total appears.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: MyMirelHub <15373565+MyMirelHub@users.noreply.github.com>

* Fix the actor dashboard actor type variable

The ACTOR TYPE variable is populated from
label_values(dapr_runtime_actor_pending_actor_calls, actor_type). That
metric only records while a call is queued behind the per-actor lock, so
on an idle or uncontended cluster it has no series at all and the
variable resolves to nothing. Every panel on the dashboard filters on
the variable, so the whole page reads No data even when actors are
running.

Sources it from label_values(actor_type) instead, which enumerates the
label across every metric that carries it, and adds an All option so the
dashboard is populated on open rather than showing a single type.

Also drops the Pending actor calls by type panel added earlier in this
branch, which charts the same metric as the existing Pending locks Per
Actor panel, and widens the two remaining panels in that row.

Adds the Workflow and Scheduler dashboards to the README.

Signed-off-by: MyMirelHub <15373565+MyMirelHub@users.noreply.github.com>

* Remove panels charting metrics that do not exist

Verified every dapr_* metric referenced by the five dashboards against
the views registered in v1.18.4. Seven panels chart names that are not
registered anywhere, so they can never draw:

Actor dashboard
  Timers dropped -- dapr_runtime_actor_timers_dropped_total. The actor
  package registers eleven views and this is not one of them.

Scheduler dashboard
  Placement streams connected, Placement leader, Incapable sidecars and
  the whole Placement dissemination row --
  dapr_scheduler_placement_streams_connected, _placement_leader,
  _placement_incapable_sidecars, _placement_disseminations_total,
  _placement_table_updates_total and
  _placement_dissemination_latency. The scheduler registers no
  placement views. The placement service publishes its own
  dapr_placement_* metrics and the system services dashboard already
  charts all four of them, so there is nothing to re-point these at.

The rest of the scheduler panels are kept: concurrency_inflight,
concurrency_pending, concurrency_throttled_total, jobs_deleted_total,
jobs_bulk_deleted_total, jobs_created_failed_total and
sidecar_errors_total are all registered in 1.18.4. They read No data
only until something records against them, because a view is not
exported before its first record.

Also sets noValue to 0 on the four stat panels that count events over a
window, so a quiet cluster shows 0 rather than No data.

Rows reflowed to close the gaps the removals left.

Signed-off-by: MyMirelHub <15373565+MyMirelHub@users.noreply.github.com>

---------

Signed-off-by: MyMirelHub <15373565+MyMirelHub@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
MirelandClaude Opus 5 authored and GitHub committed 2026-09-17 18:41:20 +00:00
1 parent 885b68f657
commit d31c4a642b
6 files changed
+4014 -271
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

grafana/README.md
+6
View File
@@ -15,6 +15,12 @@ You can set up [Prometheus](https://docs.dapr.io/operations/monitoring/prometheu
3. [Dapr Actor Dashboard](./grafana-actor-dashboard.json)
- Shows Dapr sidecar status - actor invocation throughput/latency, timer/reminder triggers, and turn-based concurrency.
4. [Dapr Workflow Dashboard](./grafana-workflow-dashboard.json)
- Shows workflow and activity execution counts, latency and failures, client operations, and payload sizes against the configured limits.
5. [Dapr Scheduler Dashboard](./grafana-scheduler-dashboard.json)
- Shows Scheduler health, job lifecycle and trigger latency, connected sidecars, placement dissemination, and the embedded etcd.
## Reference
* [Supported Dapr metrics](../docs/development/dapr-metrics.md)
grafana/grafana-actor-dashboard.json
+467 -8
View File
@@ -793,6 +793,461 @@
"align": false,
"alignLevel": null
}
},
{
"collapsed": false,
"datasource": "Dapr",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 35
},
"id": 174,
"panels": [],
"title": "Reminders and timers",
"type": "row"
},
{
"datasource": "Dapr",
"description": "Dapr Workflow runs on actor reminders, so the internal dapr.internal.<ns>.<app>.workflow and .activity types appear here. success=false is a reminder that fired but failed to execute.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisLabel": "",
"axisPlacement": "auto",
"drawStyle": "line",
"fillOpacity": 10,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "reqps"
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 8,
"x": 0,
"y": 36
},
"id": 175,
"options": {
"legend": {
"calcs": [
"lastNotNull",
"max"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"targets": [
{
"expr": "sum(rate(dapr_runtime_actor_reminders_fired_total{namespace=\"$namespace\", actor_type=~\"$dapr_actor_type\"}[5m])) by (actor_type, success)",
"format": "time_series",
"instant": false,
"interval": "",
"legendFormat": "{{actor_type}} success={{success}}",
"refId": "A"
}
],
"title": "Reminders fired",
"type": "timeseries"
},
{
"datasource": "Dapr",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisLabel": "",
"axisPlacement": "auto",
"drawStyle": "line",
"fillOpacity": 10,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "reqps"
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 8,
"x": 8,
"y": 36
},
"id": 176,
"options": {
"legend": {
"calcs": [
"lastNotNull",
"max"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"targets": [
{
"expr": "sum(rate(dapr_runtime_actor_timers_fired_total{namespace=\"$namespace\", actor_type=~\"$dapr_actor_type\"}[5m])) by (actor_type, success)",
"format": "time_series",
"instant": false,
"interval": "",
"legendFormat": "{{actor_type}} success={{success}}",
"refId": "A"
}
],
"title": "Timers fired",
"type": "timeseries"
},
{
"datasource": "Dapr",
"description": "Current registered timer count per actor type, not a rate. dapr_runtime_actor_reminders is registered but has no recorder in 1.18, so it is not charted.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisLabel": "",
"axisPlacement": "auto",
"drawStyle": "line",
"fillOpacity": 10,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "short"
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 8,
"x": 16,
"y": 36
},
"id": 178,
"options": {
"legend": {
"calcs": [
"lastNotNull",
"max"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"targets": [
{
"expr": "sum(dapr_runtime_actor_timers{namespace=\"$namespace\", actor_type=~\"$dapr_actor_type\"}) by (actor_type)",
"format": "time_series",
"instant": false,
"interval": "",
"legendFormat": "timers {{actor_type}}",
"refId": "A"
}
],
"title": "Active timers",
"type": "timeseries"
},
{
"collapsed": false,
"datasource": "Dapr",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 44
},
"id": 179,
"panels": [],
"title": "Placement and lifecycle",
"type": "row"
},
{
"datasource": "Dapr",
"description": "Every replica change of an actor-hosting application disseminates a new placement table and rebalances actors. A burst of dropped timers usually follows.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisLabel": "",
"axisPlacement": "auto",
"drawStyle": "line",
"fillOpacity": 10,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "reqps"
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 45
},
"id": 181,
"options": {
"legend": {
"calcs": [
"lastNotNull",
"max"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"targets": [
{
"expr": "sum(rate(dapr_runtime_actor_rebalanced_total{namespace=\"$namespace\", actor_type=~\"$dapr_actor_type\"}[5m])) by (actor_type)",
"format": "time_series",
"instant": false,
"interval": "",
"legendFormat": "rebalanced {{actor_type}}",
"refId": "A"
}
],
"title": "Actor rebalancing",
"type": "timeseries"
},
{
"datasource": "Dapr",
"description": "Failed deactivations leave actors pinned to a host that is trying to release them.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisLabel": "",
"axisPlacement": "auto",
"drawStyle": "line",
"fillOpacity": 10,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "reqps"
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 45
},
"id": 182,
"options": {
"legend": {
"calcs": [
"lastNotNull",
"max"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"targets": [
{
"expr": "sum(rate(dapr_runtime_actor_deactivated_total{namespace=\"$namespace\", actor_type=~\"$dapr_actor_type\"}[5m])) by (actor_type)",
"format": "time_series",
"instant": false,
"interval": "",
"legendFormat": "deactivated {{actor_type}}",
"refId": "A"
},
{
"expr": "sum(rate(dapr_runtime_actor_deactivated_failed_total{namespace=\"$namespace\", actor_type=~\"$dapr_actor_type\"}[5m])) by (actor_type)",
"format": "time_series",
"instant": false,
"interval": "",
"legendFormat": "deactivation failed {{actor_type}}",
"refId": "B"
}
],
"title": "Deactivations",
"type": "timeseries"
}
],
"refresh": "10s",
@@ -829,22 +1284,26 @@
"useTags": false
},
{
"allValue": null,
"allValue": ".*",
"current": {
"selected": false,
"text": "StateActor",
"value": "StateActor"
"selected": true,
"text": [
"All"
],
"value": [
"$__all"
]
},
"datasource": "Dapr",
"definition": "label_values(dapr_runtime_actor_pending_actor_calls,actor_type)",
"definition": "label_values(actor_type)",
"error": null,
"hide": 0,
"includeAll": false,
"includeAll": true,
"label": "ACTOR TYPE",
"multi": true,
"name": "dapr_actor_type",
"options": [],
"query": "label_values(dapr_runtime_actor_pending_actor_calls,actor_type)",
"query": "label_values(actor_type)",
"refresh": 1,
"regex": "",
"skipUrlSync": false,
@@ -876,4 +1335,4 @@
"title": "Dapr Actor Dashboard",
"uid": "Ym9RqIuWk1",
"version": 3
}
}
grafana/grafana-scheduler-dashboard.json
+1480
View File
File diff suppressed because it is too large. Load diff
grafana/grafana-sidecar-dashboard.json
+6 -262
View File
@@ -1605,262 +1605,6 @@
},
"id": 44,
"panels": [
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "Dapr",
"description": "This shows when Daprd reports host status to placement service. Non-Actor service reports host status in the beginning. Actor service reports host status to placement periodically. If there is a failure, virtual actors will not be distributed properly.",
"fieldConfig": {
"defaults": {
"custom": {},
"links": []
},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 7,
"w": 9,
"x": 0,
"y": 42
},
"hiddenSeries": false,
"id": 46,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"nullPointMode": "null",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.3.3",
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": false,
"steppedLine": true,
"targets": [
{
"expr": "rate(dapr_runtime_actor_status_report_total{app_id=~\"$dapr_app_id\", namespace=\"$namespace\"}[5m])",
"legendFormat": "OK ({{app_id}})",
"refId": "A"
},
{
"expr": "rate(dapr_runtime_actor_status_report_fail_total{app_id=~\"$dapr_app_id\", namespace=\"$namespace\"}[5m])",
"legendFormat": "Error ({{app_id}})",
"refId": "B"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "Status report status",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "none",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
},
{
"cacheTimeout": null,
"datasource": "Dapr",
"description": "This shows the failures when Daprd reports host status to placement.",
"fieldConfig": {
"defaults": {
"custom": {},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 10
}
]
}
},
"overrides": []
},
"gridPos": {
"h": 7,
"w": 5,
"x": 9,
"y": 42
},
"id": 59,
"links": [],
"options": {
"displayMode": "basic",
"orientation": "horizontal",
"reduceOptions": {
"calcs": [
"max"
],
"fields": "",
"values": false
},
"showUnfilled": true
},
"pluginVersion": "7.3.3",
"targets": [
{
"expr": "sum(dapr_runtime_actor_status_report_fail_total{app_id=~\"$dapr_app_id\", namespace=\"$namespace\"}) by (app_id, operation)",
"legendFormat": "{{operation}} ({{app_id}})",
"refId": "A"
}
],
"timeFrom": null,
"timeShift": null,
"title": "Report status error reasons",
"type": "bargauge"
},
{
"aliasColors": {},
"bars": true,
"dashLength": 10,
"dashes": false,
"datasource": "Dapr",
"description": "This shows the placement table update status based on the response from placement service. Daprd maintains the actor placement table in its memory. This table is periodically updated.",
"fieldConfig": {
"defaults": {
"custom": {},
"links": []
},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 7,
"w": 10,
"x": 14,
"y": 42
},
"hiddenSeries": false,
"id": 61,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": false,
"linewidth": 1,
"nullPointMode": "null",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.3.3",
"pointradius": 1,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": true,
"steppedLine": false,
"targets": [
{
"expr": "sum(dapr_runtime_actor_table_operation_recv_total{app_id=~\"$dapr_app_id\", namespace=\"$namespace\"}) by (app_id, operation)",
"format": "time_series",
"intervalFactor": 3,
"legendFormat": "{{operation}} ({{app_id}})",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "Placement table operation",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
},
{
"aliasColors": {},
"bars": true,
@@ -1879,9 +1623,9 @@
"fillGradient": 0,
"gridPos": {
"h": 7,
"w": 9,
"w": 12,
"x": 0,
"y": 49
"y": 42
},
"hiddenSeries": false,
"id": 65,
@@ -1980,9 +1724,9 @@
"fillGradient": 0,
"gridPos": {
"h": 7,
"w": 7,
"x": 9,
"y": 49
"w": 12,
"x": 12,
"y": 42
},
"hiddenSeries": false,
"id": 63,
@@ -2487,4 +2231,4 @@
"title": "Dapr Sidecar Dashboard",
"uid": "Ym9RqIuWk",
"version": 8
}
}
grafana/grafana-system-services-dashboard.json
+497 -1
View File
@@ -12,7 +12,7 @@
}
]
},
"description": "This dashboard shows the metrics of Dapr control plane services",
"description": "This dashboard shows the metrics of Dapr control plane services Includes placement leadership, injector failure reasons and operator reconciliation.",
"editable": true,
"gnetId": null,
"graphTooltip": 0,
@@ -1335,6 +1335,502 @@
"align": false,
"alignLevel": null
}
},
{
"collapsed": false,
"datasource": "Dapr",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 43
},
"id": 145,
"panels": [],
"title": "Placement leadership",
"type": "row"
},
{
"datasource": "Dapr",
"description": "1 when this instance holds leadership. Both series flapping indicates raft instability; placement leadership is what sidecars follow.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisLabel": "",
"axisPlacement": "auto",
"drawStyle": "line",
"fillOpacity": 10,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"min": 0,
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "short"
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 44
},
"id": 146,
"options": {
"legend": {
"calcs": [
"lastNotNull",
"max"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"targets": [
{
"expr": "max(dapr_placement_leader_status)",
"format": "time_series",
"instant": false,
"interval": "",
"legendFormat": "placement leader",
"refId": "A"
},
{
"expr": "max(dapr_placement_raft_leader_status)",
"format": "time_series",
"instant": false,
"interval": "",
"legendFormat": "raft leader",
"refId": "B"
}
],
"title": "Placement leadership status",
"type": "timeseries"
},
{
"datasource": "Dapr",
"description": "Hosts reported to the placement service. Actor hosts is the count that drives placement dissemination, since every change redistributes actors.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisLabel": "",
"axisPlacement": "auto",
"drawStyle": "line",
"fillOpacity": 10,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "short"
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 44
},
"id": 147,
"options": {
"legend": {
"calcs": [
"lastNotNull",
"max"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"targets": [
{
"expr": "sum(dapr_placement_runtimes_total)",
"format": "time_series",
"instant": false,
"interval": "",
"legendFormat": "all hosts",
"refId": "A"
},
{
"expr": "sum(dapr_placement_actor_runtimes_total)",
"format": "time_series",
"instant": false,
"interval": "",
"legendFormat": "actor hosts",
"refId": "B"
}
],
"title": "Hosts reported to placement",
"type": "timeseries"
},
{
"collapsed": false,
"datasource": "Dapr",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 52
},
"id": 148,
"panels": [],
"title": "Injector and operator",
"type": "row"
},
{
"datasource": "Dapr",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisLabel": "",
"axisPlacement": "auto",
"drawStyle": "line",
"fillOpacity": 10,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "reqps"
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 53
},
"id": 149,
"options": {
"legend": {
"calcs": [
"lastNotNull",
"max"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"targets": [
{
"expr": "sum(rate(dapr_injector_sidecar_injection_requests_total[5m]))",
"format": "time_series",
"instant": false,
"interval": "",
"legendFormat": "requests",
"refId": "A"
},
{
"expr": "sum(rate(dapr_injector_sidecar_injection_succeeded_total[5m]))",
"format": "time_series",
"instant": false,
"interval": "",
"legendFormat": "succeeded",
"refId": "B"
},
{
"expr": "sum(rate(dapr_injector_sidecar_injection_failed_total[5m]))",
"format": "time_series",
"instant": false,
"interval": "",
"legendFormat": "failed",
"refId": "C"
}
],
"title": "Sidecar injection rate by outcome",
"type": "timeseries"
},
{
"datasource": "Dapr",
"description": "Failed injections leave pods running without a sidecar. The reason label is the fastest way to tell a webhook or certificate problem from a bad annotation.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisLabel": "",
"axisPlacement": "auto",
"drawStyle": "line",
"fillOpacity": 10,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "reqps"
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 53
},
"id": 150,
"options": {
"legend": {
"calcs": [
"lastNotNull",
"max"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"targets": [
{
"expr": "sum(rate(dapr_injector_sidecar_injection_failed_total[5m])) by (reason)",
"format": "time_series",
"instant": false,
"interval": "",
"legendFormat": "{{reason}}",
"refId": "A"
}
],
"title": "Injection failures by reason",
"type": "timeseries"
},
{
"datasource": "Dapr",
"description": "The operator creates one <app-id>-dapr Service per Dapr-enabled deployment. A persistent update rate with no deployments changing indicates a reconcile loop.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisLabel": "",
"axisPlacement": "auto",
"drawStyle": "line",
"fillOpacity": 10,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "reqps"
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 61
},
"id": 151,
"options": {
"legend": {
"calcs": [
"lastNotNull",
"max"
],
"displayMode": "table",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"targets": [
{
"expr": "sum(rate(dapr_operator_service_created_total[5m]))",
"format": "time_series",
"instant": false,
"interval": "",
"legendFormat": "created",
"refId": "A"
},
{
"expr": "sum(rate(dapr_operator_service_updated_total[5m]))",
"format": "time_series",
"instant": false,
"interval": "",
"legendFormat": "updated",
"refId": "B"
},
{
"expr": "sum(rate(dapr_operator_service_deleted_total[5m]))",
"format": "time_series",
"instant": false,
"interval": "",
"legendFormat": "deleted",
"refId": "C"
}
],
"title": "Operator service reconciliation",
"type": "timeseries"
}
],
"refresh": "5s",
grafana/grafana-workflow-dashboard.json
+1558
View File
File diff suppressed because it is too large. Load diff