mirrors/quickwit
0
0 Fork 0
mirror of https://github.com/quickwit-oss/quickwit.git synced 2026-10-28 10:45:05 +08:00
Code Activity
Files
main
quickwit/Makefile
T
Luca Cominardi a343115707 ci: track metrics inventory and fail when it is out of date (#6935) 
* ci: track metrics inventory and warn on removed metrics

- Generate the metrics inventory as YAML (module -> metric -> kind,
  description) and commit it in quickwit-metrics-inventory/metrics.yaml.
- Build the inventory with --all-features so feature-gated metrics
  (e.g. PostgreSQL metastore) are included.
- Add 'make metrics-inventory' to regenerate the inventory.
- Add an advisory CI workflow that diffs the generated inventory against
  the committed one and warns about metrics removed relative to the base
  revision. It never fails the build.

* ci: fail when the metrics inventory is out of date

Make the metrics inventory check blocking: the job fails if the generated
inventory differs from the committed one, asks to run
'make metrics-inventory', and lists removed metrics with a reminder to
identify their users, assess impact, and communicate the removal.

Paths are filtered at the step level so the job always reports a status
and can be used as a required check.

* ci: classify metrics inventory changes

Move the inventory comparison to scripts/check.sh and classify changes:
added metrics and description changes only require regenerating the
inventory, while removed or renamed metrics, label changes, and type
changes are listed as requiring attention (identify users, assess impact,
communicate the change).

* ci: simplify metrics inventory check

Parse the inventory with yq instead of regexes, and flag any committed
(kind, name+labels) pair that is no longer generated: this covers removed,
renamed, relabeled, and retyped metrics, while additions and description
changes only require regenerating the inventory.

* ci: always print the metrics inventory report to stdout

* metrics-inventory: key the inventory by metric name

Sort metrics alphabetically by name, record the declaring module in a
module field, and list static labels in a separate labels field.
Generation fails if a metric name is declared more than once, instead of
silently dropping one of the declarations.

* ci: always check the metrics inventory on manual runs

On the default branch, paths-filter only looks at the last commit, so a
manual run could skip every step and pass without checking anything.

* metrics-inventory: address review feedback
2026-10-26 19:22:23 +00:00

114 lines
3.5 KiB
Makefile
Raw Permalink Blame History

DOCKER_SERVICES ?= all
QUICKWIT_SRC = quickwit
help:
@grep '^[^\.#[:space:]].*:' Makefile
IMAGE_TAG := $(shell git branch --show-current | tr '\#/' '-')
QW_COMMIT_DATE := $(shell TZ=UTC0 git log -1 --format=%cd --date=format-local:'%Y-%m-%dT%H:%M:%SZ')
QW_COMMIT_HASH := $(shell git rev-parse HEAD)
QW_COMMIT_TAGS := $(shell git tag --points-at HEAD | tr '\n' ',')
docker-build:
@docker build \
--build-arg QW_COMMIT_DATE=$(QW_COMMIT_DATE) \
--build-arg QW_COMMIT_HASH=$(QW_COMMIT_HASH) \
--build-arg QW_COMMIT_TAGS=$(QW_COMMIT_TAGS) \
-t quickwit/quickwit:$(IMAGE_TAG) .
# Usage:
# `make docker-compose-up` starts all the services.
# `make docker-compose-up DOCKER_SERVICES='jaeger,localstack'` starts the subset of services matching the profiles.
docker-compose-up:
@echo "Launching ${DOCKER_SERVICES} Docker service(s)"
COMPOSE_PROFILES=$(DOCKER_SERVICES) docker compose -f docker-compose.yml up -d --remove-orphans --wait
docker-compose-down:
docker compose -p quickwit down --remove-orphans
docker-compose-logs:
docker compose logs -f docker-compose.yml -t
docker-compose-monitoring:
COMPOSE_PROFILES=monitoring docker compose -f docker-compose.yml up -d --remove-orphans
docker-rm-postgres-volume:
docker volume rm quickwit_postgres_data
docker-rm-volumes:
docker volume rm quickwit_azurite_data quickwit_fake_gcs_server_data quickwit_grafana_conf quickwit_grafana_data quickwit_localstack_data quickwit_postgres_data
doc:
@$(MAKE) -C $(QUICKWIT_SRC) doc
fmt:
@$(MAKE) -C $(QUICKWIT_SRC) fmt
fix:
@$(MAKE) -C $(QUICKWIT_SRC) fix
migrations-lint:
@$(MAKE) -C $(QUICKWIT_SRC) migrations-lint
metrics-inventory:
@$(MAKE) -C $(QUICKWIT_SRC) metrics-inventory
typos:
typos
# Usage:
# `make test-all` starts the Docker services and runs all the tests.
# `make -k test-all docker-compose-down`, tears down the Docker services after running all the tests.
test-all: docker-compose-up
@$(MAKE) -C $(QUICKWIT_SRC) test-all
test-failpoints:
@$(MAKE) -C $(QUICKWIT_SRC) test-failpoints
# This will build and push all custom cross images for cross-compilation.
# You will need to login into Docker Hub with the `quickwit` account.
IMAGE_TAGS = x86_64-unknown-linux-gnu aarch64-unknown-linux-gnu x86_64-unknown-linux-musl aarch64-unknown-linux-musl
.PHONY: cross-images
cross-images:
@set -e; \
for tag in ${IMAGE_TAGS}; do \
docker build --tag quickwit/cross:$$tag --file ./build/cross-images/$$tag.dockerfile ./build/cross-images; \
docker push quickwit/cross:$$tag; \
done
# TODO: to be replaced by https://github.com/quickwit-oss/quickwit/issues/237
.PHONY: build
build: build-ui
$(MAKE) -C $(QUICKWIT_SRC) build
# Usage:
# `BINARY_FILE=path/to/quickwit/binary BINARY_VERSION=0.1.0 ARCHIVE_NAME=quickwit make archive`
# - BINARY_FILE: Path of the quickwit binary file.
# - BINARY_VERSION: Version of the quickwit binary.
# - ARCHIVE_NAME: Name of the resulting archive file (without extension).
.PHONY: archive
archive:
@echo "Archiving release binary & assets"
@mkdir -p "./quickwit-${BINARY_VERSION}/config"
@mkdir -p "./quickwit-${BINARY_VERSION}/qwdata"
@cp ./config/quickwit.yaml "./quickwit-${BINARY_VERSION}/config"
@cp ./LICENSE "./quickwit-${BINARY_VERSION}"
@cp "${BINARY_FILE}" "./quickwit-${BINARY_VERSION}"
@tar -czf "${ARCHIVE_NAME}.tar.gz" "./quickwit-${BINARY_VERSION}"
@rm -rf "./quickwit-${BINARY_VERSION}"
workspace-deps-tree:
$(MAKE) -C $(QUICKWIT_SRC) workspace-deps-tree
.PHONY: build-rustdoc
build-rustdoc:
$(MAKE) -C $(QUICKWIT_SRC) build-rustdoc
.PHONY: build-ui
build-ui:
$(MAKE) -C $(QUICKWIT_SRC) build-ui
View Git Blame Copy Permalink