Skip to content

Runbook: Rotating Alloy mTLS Certificates

Status: Draft — implementation pending

Purpose

Rotate the client certificates used by Grafana Alloy to authenticate to the management-plane Mimir and Loki endpoints.

Prerequisites

  • Access to the certificate authority or Vault PKI role used by the client
  • Access to the client cluster
  • Access to the management-plane observability stack

Steps

  1. Identify the client certificate currently in use.
  2. Issue a new certificate from the CA.
  3. Update the Kubernetes secret referenced by Alloy.
  4. Restart the Alloy DaemonSet to pick up the new certificate.
  5. Verify Alloy can still remote-write to Mimir and Loki.
  6. Revoke or expire the old certificate.

TODO

  • Add cert-manager or Vault PKI-specific commands when tooling is selected.
  • Define certificate lifetime and rotation frequency.
  • Add validation commands.