Your monitoring stack is eleven majors behind. Jumping straight to the newest release would break it; bumping one patch at a time would take a year. The assistant reads every changelog in between, checks which breaking changes actually touch your configuration, and proposes the biggest safe step.
The daily upgrade scan starts from the live HelmReleases, OCI sources and container images, and links each back to the file in git that declares it. A release that exists only in git, or only in the cluster, is handled differently from one that is in both.
Here: kube-prometheus-stack, pinned at
61.3.2, with a values block your team has tuned over
the years.
It fetches the chart's index.yaml (or lists OCI tags),
with your repository credentials where needed, drops pre-releases,
and skips versions you or we have marked known-bad.
Container images get the same treatment through Docker Hub, GHCR and your own registries.
Release notes come from GitHub releases, then from
CHANGELOG.md / UPGRADE.md when a project
publishes no releases. Each breaking change is extracted and
classified: a values rename, a required step, a CRD change, a
migration path.
If a changelog could not be fetched, the MR says so in a banner and risk goes up. Missing evidence is never reported as "no breaking changes".
Each finding is checked against your HelmRelease values, and against a render of the chart with those values. Helm never installs the versions in between, so a change introduced in 63 and reverted in 64 is irrelevant to a jump past 64.
Required-step changes are hard stops no matter what your values say.
The rule is simple and written into the MR: with breaking-change evidence in range, stop just before it and quote the excerpt. With none, go up to a year of releases in one step.
Here the first stop that applies is 67.0.0, so the
target is 66.7.1. That is five majors in one reviewed
MR, and the next step becomes its own MR once this one is merged.
helm template runs at the current and the target
version with your values, and the two outputs are diffed. That
shows what the version string hides: new CRDs, changed selectors,
a container that gains an argument.
Changes to load-bearing fields (container specs, Service ports, RBAC, storage) raise the risk and are called out in the MR.
The rendered objects are validated against the CRD schemas the cluster actually serves, and sent as a server-side dry-run under a dedicated identity that an admission policy limits to dry-run only. It can ask, it can never apply.
The diff is tiny. The MR body carries the reasoning: why this version and not the newest, which breaking changes apply and what to do about them, what could not be evaluated, and what to watch after merging. The full MR is below.
apiVersion: helm.toolkit.fluxcd.io/v2 kind: HelmRelease metadata: name: kube-prometheus-stack namespace: monitoring spec: chart: spec: chart: kube-prometheus-stack version: 61.3.2 sourceRef: { kind: HelmRepository, name: prometheus-community } values: prometheus: prometheusSpec: retention: 15d serviceMonitorSelectorNilUsesHelmValues: false alertmanager: { … 40 lines … }
# quoted from the 67.0.0 release notes "This version upgrades Prometheus to 3.x. Review the Prometheus 3 migration guide before upgrading; some flags and feature flags were removed."
Deployment/monitoring/kube-prometheus-stack-operator - image: quay.io/prometheus-operator/prometheus-operator:v0.75.2 + image: quay.io/prometheus-operator/prometheus-operator:v0.79.2 Prometheus/monitoring/kube-prometheus-stack-prometheus - version: v2.53.1 + version: v2.55.1 CustomResourceDefinition (6) schema updated ServiceMonitor (14) unchanged Service ports · RBAC · PVC templates unchanged
✓ 212 objects schema-valid ✓ server-side dry-run: accepted a real apply by this identity is refused by admission policy
chart: kube-prometheus-stack - version: 61.3.2 + version: 66.7.1
Illustrative, but shaped exactly like the real thing: the section headings and the target rule are the ones the assistant writes today.
gitops-ai/upgrade-kube-prometheus-stack-66.7.1 into main · 1 file · +1 −1Upgrades the kube-prometheus-stack chart in monitoring from 61.3.2 to 66.7.1. Newest published: 72.9.1 — this MR deliberately stops at 66.7.1.
Stopped at 66.7.1 because of a breaking change in 67.0.0 (step_required):
This version upgrades Prometheus to 3.x. Review the Prometheus 3 migration guide before upgrading; some flags and feature flags were removed.
The 63.0.0 selector deprecation matches a key set in this HelmRelease, but 64.0.0 reverts it, so it does not apply to a jump past 64.
Crossing 67.0.0 needs the Prometheus 3 checks listed under Next step. A follow-up MR will propose 66.7.1 → 68.x once this one is merged and healthy.
Nothing. 38 of 38 release notes fetched; render pair and dry-run completed.
| Version | Class | Applies to this release? |
|---|---|---|
| 62.0.0 | CRD update | Handled by the chart's CRD upgrade job. No action. |
| 63.0.0 | values_incompatibility | Moot: reverted in 64.0.0, never installed. |
| 65.0.0 | dependency major (grafana subchart) | Not applicable: grafana.enabled: false. |
| 67.0.0 | step_required | Hard stop. Outside this MR. |
flux get hr -n monitoring kube-prometheus-stack shows 66.7.1, Ready.PrometheusOperatorRejectedResources alerts in the first 30 minutes.Revert this MR. The chart's CRDs are backward compatible within this range.
chart:
spec:
chart: kube-prometheus-stack
- version: 61.3.2
+ version: 66.7.1
sourceRef:
kind: HelmRepository
name: prometheus-community
Re-read the 62–67 release notes and confirmed the 63/64 revert pair and the 67 stop. Render diff matches the claimed scope. LGTM.
The same research drives the cases where a bump alone would upgrade nothing, or would break something.
updateStrategy: OnDelete keeps running the old image after a chart bump. The MR says so and lists the per-pod rollout steps (for Vault: delete, then unseal, one pod at a time).