Home / Stories / DNS as code
Story · Config as code Roadmap

Retire the load balancer. The DNS records go in the same wave.

You retire an ingress controller and release its address. Three DNS records still point at it, proxied, returning errors, and nobody notices until someone sweeps every zone by hand. With DNS in git and an agent that checks every record against what actually runs, those records become a draft MR next to the retirement. This lane is designed; the read-only Cloudflare access it builds on is live.

Starts fromCloudflare DNS for every account and zone, imported into a git repository with a CI apply.
Looks atRecords in the repo and in Cloudflare, live Ingress / IngressRoute / HTTPRoute hosts, LoadBalancer addresses on each cluster.
You getDraft MRs to delete or repoint dangling records in zones you let us manage; issues for zones you manage yourself.
01 · The retirement

An old ingress controller goes, and its address is released.

A migration wave moves the last routes from the old controller to the new one. The old controller is removed, and its static load-balancer address is released back to the cloud provider.

The Kubernetes side is clean. DNS lives somewhere else.

02 · What gets left behind

Three records, still pointing at the freed address.

The records are proxied, so Cloudflare keeps answering for them and returns errors to visitors. Nothing in the cluster knows they exist.

The way this is found today is a one-off sweep: every account, every zone, hundreds of records, joined by hand against the addresses that are being released.

03 · DNS in git Roadmap

One repository, imported from live, applied by CI.

Every account and zone is imported from Cloudflare, never hand-written, recording which objects the repo declares and which it only observed. Merge requests show a plan; main applies it.

The write token lives only in CI. The assistant keeps its read-only Cloudflare tokens, the same ones its Cloudflare posture review already uses in production.

04 · The join Roadmap

Every record against every route and every address.

A deterministic agent, with no LLM in the loop, joins repo records and live records against the hosts declared by live Ingress, IngressRoute and HTTPRoute objects, and against the LoadBalancer addresses on each cluster.

It only looks at A, AAAA and CNAME records. A verification TXT record on the same name would otherwise mask the address record and produce false findings.

05 · Four buckets Roadmap

Each mismatch has one meaning and one next step.

  • Dangling: record points at a retired or absent address.
  • No route: record points at a live load balancer, but no route declares the host.
  • No record: a route declares a host nobody can reach.
  • Unknown address: neither a cluster address nor a known third party.
06 · Your zone, your call Roadmap

Managed zones get MRs. Advisory zones get issues.

Each account or zone has an explicit management mode. Managed: the DNS repo is authoritative for what it declares, CI holds a write token, and the assistant proposes draft MRs. Advisory: you manage the zone yourself; the assistant only reads it and files issues.

The emission surface follows the mode, never the mere presence of a token.

07 · Out-of-band edits Roadmap

Declared means managed. Everything else is left alone.

Proxy, page-rule and Access settings can live in the repo, but not exclusively. An object the repo does not declare is reported as drift to review, never proposed for deletion, and the apply never purges it.

A dashboard edit to a declared record is real drift: an MR to accept the live value, or an issue to discuss it.

08 · Same wave Roadmap

The retirement MR names the records it orphans.

When an MR retires a host or a load balancer, the DNS lane joins the retired address against the records and cites the ones that would be left dangling, with a link to the matching DNS MR. The cleanup is part of the change, not a sweep months later.

09 · The proposal

Delete three records, with the proof that nothing serves them.

The diff removes three records from the DNS repo. The MR body shows why each one is dangling, what the CI plan will do, and what to check after the apply. The full MR is below.

Merged · Retire the legacy ingress controller
- infrastructure/ingress-legacy/helmrelease.yaml
- infrastructure/ingress-legacy/service.yaml
  loadBalancerIP: 203.0.113.10   # released
legacy controller0 routes
new controllerall routes
203.0.113.10released
cluster: clean · Flux: Ready
Cloudflare · acme.example · A / CNAME
nametypecontentproxied
shopA203.0.113.20yes
loginA203.0.113.10yes
apiA203.0.113.20yes
previewA203.0.113.10yes
grafana-oldA203.0.113.10yes
wwwCNAMEshop.acme.exampleyes
3 proxied records → a released address · found by a manual sweep
dns/zones/acme.example.yaml · imported from live
zone: acme.example
mode: managed
records:
  - { name: shop, type: A, content: 203.0.113.20, proxied: true }
  - { name: login, type: A, content: 203.0.113.10, proxied: true }
  - { name: www,  type: CNAME, content: shop.acme.example }
observed_only:
  - page rule "legacy-redirect"   # not declared, never purged
MRplan in CI
mainCI apply · write token
assistantread-only tokens
DNS recordsrepo ∪ live · A/AAAA/CNAME
Route hostsIngress · IngressRoute · HTTPRoute
LB addressesper cluster
join · no LLM
 shop.acme.example      → 203.0.113.20 → route shop/storefront
 login.acme.example     → 203.0.113.10 → no load balancer
· _verify.shop  TXT      ignored: not an address record
Dangling→ retired address · MR: delete or repoint
No route→ live LB, no host · issue: controller returns 404
No recordroute with no DNS · issue: unreachable route
Unknown addressnot ours, not a known provider · issue: identify
this wave: 3 dangling · 0 no route · 1 no record · 0 unknown
Finding: 3 dangling records
acme.example · the zone's mode decides the surface
mode: manageddraft MR to the DNS repo
mode: advisoryissue only · no write path
a write token alone never turns an advisory zone into a managed one
repo ↔ live · acme.example
  • api · A · declared 203.0.113.20, live 203.0.113.21 declared drift → MR or issue
  • page rule "legacy-redirect" not declared → reported, never deleted
  • Access app "admin" not declared → reported, never deleted
Draft · Retire the legacy ingress controller · note
Ingress MRretires the address
DNS MRremoves the records
Draft · Remove 3 records pointing at retired 203.0.113.10
-  - { name: login,       type: A, content: 203.0.113.10, proxied: true }
-  - { name: preview,     type: A, content: 203.0.113.10, proxied: true }
-  - { name: grafana-old, type: A, content: 203.0.113.10, proxied: true }
category:reliabilitysource:dnsrisk:low
↓ the full merge request is below

The merge request you would get.

Illustrative and on the roadmap. The record format is neutral on purpose: which DNS-as-code tool to use is still an open decision.

Draft

Remove 3 DNS records pointing at retired address 203.0.113.10

from gitops-ai/dns-dangling-acme-example into main · 1 file · +0 −3
category:reliabilitysource:dnsrisk:lowxrev::LGTM
OverviewEvidencePlan & checksChanges
Summary

Three proxied A records in acme.example point at 203.0.113.10, the load-balancer address of the legacy ingress controller retired in the companion MR. No live load balancer holds that address and no route declares these hosts. Visitors currently get an error page.

Why delete, not repoint

None of the three hosts is declared by any live route on any cluster, so there is no correct target to repoint to. If one of them should stay, repoint it to 203.0.113.20 and add a route for it instead.

Zone mode

acme.example is managed: the repo is authoritative for the records it declares. Only these three declared records are touched; nothing undeclared is purged.

RecordPoints atLoad balancerRoute declaring the host
login203.0.113.10none (released)none on any cluster
preview203.0.113.10none (released)none on any cluster
grafana-old203.0.113.10none (released)none on any cluster

Only A, AAAA and CNAME records were joined. Every cluster's routes and addresses were read completely; a partial read would have produced an issue instead of this MR.

What the CI plan should show
  • 3 deletions in acme.example, no creations, no modifications.
  • No changes to page rules, Access applications or zone settings.
After the apply, expect
  • dig login.acme.example returns NXDOMAIN.
  • The next DNS reconciliation reports 0 dangling records for this zone.
Rollback

Revert this MR; CI recreates the three records as they were.

   records:
     - { name: shop,           type: A,     content: 203.0.113.20, proxied: true }
-    - { name: login,          type: A,     content: 203.0.113.10, proxied: true }
     - { name: api,            type: A,     content: 203.0.113.20, proxied: true }
-    - { name: preview,        type: A,     content: 203.0.113.10, proxied: true }
-    - { name: grafana-old,    type: A,     content: 203.0.113.10, proxied: true }
     - { name: www,            type: CNAME, content: shop.acme.example }
R
independent reviewer second opinion, no pipeline context

Confirmed the address is no longer allocated and that no route declares the three hosts. Diff touches only declared records. LGTM.

What keeps it honest.

DNS mistakes are public within minutes. The design leans on read-only access and explicit ownership.

Read-only assistantThe assistant never holds a DNS write token. CI applies what a human merged.
Declared-only deletesThe repo is authoritative only for what it declares. Objects managed out-of-band are reported as drift and never purged.
Mode, not tokenWhether a zone gets MRs or issues is an explicit per-account or per-zone setting, not inferred from which credentials happen to exist.
Address records onlyThe join ignores TXT and other record types that share a name, a bug class that produced false findings in the manual sweeps.
Partial reads abstainIf any cluster or zone could not be read completely, "no route" is not asserted. Unknown stays unknown.
Existing safety layerCloudflare findings already pass a token-echo scrub and an exact-match exempt list before they are emitted. The DNS lane reuses both.

Where this stands.

Live
  • Read-only Cloudflare access across accounts and zones
  • Cloudflare edge-posture review, issue-only and deterministic
  • Live Ingress and LoadBalancer Service reads on every cluster
In development
  • No code yet. The design is written and its scope questions (customer zones, non-exclusive management, optional write tokens) are answered
Roadmap
  • DNS repository with CI apply, imported from live (tool still to choose)
  • Deterministic record ↔ route ↔ address reconciliation, four buckets
  • Managed / advisory mode per account or zone
  • Repo ↔ live drift reporting
  • Retirement MRs citing the records they orphan
← Previous storyAlert silences as code