Continuous Delivery on Kubernetes with ArgoCD and GitOps

Push-based CD — where a pipeline holds cluster credentials and runs kubectl apply — works until you have more than one cluster, more than one team, and an auditor asking who changed what. At that point the model's weaknesses become structural. This article covers how to design a pull-based, self-healing delivery workflow on Kubernetes with ArgoCD, and the decisions that separate a demo from a fleet you can run at scale.

GitOps principles, and why pull beats push

GitOps rests on four properties: the desired state is declarative, it is versioned and immutable in Git, it is pulled by an agent in the cluster, and it is continuously reconciled so the running state converges on Git automatically.

The "pulled" part is the one teams underrate. With push CD, your pipeline needs cluster-admin-equivalent credentials sitting in a CI system that is, by design, exposed to the internet and to every contributor with pipeline edit rights. With pull CD, the agent runs inside the cluster and reaches out to Git; no external system holds kube credentials at all.

Dimension	Push CD (pipeline applies)	Pull CD (ArgoCD reconciles)
Credential location	In external CI	Inside the cluster
Drift handling	Until next deploy	Continuous, auto-corrected
Source of truth	Pipeline run logs	Git, always
Multi-cluster	N credential sets in CI	Agents pull independently
Rollback	Re-run old pipeline	git revert

The single biggest security win of GitOps is that no CI system needs standing access to your clusters. If your pipeline still runs kubectl apply against production, you have not adopted GitOps — you have adopted YAML.

ArgoCD architecture

ArgoCD is a set of controllers running in the cluster. The application-controller is the reconciliation engine: it compares desired state (from Git) with live state (from the Kubernetes API) and acts on the difference. The repo-server renders manifests — running Kustomize, Helm or plain YAML to produce the final objects. The API server backs the UI, CLI and SSO. State lives in Kubernetes itself (and Redis as a cache), so ArgoCD is largely stateless and recoverable.

The unit of work is an Application: it points at a repo path and a destination cluster/namespace, and defines how to sync.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: payments-prod
  namespace: argocd
spec:
  project: payments
  source:
    repoURL: https://github.com/acme/k8s-config.git
    targetRevision: main
    path: apps/payments/overlays/prod
  destination:
    server: https://kubernetes.default.svc
    namespace: payments
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true
    retry:
      limit: 5
      backoff: { duration: 10s, factor: 2, maxDuration: 3m }

selfHeal: true reverts any manual kubectl edit back to Git within the reconciliation window. prune: true deletes objects removed from Git — powerful and dangerous in equal measure, which is why you scope it carefully.

Separate the config repo from app repos

Keep application source code and deployment configuration in separate repositories. The app repo builds and pushes an image; a CI step then opens a commit or PR against the config repo bumping the image tag. ArgoCD only ever watches the config repo.

This separation matters because mixing them creates an infinite loop (a config commit retriggers the app build) and muddies access control — application developers should not need write access to production deployment manifests, and platform engineers reviewing a tag bump should not have to wade through application diffs.

App-of-apps and ApplicationSets

Managing one Application per service per environment by hand does not scale. Two patterns solve this. App-of-apps is a single parent Application whose Git path contains child Application manifests — bootstrap one, get many. ApplicationSets go further, generating Applications from a template plus a generator (lists, Git directories, or cluster registrations).

A matrix generator across clusters and environments:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: payments
  namespace: argocd
spec:
  goTemplate: true
  generators:
    - matrix:
        generators:
          - clusters:
              selector:
                matchLabels: { argocd.argoproj.io/secret-type: cluster }
          - list:
              elements:
                - env: staging
                - env: prod
  template:
    metadata:
      name: 'payments-{{.values.env}}-{{.name}}'
    spec:
      project: payments
      source:
        repoURL: https://github.com/acme/k8s-config.git
        targetRevision: main
        path: 'apps/payments/overlays/{{.values.env}}'
      destination:
        server: '{{.server}}'
        namespace: payments
      syncPolicy:
        automated: { prune: true, selfHeal: true }

Test ApplicationSet changes carefully. A bad template edit can generate or delete Applications across every cluster at once. Use preserveResourcesOnDeletion and review the generated set before merging.

Environment promotion

Model environments as Kustomize overlays (or Helm values files) over a shared base. Promotion is then a Git operation: a reviewed PR that copies the tested image tag from the staging overlay to the prod overlay.

# apps/payments/overlays/prod/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - ../../base
namespace: payments
images:
  - name: acme/payments
    newTag: "1.8.3"   # promoted from staging via PR
patches:
  - target: { kind: Deployment, name: payments }
    patch: |-
      - op: replace
        path: /spec/replicas
        value: 6

PR-based promotion gives you the audit trail and the approval gate for free: the diff is the change, and git revert is the rollback.

Secrets — never plaintext in Git

Manifests live in Git, so secrets cannot. Two production-grade approaches:

• External Secrets Operator (ESO) keeps the truth in AWS Secrets Manager, Vault or Azure Key Vault. Git holds only a reference; ESO materialises the Secret at runtime. This is the better fit for centralised secret management and rotation.
• Sealed Secrets encrypts values with a cluster-held key so the ciphertext is safe to commit. Simpler, but rotation and multi-cluster key management are more manual.

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: payments-db
spec:
  refreshInterval: 1h
  secretStoreRef: { name: aws-secrets, kind: ClusterSecretStore }
  target: { name: payments-db }
  data:
    - secretKey: password
      remoteRef: { key: prod/payments/db, property: password }

Ordering with sync waves and hooks

ArgoCD applies resources in waves and runs hooks at defined phases. Use them to enforce ordering — run a database migration before the new Deployment rolls.

metadata:
  annotations:
    argocd.argoproj.io/hook: PreSync
    argocd.argoproj.io/hook-delete-policy: HookSucceeded
    argocd.argoproj.io/sync-wave: "-1"

A PreSync migration Job at wave -1 completes before the application Deployment at wave 0. A failed hook fails the sync, so a broken migration never ships a half-applied release.

Progressive delivery and rollback

Built-in sync gives you availability during rollout but no traffic control. Argo Rollouts replaces the Deployment with a Rollout that supports canary and blue-green, gated by automated analysis.

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: payments
spec:
  strategy:
    canary:
      steps:
        - setWeight: 10
        - pause: { duration: 5m }
        - analysis:
            templates: [{ templateName: success-rate }]
        - setWeight: 50
        - pause: { duration: 5m }

The analysis step queries Prometheus for error rate or latency and aborts the rollout automatically if the new version regresses, shifting traffic back to stable. Because everything is still declared in Git and reconciled by ArgoCD, the failsafe rollback for any problem is the same: git revert the offending commit and let the controller converge.

Self-heal and a manual kubectl rollback fight each other — ArgoCD will immediately undo a hand-rolled rollback to match Git. Always roll back through Git, never through the cluster.

Sequencing the adoption

Install ArgoCD and bring one non-critical service under management with selfHeal off, so you can observe diffs without surprises. Add the secrets operator before any real workload. Turn on automated sync and self-heal once the team trusts the diffs, layer ApplicationSets when you have more than a couple of environments, and introduce Argo Rollouts last — progressive delivery is worthless until the reconciliation foundation is solid.

---

A self-healing, pull-based delivery platform is straightforward to demo and genuinely hard to run across a fleet of clusters and teams. Talk to i2zone and we will design your repo topology, ApplicationSet strategy and secrets model — then roll it out service by service so production never skips a beat.