Argo CD v1.6 — Democratizing GitOps with GitOps Engine

Today, the Argo CD team is happy to announce the first release of the GitOps Engine! GitOps Engine packages core Argo CD functionality into a reusable library and makes it available for everyone. The library is open-sourced and available on Github: https://github.com/argoproj/gitops-engine.

GitOps Engine empowers you to quickly build specialized tools that implement specific GitOps use cases, such as bootstrapping a Kubernetes cluster, or decentralized management of namespaces. In such cases, the user management and other management abstractions provided by Argo CD may get in your way and introduce unnecessary complexity.

Argo CD itself is also now rebased on GitOps Engine and remains focused on enterprise use-cases including:

• Core GitOps features. These include basic features such as reconciliation of the desired state, stored in Git, with that on a Kubernetes cluster, deviation detection of the differences between the two states, and advanced features such as health assessment and synchronization hooks.
• Features that deliver GitOps as a service to a large enterprise. Argo CD provides SSO integration, flexible access control system, and abstractions such as Application and Project implemented as Kubernetes CRDs. These features enable a centralized infrastructure team to move GitOps function into the infrastructure layer and provide it as a service to the whole organization.

The Argo CD team is also working with the CNCF Flux team in leveraging GitOps Engine for Flux: fluxcd/flux/pull/2886.

GitOps Engine

So what exactly is included in the GitOps Engine? The listing below demonstrates the top-level Golang packages that are included in v0.1.1:

.
├── agent         # provides CLI driven interface 
└── pkg
    ├── cache     # implements k8s caching
    ├── diff      # implements resource diffing
    ├── engine    # provides a high-level gitops interface
    ├── health    # implements resources health assessment
    ├── sync      # implements synchronization
    └── utils     # utility packages

Cache

https://pkg.go.dev/github.com/argoproj/gitops-engine/pkg/cache

During Argo CD development, the team encountered a lot of performance and scalability issues. Even the single cluster use case is very challenging and requires solving a lot of edge cases. Read this blog post to learn more about the journey. The cache package allows quickly getting the Kubernetes cluster live state to perform the reconciliation by storing lightweight resource metadata and leveraging the Kubernetes watch API to update the cache. This design significantly reduces the number of Kubernetes API requests and provides instant access to the required metadata.

Diff

https://pkg.go.dev/github.com/argoproj/gitops-engine/pkg/diff

The diff package helps reconcile resources stored in Git with live Kubernetes resources. The package uses a three-way merge and implements the logical equivalent to kubectl diff .

Health

https://pkg.go.dev/github.com/argoproj/gitops-engine/pkg/health

The health package enables resource health assessment. This allows Argo CD to find applications with Degraded deployments, allows efficiently integrate post-deployment verification into the CI pipeline, and is needed for Sync Hooks and Sync Waves. The v0.1.1 version includes support for only built-in Kubernetes resources. In future releases, we plan to include Custom Resource support, which already exists in Argo CD (argoproj/gitops-engine/#48).

Sync

https://pkg.go.dev/github.com/argoproj/gitops-engine/pkg/sync

The sync package leverages diff and health packages and implements resource synchronization functionality. Advanced features such as Sync Hooks and Sync Waves are also included.

Engine

https://pkg.go.dev/github.com/argoproj/gitops-engine/pkg/engine

The GitOps engine package is what pulls everything together and provides a high-level API for GitOps.

GitOps Agent

Finally, we need a good end-to-end example that illustrates how to implement a GitOps operator using the engine (https://github.com/argoproj/gitops-engine/tree/master/agent). The GitOps Agent leverages the GitOps Engine and provides access to many engine features via a simple CLI interface. The agent is a perfect fit for the use cases that require basic GitOps operations but don’t require SSO or other multi-tenancy features.

One demo is worth a thousand words! https://asciinema.org/a/168161

https://asciinema.org/a/334213

The GitOps Agent’s main purpose is to inspire you to use the GitOps Engine in your project and to gather community feedback. Please give the agent a try and let us know what would you want to see in the next version.

Argo CD + GitOps Agent

The Argo CD v1.6 release migrates Argo CD to the GitOps Engine. We are hoping that the migration simplifies the contribution of additional GitOps features, bug fixes, and ultimately accelerate project development. In addition to the GitOps engine refactoring, the v1.6 release includes several enhancements and bug fixes. Learn more about release changes in the release description: https://github.com/argoproj/argo-cd/releases/tag/v1.6.0-rc1 .

Thanks to the Community!

A huge thanks to everyone for your great feedback for improvements, help with testing, and contributions!