Using Argo CD with vclusters

Managing deployment to multiple Kubernetes clusters is a core feature of Argo CD, but developing and testing external cluster deployment scenarios can be challenging because of needing to set up multiple clusters. Luckily, there is a tool which helps simplify creating clusters that works great with Argo CD.

vcluster is an open source project that allows you to create Kubernetes clusters on a host cluster. This means you can use a Kubernetes cluster to host multiple Kubernetes clusters.

The idea of vcluster is discussed in greater detail on the vcluster webiste, but a summary from these docs is as follows for what vclusters are:

vclusters run as a single pod (scheduled by a StatefulSet) that consists of 2 containers:

• Control Plane: This container contains API server, controller manager and a connection (or mount) of the data store.
• Syncer: What makes a vcluster virtual is the fact that it does not have a scheduler. Instead, it uses a so-called syncer which copies the pods that need to be scheduled from the vcluster to the underlying host cluster.

vclusters are also a certified Kubernetes distribution, so the experience of using these clusters is pretty indistinguishable from other clusters.

Hopefully vclusters sound interesting to you, but let’s put them to the test by using Argo CD to deploy to a vcluster.

Installation

Let’s start by installing Argo CD to a Kubernetes cluster. This walkthrough assumes that your cluster can support a Service of type LoadBalancer.

You can use the core installation of Argo CD and the argocd CLI for interacting with the Argo CD instance:

kubectl create ns argocd
kubectl apply -f https://raw.githubusercontent.com/argoproj/argo-cd/v2.2.5/manifests/core-install.yaml -n argocd
kubectl config set-context --current --namespace=argocd
argocd login --core

Next, let’s install the the vcluster CLI. NOTE: The vcluster CLI also requires helm to be installed.

Once these tools are installed, you are ready to create a vcluster.

vcluster Creation

Creating a vcluster is as simple as running the command below:

vcluster create vc-argocd -n argocd --expose

Once the command completes, you should have a vcluster running on your host cluster. In order to access it using kubectl, you can run the following command:

vcluster connect vc-argocd -n argocd

After running the command, you will have access to a kubeconfig file named kubeconfig.yaml. You can make use of it with kubectl like the following example:

kubectl get ns --kubeconfig=kubeconfig.yaml

If you run the command above, you will notice the argocd namespace isn’t present on this cluster. It’s a fresh, independent cluster that is ready for us to use.

Configure vcluster with Argo CD

With the cluster created, you will want to register it with Argo CD. You can do so by running the following command to add the vcluster context:

argocd cluster add Default --kubeconfig=kubeconfig.yaml

The argocd CLI will prompt you to install RBAC on the vcluster to give Argo CD permission to create resources on the vcluster. You can enter y to allow the process to proceed. Once the command completes, the vcluster is configured with Argo CD.

Deploy to the vcluster

All the setup should be complete, so the last step is creating an Application to deploy to the vcluster:

argocd app create guestbook --repo https://github.com/argoproj/argocd-example-apps.git --path guestbook --dest-name Default --dest-namespace default

If you run argocd app get guestbook after the Application is created, you can see the Application Sync Status is OutOfSync. So it looks like Argo CD is able to properly recognize the differences between the git repository and the vcluster.

Go ahead and sync the Application to deploy to the vcluster:

argocd app sync guestbook

With the sync complete, you can check that the Application didn’t deploy the sample app to the host cluster’s default namespace:

kubectl get deployments,pods -n default

You shouldn’t see any Deployment or running Pods in the namespace. However, if you use the kubeconfig for the vcluster, you should see the sample app was successfully deployed:

kubectl get deployments,pods -n default --kubeconfig=kubeconfig.yaml

Conclusion

Argo CD and vclusters offer a lot of exciting possibilities when used together. The ability to easily spin up clusters to develop and test out external cluster deployment scenarios with Argo CD is really empowering. It lets developers not be blocked by needing multiple clusters, provides greater confidence in terms of creating multi-cluster set ups with Argo CD, and helps to reduce costs of using extra clusters.

I hope this blog helped you to learn more about vclusters and Argo CD. If you would like to see this walkthrough in a video, you can watch the video below to see everything in action. I would love to hear your ideas and thoughts on this topic. Thanks for reading!