Argo Workflows & Events 2023 User Survey Results

We’re pleased to present the Argo Project user survey results for 2023. Conducted in April & May, and with responses from a diverse range of users, the survey sheds light on which features and capabilities of Argo are most valued, as well as areas where improvements and enhancements could be made.

This year, 207 respondents offered a view into their usage and adoption of Argo Project tools. (155 responses for CD/Rollouts, 52 responses for Workflows/Events).

In this post, we summarize the key findings from the survey for Argo Workflows and Events, and explore what they mean for the future of the projects. You can check out the Argo CD & Rollouts survey results blog post here on Medium as well.

Argo Workflows / Events Survey Overview

Argo Workflows reported an NPS of 27 in this year’s survey, which skewed towards newer Argo Workflows and Events users than prior years. This year’s survey indicated 58% of respondents are running Argo Workflows in production, while 42% are still evaluating the tool. An extraordinary 81% of respondents also use Argo CD or Argo Rollouts alongside Argo Workflows and Events, reflecting how well the tools integrate with one another.

Why do users like and use Argo Workflows and Events?

Users had various reasons for favoring Argo Workflows and Events depending on their use cases and functional disciplines:

• “Very generic, versatile, Kubernetes-native, no vendor lock-in”
• “Enables better cloud resource utilisation than MLflow or Kubeflow.”
• “We needed a scheduler for various tasks and found it integrated nicely with Argo CD.”
• “Flexible architecture”
• “Easy to learn, simply works”
• “Easy DAGs and Kubernetes-native”
• “Perfect stepping stone to migrating E2E tests to Kubernetes and take advantage of auto scaling given the bursts nature of E2E tests”

Job Roles & Use Cases

Given its generalizable utility, Argo Workflows and Events are used by a variety of engineering disciplines, with a small uptick in DevOps Engineers (and corresponding drop in Architects) over last year’s survey:

• DevOps Engineers: 44%
• Software Engineers: 23%
• Architects: 15%
• Data Engineer / Data Scientist / ML Engineer: 13%

However, use cases were distributed much more evenly than we would expect to see based on job titles alone. This year’s survey indicated that several use cases have increased for Argo Workflows & Events compared to prior years, including ETL, Machine Learning, and CI/CD.

This may imply that both DevOps and Software Engineer generalists are flexing their expertise with Argo tools to solve data and machine learning engineering challenges in addition to their work with CI/CD pipelines and platform engineering.

Production Usage

This year’s survey showed a large increase in newer users, as the production usage among respondents, 58%, was lower than last year’s number of 75%.

Scale and Size

This year’s survey showed an increase in the median number of workflows running per day and median number of pods per workflow compared to last year. As with last year, the majority of users (85%) are running fewer than 1000 workflows a day, and fewer than 1000 pods per workflow (75% of users).

However, a higher percentage of users are running between 100–1000 workflows per day compared to last year’s survey. Additionally, this year saw a slight increase in the percentage of users running over 1000 pods per workflow. Overall, this year’s survey data indicate that users are steadily increasing the scale of their Argo Workflow operations year-over-year.

Ecosystem

This year’s survey included new tooling options for users to select from. The stand-out tool from the survey is Argo CD/ Rollouts, which 81% of respondents said they use with Argo Workflows. In prior years, we had not been keeping track of the cross-pollination between the Argo Workflows and Argo CD projects, but this year’s survey makes it clear how common it is to use two or more of the Argo tools together, regardless of one’s function or use case.

Three of the next four most popular tools used with Argo Workflows and Events are data engineering and/or data science tools: Jupyter Notebooks, Apache Kafka, and Airflow.

Kyverno is another tool that we started surveying for this year, and the survey confirms its popularity among Argo users for managing policies on Kubernetes.

Opportunities

Responses to this year’s survey highlighted a number of growing trends that present opportunities for the Argo Workflows and Events projects to serve.

First, we’re seeing more DevOps and Platform Engineers start to use Argo Workflows to support both data/ ML engineering use cases and CI/CD use cases. Adding more documentation around CI/CD use cases for Argo Workflows + Argo CD could be considered an opportunity. Additionally, adding features to Argo Workflows and Events to bring them closer in parity to other legacy data engineering tools could be considered another opportunity.

Second, the survey showed there’s increasing interest in Argo Workflows among users who have not yet taken the project to production. This likely reflects a growing interest across DevOps/ Platform Engineers, Software Engineers, and Data/ML Engineers in finding a cloud-native orchestration solution for all types of DAG-based workflows. While it seems Argo Workflows is considered the go-to cloud-native tool for the job, these users left feedback that the docs and UI could be improved to make the first-time user experience better. In response to this, the project’s maintainers are investing in docs and UI/UX improvements this year. As always, PRs welcome!

Third, the survey showed a small upwards trend in the scale of workloads that users are using Argo Workflows to run. Another opportunity for Argo Workflows and Events is to consider what challenges lie ahead for supporting workloads at the extreme end of the scale to ensure they remain the go-to cloud-native solutions for running all types of workflows on Kubernetes.

Final Words

A big thank you to everyone who completed the Argo surveys this year! Your feedback is critical in helping us shape the future of the Argo Projects. As contributors and maintainers, we are dedicated to improving these tools to meet the evolving needs of users like yourself. We look forward to sharing more updates and insights with the Argo community in the coming months. Thank you for being a part of this journey with us!

An extra big THANK YOU to everyone who also showed a willingness to come and present at Argo community meetings, do a user study, provide a quote, or write a blog post. We will be following up over email with you to discuss how to get more involved. As always, the community welcomes everyone to participate and contribute!

And finally, a thank you to Michael Crenshaw, Tim Collins, Katie Lamkin, Harriet Lawrence, and Henrik Blixt for their time in helping facilitate this survey and blog post.