Argo Events v1.6: More Powerful Than Ever

Derek Wang
Argo Project
Published in
4 min readFeb 15, 2022

Argo team is pleased to announce the release of Argo Events v1.6, which consists of over 20 enhancements and dozens of bug fixes contributed by over 30 developers.

Let’s take a look to see what’s new!

Photo by Andrey Metelev on Unsplash

EventSource Filtering

Filtering features are greatly strengthened in v1.6, which include a new filtering capability that is available to all the EventSources.

EventSource filtering allows you to specify a filter expression, so that it only ingests the data that you are interested in. This will improve performance especially when you have a lot of events.

Example: an online store puts daily sold item information into a Kafka topic, as shown below:

Event 1: {"id": 102,"name": "Electric Kettle", sold: 4, "category":{"id": 12,"name": "Kitchen"}}Event 2: {"id": 103,"name": "Real Simple", sold: 6, "category":{"id": 15,"name": "Books"}}

If we want to do something when any book has sold more than 5 copies, we can put an EventSource spec like below.

apiVersion: argoproj.io/v1alpha1
kind: EventSource
metadata:
name: kafka
spec:
kafka:
example:
url: kafka-broker:9092
topic: topic-2
jsonBody: true
filter:
expression: "(body.sold > 5) && (body.category.name = 'Books')"

In this case, the first event about “Electric Kettle” will be dropped, only the second one will be picked.

The filtering capability can also be achieved in Sensor, but giving another option to do it in EventSource make it more flexible, and will reduce redundant data from going into the internal EventBus in case the event source has a lot of data you don’t want.

Thanks to @dpadhiar for implementing this feature!

Sensor Filtering

Filtering in Sensor is also enhanced in the new release. This allows you to specify logic operators and or or for multiple filters you specified.

Logic operator for data filter:

filters:
dataLogicalOperator: "or"
data:
- path: "a"
type: "bool"
value:
- "true"
- path: "b.c"
type: "number"
value:
- "3.14"

Logic operator for expr filter:

filters:
exprLogicalOperator: "or"
exprs:
- expr: a == "b" || c != 10
fields:
- name: a
path: a
- name: c
path: c
- expr: e == false
fields:
- name: e
path: d.e

Or, if you have a complex filtering configuration where both data filter and expr filter are configured, you can specify a top level logical operator for them. Check here for more details. Many thanks to @bygui86 who made the enhancements and refined all the filtering related documents.

spec:
dependencies:
- name: sample-dependency
eventSourceName: sample-dep
eventName: sample-event
filtersLogicalOperator: "or"
filters:
...

Event Transformation

The event data can now be transformed in the Sensor before applying filters, or being used as parameters for triggers. The transformation can be done either through Lua Script, or JQ expressions.

The example below uses a Lua script to set body.message to be updated for all the events before they are further processed.

dependencies:
- name: test-dep
eventSourceName: webhook
eventName: example
transform:
script: |-
event.body.message='updated'
return event

The following case shows an example to double the value of body.message .

dependencies:
- name: test-dep
eventSourceName: webhook
eventName: example
transform:
jq: ".body.message *= 2"

Check the details to get more information about event transformations. Using transformations together with Sprig templates to pass parameters to triggers can manage very complex use cases.

Credits to @VaibhavPage from Blackrock for the implementation.

Trigger Conditions Reset

Trigger conditions reset was first introduced in v1.5.1 as a new feature, it is now GA in v1.6. It’s very useful for the case where you have a trigger condition like “A && B”, however A failed to emit an event but B succeeded, and you want to reset them to a fresh start state. For this case, you could do something like below:

spec:
triggers:
- template:
conditions: "A && B"
conditionsReset:
- byTime:
# Reset conditions at 23:30
cron: "30 23 * * *"
# Optional, defaults to UTC
timezone: America/Los_Angeles
name: my-trigger

BitBucket EventSources

BitBucket Server

BitBucket Server EventSource now supports configuring multiple repositories in one object, check this example for the details.

BitBucket Cloud

In addition to BitBucket Server, BitBucket Cloud EventSource implementation is also added in v1.6, it becomes the 26th EventSource that Argo Events supports. Check the example for the spec detail if that is helpful to you.

Tons of thanks to @daniel-codefresh for the contribution.

Github EventSource

Github EventSource also got improved.

  1. Github EventSource is now able to do organization level webhook configuration as well as repository level.
  2. Github App Credential can be used in the EventSource for authentication in addition to API Key.

These improvements were also contributed by @daniel-codefresh.

Installation

For installation and more detail about the release, check the release notes.

Thanks to everyone in the list below who made contributions to the new version! And special thanks to everyone in the community who helped for testing, raised issues and feedback, all of these made Argo Events more powerful and better than ever!

@daniel-habib
@mintbomb27
@usamaB
@christophercutajar
@hblixt
@juliev0
@blkperl
@Nizar-opt
@Capable-Disk4147
@hamza3202
@bygui86
@icecoffee531
@VaibhavPage
@tooptoop4
@xi2817-aajgaonkar
@arnarg
@dpadhiar
@ntdt
@mweibel
@mseiwald
@alexec
@dubeau
@daniel-codefresh
@kromanow94
@whynowy
@jcantosz
@terrytangyuan
@sharon-codefresh
@anencore94
@luca-iachini

Sign up to discover human stories that deepen your understanding of the world.

No responses yet

What are your thoughts?