Use Spot Instances with your Kubernetes Clusters on AWS

Written by Shrinand Javadekar, Ed Lee

Spot instances provide an attractive option to use cost-effective compute. They can be up to 10x cheaper than on-demand instances. However, they do come with their own caveats and limitations. To use a spot instance on AWS, the user has to continuously submit a “bid price”. If this “bid price” is ever less than the “clearing price” determined by AWS, the spot instance will be terminated.

Short-lived jobs and workflows are perfect use cases for leveraging spot instances. Further, any application that can tolerate node failures can also benefit from using spot instances. This is especially true when running applications on a cluster, such as Kubernetes, that can automatically restart failed jobs and applications.

Let’s look at spot price trends

The graph below shows the prices of “m3.large” instances for the months of Jun-Aug 2017 in the us-west-2 region. The price of a m3.large spot instance is approximately $0.026/hour whereas that of an on-demand instance is $0.133/hour. Notice that the on-demand instance price is constant, but the spot instance price has had few fluctuations with a spike to $0.03/hour. Still, continuously over a period of three months, the spot price has always been significantly below that of the on-demand price.

“m3.large” Spot Instance prices for the months of June — August 2017 in AWS US-West-2 region

Given this data, using an on-demand instance is ~5x more expensive than using a spot instance during this period. That is, if we had set the bid price to, say $0.035/hour, we could have run the same EC2 instances at 1/5th the price of an on-demand instance. However, the bid prices can have really large spikes, leading to losing instances.

The graph below illustrates spikes in spot instance prices. Note that prices start to spike in the us-west-2a region but then spreads to the us-west-2c and finally the us-west-2b region. The price spikes can recur regularly over periods of days or even weeks and can actually exceed the on-demand price! In the example below, the spot price reaches 10x the on-demand price.

Fluctuations in “r4.large” Spot Instance price

In order to achieve the best results, the Keiko Project has a “minion-manager” that bids for spot instances using the AWS pricing API and automatically switches between spot and on-demand instances as the spot price approaches the on-demand price. This ensures that you never pay more for spot instances compared to on-demand and also that you are never left without any instances, because of high spot prices, for very long.

Try out the spot instance minion-manager for yourself

[UPDATE JULY-2018] The minion-manager is now available as a separate github project here. See instructions there for how to run

[Old blog content starts here …]

If you would like to try out spot instances with your Kubernetes cluster on AWS, here’s the YAML definition for running the Argo minion-manager on your cluster.

• Copy the file onto your local machine, as say minion_manager.yaml
• Edit the <aws_region>. E.g. MM_REGION: “us-west-2”
• The minion-manager operates on one or more AutoScalingGroups (ASGs). Put the names of the ASGs as a space separated string for which you want to use spot-instances for the MM_SCALING_GROUPS option. (i.e. replace <asg_1> <asg_2>). E.g. MM_SCALING_GROUPS: “my-k8s-asg”
• Run kubectl create -f minion_manager.yaml

The minion-manager should be up and running!

Under the hood

The following flowchart explains the workings of the minion-manager. It provides a high level overview of how the changes to the spot prices and switching between on-demand and spot instances work.

Basically, this is what happens:

• The minion-manager periodically checks if the desired number of instances are running in the ASG.
• If the desired number of instances are not running, the minion-manager looks at the pricing information of both the on-demand and the spot instances. If at that point in time, the spot instance prices are less than on-demand instance prices, the minion-manager updates the launch-config with the spot-instance price. Otherwise, it will change the launch-config with the on-demand instance price.

Check out the code here for details.

“Bimodal” (a.k.a Basic) bid strategy

The spot instance prices on AWS rarely change gradually. For a given instance type, the price is usually either very low or very high relative to the on-demand price.

This observation simplifies the bidding strategy. The minion-manager has access to both the on-demand and spot instance prices. It finds the lower of the two prices and chooses instances of that type.

If the spot instance price is higher than the on-demand price, the minion-manager periodically checks to see when the spot instance price drops below the on-demand price. As soon as it does, it

• Changes the launch-config to use the spot-instance bid price
• Terminates the on-demand instances near the end of the current billing hour (since AWS charges by the hour, there is no point in terminating an instance before the end of the billing hour)

Spot instance prices are a result of how users bid for and use spot instances as well as internal Amazon pricing policies. As more people discover the value of spot instances and implement strategies to take advantage of them, pricing patterns will change. This will lead to more interesting and sophisticated bidding strategies in the future.

If you try out spot instances with your Kubernetes cluster or try out Keiko, give us a shout at @keikoproj.

Shrinand Javadekar is a member of the technical staff and Ed Lee is the CTO and cofounder of Applatix, a provider of products and training, support and consulting services for Kubernetes and containers.