Many companies are struggling with the need for comprehensive load testing, the cost of load testing and limited budgets. This post will tell you about something you may not have considered: Load Testing with AWS Spot vs On-Demand Instances.
To begin with, RedLine13 understands the high cost of load testing. If you don’t know the RedLine13 story, that’s what led us to create RedLine13 for our own testing.
There are many different types of load testing that you may need to do, which just increases the demands. For those relatively new to load testing, this tutorial by Guru99 is full of good information on the many types: Performance Testing Tutorial: What is, Types, Metrics & Example.
The cost of load testing is what leads many companies to RedLine13 and significant savings. But there are savings being left on the table, even by RedLine13 customers. The reason is the choice of AWS instances being used for load testing.
AWS Spot Instances
If you only use AWS On-Demand Instances and don’t use AWS Spot Instances, then this is something to consider for saving more money when load testing. When deciding between AWS Spot vs On-Demand Instances you first need to understand about AWS Spot instances.
From the AWS website:
A Spot Instance is an unused EC2 instance that is available for less than the On-Demand price. Because Spot Instances enable you to request unused EC2 instances at steep discounts, you can lower your Amazon EC2 costs significantly.
Are there any negatives to Spot Instances? AWS says:
Spot Instances are a cost-effective choice if you can be flexible about when your applications run and if your applications can be interrupted.
AWS gives three reasons to use Spot Instances:
- Low, predictable prices
- Massive scale
- Easy to use
Pricing – AWS Spot vs On-Demand Instances
AWS says:
You can purchase Spot Instances at prices up to 90% lower than On-Demand Instances.
RedLine13 customer KMI Learning, based in Columbus, Ohio, builds custom eLearning solutions, and uses AWS Spot Instances. Eric Blevins, Systems Engineer at KMI Learning:
Redline13 has made it easy for us to save money by making Spot Instance bidding a built-in part of the RedLine13 load testing system.
See what you can save BEFORE you start your test
RedLine13 makes it easy for you to see what you can save before you start your test. When you start a test, RedLine13 displays the anticipated hourly cost for On-Demand and for Spot.
New RedLine13 Enhancement for Load Testing with AWS Spot vs On-Demand Instances
If AWS Spot Instances are what is best for you, RedLine13 lets you set the default to Spot Instances in your settings. Just go to Account > Your Settings and click on “Prefer spot instances”:
AWS Spot Instance Limits
Of note especially important as you scale your test, your AWS account will need the appropriate Spot Instance limits in place to ensure you have the capability to provision what you request. The limits (or “Service Quotas“) are applicable to all AWS accounts, though your specific limit will vary on how long you’ve had your account with AWS, as well as your particular use of AWS services.
You may encounter the following message on RedLine13 as you scale your test:
To increase your Spot Instance limit, your will need to log into your AWS Console, open the Service Quota Dashboard, and search for “EC2“. You will find the appropriate “Spot Instance Requests” item for your instance type. In the example below, this limit is set to 640 vCPUs:
Clicking on “Request quota increase” will open a form where you can specify a new limit. Usually, AWS will respond to these requests quickly (often as soon as minutes to hours). Once the new limit is in place, you will be able to utilize as many spot instance vCPU cores as you have specified. Service quota increases are also explained in more detail in a related post.
See for yourself how much you’ll save with AWS Spot Instances and if it is right for you. You can easily run your own RedLine13 test and check out Load Testing with AWS Spot vs On-Demand instances.