A question sometimes comes up: can you load test an AWS hosted site, that uses products such as AWS WAF, AWS CloudFront, and the AWS Elastic Container Service, with AWS Load Generators?
The concern is that AWS networking may optimize routes and utilize internal large network pipes, etc. The premise is that that will render the results inaccurate compared to real world results.
Let’s look at Load Testing AWS from AWS and what you need to know.
I won’t bury the lead. Load testing AWS from AWS can be done without worrying about network optimizations. You can easily verify this, but if you still feel avoiding the optimizations will give you a better sense of response time, then you have options.
Options to Load Test an AWS Hosted Site
1) Test from a different region and set up a secondary account just for running load tests. Testing across accounts or regions will force IP traffic over public routes that will land back in AWS. Remember, AWS networks not only have optimizations for performance, but a lot of rules about keeping network traffic separated to protect account security. With all the AWS regions available, there are a lot of options.
We have performance tested AWS region to region communication for a large scale streaming service, specifically for passing text messages through real-time content classifiers. The classifier runs around 30ms if you call from within the region, and almost 180 ms if you call from EU to US-EAST. So calling between regions definitely has public internet latency.
2) Add latency or control throughput. Since you cannot accurately simulate a user’s browser or mobile phone, use generated latency. Every home connection, mobile connection, or that connection we use at Starbucks has so many variables that impact actual network flow. JMeter has a feature to control bandwidth so you can create different kinds of connections.
This can be passed into RedLine13 via command line options.
Another idea is not to worry about network costs in load testing. This very much depends on what you are trying to test. If latency is not controllable, we try to improve and manage the performance time of our services. We look at API response times of /api or /registration or /something else.
What we are really trying to determine and optimize on is the SLA of ‘My Service’ (see below). Reducing or controlling for latency gives us a better understanding of actual performance time. That gives us options to load test an AWS hosted site.
