Depending on your testing goals, whether using JMeter or something else, pushing your target application to a point of ultimate failure may be one of your objectives. In this post we will discuss what signs you might see in your load test when your target test application becomes “overwhelmed”. We will also explore some cases where problems in the test or the network can mimic an overwhelmed state.
The extreme scalability of a cloud-based architecture enables a simple load test to be duplicated dozens or even hundreds of times. This is a strong feature of RedLine13, as we make this type of scaling nearly effortless. Unfortunately computing resources are not limitless but rather carefully allocated. One goal of performance testing is to determine the ultimate capacity of these resources. We must also be mindful that network resources can cause the system to fail in similar ways. It is therefore important to distinguish limits encountered in both limits and the target test application.
When running your load tests at scale, there are a few things you should watch for:
Increased response times:
While increased response times would be expected when the target test system becomes overwhelmed, there are other scenarios in which this also manifests. An example would be a network bottleneck or otherwise limited network resource. It can also be potentially caused by load generators with inadequate resources, however it is less likely this would go unnoticed.
Examples causes:
- Network bottleneck → A network bottleneck or an overtly slow network connection can be a factor increasing response times which is unrelated to the capabilities of the target test application system.
- True performance problems → Discovering this is a primary goal of performance testing. We may see increased response times due to inefficiencies in application logic, inadequate processing resources (see below), and other organic problems with the target test application.
Excessive errors – especially “Unexpected Response” from server errors:
Errors especially of this type can be associated with an overwhelmed system that gives partial, invalid, or empty responses to requested resources. Though such can be experienced with access control lists (ACLs) and other network security entities, receiving unexpected response errors can be highly suggestive of an overwhelmed system state.
Examples causes:
- Inadequate processing resources → this is a case where the target application server is truly overwhelmed. An instance size should be selected that affords processing of the requests. This can be CPU, and similarly be extended to memory and disk space.
- Network adapter selection → if an instance network adapter lacks the bandwidth for the number of requests or amount of data requested, it can mimic the signs of an application server which lacks processing power to complete requests.
- Firewall or Network ACL → if network requests are denied or blocked before they can reach their destination, this too can cause unexpected responses which might at first glance seem similar to an application server behaving as if it were overwhelmed.
Caveat: Excessive errors could be due to dDOS attack prevention or other similar network policy configuration.
Test takes significantly longer than expected to complete:
Equally so this may be a symptom of mismatched load generators or the target test application to a requested load. When tests exceed their expected duration, any cause should be suspect.
Examples causes:
- Processing is delayed → A test which eventually completes may represent a network bottleneck, or true performance problems with the target test application (see above for explanations of these).
- Problems with test plan → one factor that is frequently overlooked are issues with the actual test plan. Whether we programmed more requests than anticipated, or inadvertently configured the test to last longer, we should investigate the test plan itself before making the assumption that there is a problem with the system being tested.
If you don’t already have a RedLine13 account, you can sign up for a no-cost trial subscription.