Performance engineers know how to run the load testing which is used to determine when a system is ready for production. When the system doesn’t handle the load, someone needs to determine why. yCrash answers performance testing questions: Why did the CPU spike up? Why is memory degraded? Why is response time increased?
RedLine13 and yCrash partnered together as valuable tools that complement each other. RedLine13 handles the generation of load data and collection of test results. As a performance testing platform, the metrics RedLine13 collects are at the network and system level. yCrash captures more stratified analytics reaching to the application level – with examples including garbage collection logs, thread dumps, and heap dumps. It also analyzes network statistics, and gathers several more artifacts from the application. yCrash automatically captures and analyzes these data points to instantly identify the root cause of potential problems. In other words, yCrash answers performance testing questions by drawing high-level conclusions from low-level granular data.
To demonstrate the complementary utility, the folks at yCrash ran a load test powered by RedLine13. In the yCrash post, they compiled the results and analysis. You can read about the environment setup and JMeter test plan details in their post.
You can also read the RedLine13 report below which shows you information about the load test and how it complements the yCrash results.
RedLine13 Results
The actual results of this example are publicly available, and we have shared these results here. Within the RedLine13 results we can see that load agent CPU utilization was pegged at 100% for nearly the entire test duration:
While it appears the test was still able to generate sufficient load for the test, the maximal load for that agent size was likely somewhat limited. As a general recommendation, we advise that load agent CPU utilization be kept below 100% at all times, with an ideal safe target around 70%-80% maximum sustained.
This finding in particular is mirrored in the yCrash report, where we see peak average CPU utilization approaching 100%. What isn’t immediately apparent are the memory inefficiencies which yCrash also reflects. This is elucidated in plain language and show in the following “Memory wasted” graphical breakdown:
Incidentally you may also notice that in this particular load test there are a relatively large number of errors indicating “Embedded resource download error…“. This is another example of how RedLine13 complements yCrash to provide a more comprehensive evaluation of your test. In this case, the majority of these errors are simply the result of the test being configured to download all linked resources at the endpoint web page. In this case it does not seem to affect the overall test, however it ends up with a greater number of reported errors at the end of the test. It is worth mentioning, as if you encounter this it can be prevented by modifying the test plan and deselecting the “Retrieve All Embedded Resources” option on the “HTTP Request Defaults” config element:
Conclusion
We have shown how yCrash elegantly complements RedLine13 test results to provide deeper dimensions of analysis to your load tests. The examples we described here in our sample test illustrate a few of the many ways these companion products can provide greater insight in deriving meaning from load test results.
Did you know that you can test drive RedLine13 with our free trial? Sign up and start testing today!