In the decades since the very first release of JMeter, there has been considerable collective experience gained not only in best practices for performance test design in general, but also in the specific methods for creating JMeter tests. In this article, we will summarize best practices for creating a JMeter test plan. For further reading, a more in-depth discussion about best practices can be found in the JMeter users manual.
Targeting the Latest Version of JMeter
The general recommendation is to use the latest JMeter release available for your test. This will allow you to take advantage of the latest performance improvements, as well as bug fixes and security patches. The advice is to avoid versions of JMeter which are older than the previous 3 releases.
Balancing Performance and Thread Count
Each test plan is unique, and the performance requirements of each JMeter test will vary. Given a hardware configuration (i.e., load generator server size) the number of threads will need to be matched. Some important considerations when choosing a thread count for your test include:
- Response Time Duration – endpoints in your target test application that return faster cause increased demands on JMeter. This may seem counterintuitive at first, but for tests against simple endpoints that return a quick response, the number of concurrent threads might need to be decreased to match performance.
- Complexity of Pre- and Post-processing – heavy logic that runs before or after your requests will consume resources on your load generator server. As a result, you may find that decreasing the thread count on a per-server basis and spreading load out amongst a larger number of servers is the route to go.
- Long-running Requests – for cases where a remote server takes a long time to process a request, this could justify increasing the thread count on your load generator servers. You can take advantage of otherwise idle time waiting for responses to long-running requests.
In addition to adjusting absolute thread counts, there are a variety of thread-focused plugins that JMeter offers. These can spread thread loading over time, or even profile the number of threads at different points in your test to attain maximal efficiency on your load generator servers.
Listeners and Overhead
When creating and debugging your test, it is necessary to employ aspects of JMeter which incur a CPU overhead cost prior to moving your test into production. The largest is probably the graphical user interface (GUI) which is a necessary component to design your test. In a previous article, we talked about how production tests should dispense with the GUI and run in non-GUI mode. Another component that is usually run while designing tests are listeners. The most common of these is the View Results Tree, which provides basic response viewing capability. Any listener that is added to a JMeter test plan should be removed or disabled prior to moving your test into production, as the added CPU overhead can detract from your test results by negatively impacting load generator performance.
Legacy Scripting
If your test contains scripted elements (e.g., JSR223 Samplers), the current advice is to migrate any legacy BeanShell scripts to Groovy. There is a significant performance penalty to using the order scripting framework. In addition to this, Groovy also has the ability to benefit from caching and compilation if these features are enabled.
Test Parameterization and Properties
While not conferring a direct performance benefit, it is recommended best practice to avoid hard-coding commonly changed configuration settings and instead use parameterization. This can be done a number of ways, from basic property variable setting, to CSV parameterization from a data set. In a previous article, we illustrated how to use Advanced JMeter Test Options in RedLine13 to achieve easy parameterization of your tests by setting simple command-line arguments.
Avoiding Deprecated Elements
Much along the same lines as using the latest release of the core JMeter application, it is advised best practice to avoid using elements and plugins which have been deprecated. You can find a list of currently supported and recently deprecated elements in the JMeter component list. As the ecosystem of supported functionality for JMeter evolves, newer components are sometimes added which improve upon previously offered functionality.
Did you know that RedLine13 offers a full-featured, time-limited free trial? Sign up now, and start testing today!