Load testing is a niche skill within the large trade of quality assurance. With any trade, being able to communicate effectively the nuances and concepts used is essential. In this brief post, we will go over several definitions that will allow you to speak the lingo using load testing terms. Below we have divided these into categories for better organization:
General load testing terms
Load generator server – Each load test is run on one or more virtual machines in the cloud. You can think of a load generator server as a complete encapsulated version of your test. To achieve scale, you can simply add more load generator servers to multiply your load.
Virtual user – Each simulated user in a load test is referred to as a virtual user, and is the test-plan equivalent of a real user. Each load generator server may have multiple concurrent virtual users accessing endpoints at the same time.
Thread – For many tests (including JMeter tests) a thread is equivalent to a virtual user. More precisely however a thread represents a single process within a load test that may contain one or more virtual users.
AWS – The cloud framework we use for test execution is Amazon Web Services, abbreviated to AWS for short. With RedLine13, you create your own AWS account, and integrate with our platform.
EC2 – This is a specific service within AWS called Elastic Cloud Computing, or EC2 for short. Each EC2 instance corresponds to a load generator server.
On-demand instances – This term refers to one of two types of AWS EC2 instances used for load generator servers. For on-demand instances, load generators are acquired and provisioned at the time your test runs.
Spot instances – The other type of AWS EC2 instance is called a spot instance. These are available at a potentially lower price than on-demand instances, however their availability is not guaranteed. You can read here about the benefits and drawbacks of spot instances.
Load test scaling – When increasing the load that your test generates in the cloud, you can scale your test by adding additional load generator servers. Each additional load generator server proportionally multiplies the capacity of your test by that which you have defined in your test plan.
Load test types
Stress test – Typically this is a ramped test that will generate increasing load in order to determine if a target test application can tolerate, or otherwise determine a breaking point.
Soak test – This refers to a test that sustains a continuous load over a protracted period of time, in order to determine if a target test application can support such a sustained load.
Performance test – Otherwise known as simply a load test, a performance test generates a predetermined load against a target test application in order to determine if that application can handle the load.
Functional test – This type of test seeks to validate a use case for a target test application, without necessarily attaining a predetermined load.
JMeter specific terms
JMeter – Apache JMeter is one of the most popular tools for creating a load test. You can read more about ruining JMeter tests in the RedLine13 Owner’s Manual.
JMX file – Test plan files created in JMeter are saved as *.jmx files. When running a JMeter load test on RedLine13, this is the file you have prepared ahead of time that contains your test instructions.
Dashboard report – Results from your JMeter tests can also be viewed using the JMeter Dashboard Report. This is a standardized HTML-based report that is familiar to most users of JMeter. It complements the results page visible in RedLine13, and can be downloaded and shared in a portable format. We also talk about specific use cases for the JMeter Dashboard Report on our blog.
RedLine13 specific features
Output files – When reading about load testing on our blog and in our online documentation, you may see mentions of enabling “output files”. These comprise extra data collected including logs that have many uses once your test completes. One feature of this is the JMeter Dashboard Report, which provides complimentary results to your load test beyond what we display on your results page. Another useful feature are log files which enable you to more easily debug your load tests once they are running in the cloud.
Plugins – Redline13 is an extensible platform that you can add plugins to gain additional functionality. You can view all available plugins from the “Your Plugins” page and control which ones are enabled for your load tests.
Agent metrics – On your results page, there is a specific section titled “Agent Metrics”. This area streams real time data about your load generator servers indicating their performance health. You can use this information to size your load generator servers and determine how many load generator servers you will need.
Specific use cases
Headless testing – This can also be referred to as request-based testing, where web transactions are run in headless mode without the overhead of a user interface.
API testing – Testing Application Program Interfaces (APIs) is a specific type of headless testing, where transactional data (usually in the form of JSON or XML) is validated either functionally or under load.
UI testing – This type of testing evaluates the “User Interface” (or UI) involves not only testing the web transaction, but also loads the browser rendering engine, images and other embedded files, and in some cases runs JavaScript code.
Request profiling – A testing technique for a large number of requests distributed over time. We cover the specifics of doing this testing technique in this blog post.
Other terms
JVM – This is the Java Virtual Machine, which is required to run many of our load tests including JMeter, Gatling, and Node.js custom tests. When you create and debug your load test locally, you will need to pay attention to the version of JVM you have installed, and ensure that it matches the version used by your test on RedLine13.
JDK – This refers to the Java Development Kit, which is the usual way that the JVM is installed on your local machine. It includes additional tools for developing test plans and applications using Java.
Did you know that RedLine13 offers a full-featured free trial? Sign up now and start load testing today!