RedLine13 returns all of the test results as graphs after a user runs a load test. There are different sections on the results page. It is a critical process to interpret and analyze the load test results.

The load test results are organized like this.

Load Test Results

  • Summary
    • Details
    • Test Servers
    • Output Files
  • Metric Sections
    • Percentiles
    • Overview metrics
    • Request metrics
      • Request Per Second
      • Request Average Response Time
      • KB Per Request
    • Error metrics
      • Error Per Second
      • Error Average Response Time
    • Agent Metrics
      • Load Agent CPU Usage
      • Free Memory
      • Network Bytes Received & Transmitted
    • Error Table

Note: make each of these a link to section below

Summary

In this section you can see a quick summary report of your load test. It covers Percentile response times, test duration, network throughput and number of failed requests. it is a very efficient report for the ability to make a quick analysis of your load test.

Load Test Results

90%, 95%, 99% – If your test generated Percentile metrics this will be the average across all endpoints. The percentile representing the average response time for X% of requests being below the performance time given.

Test Time – The time it took to run the test until the test completed.

Failed/Threads – The number of failed threads vs total threads. A thread in JMeter is the same idea as a virtual user.

Errors/Requests – The count of requests which generated and error vs total requests. A request is each end point which generated a metric. A single Virtual User can generate many requests.

Agent CPU Usage – The average Load Agent CPU Usage, this is useful to determine if the load agent was stressed and a large CPU instance should be utilized.

Network Received – The bytes received from your requests. Every request to an API and Web page will generate data, this represents the total size of all data returned across every request.

Network Sent – To make requests to APIs and Web Pages data must be sent along. A simple page request might be small but a load test to determine performance of photo upload could be large. This is the total size of all data sent for making requests.

Details

In this section, you can see details of your test plan. It shows the starting and ending time of the test, your JMeter scripts and your testing artifacts. Also it shows a brief description about the load agent and other information you specified for your test. Load Test Results

Test Servers

In this section you can see details of the server for your load testing. it shows Server name , ip address, instance type, etc. The information provided is after the agent is started and information we receive back from AWS.

Server – The server name for RedLine13 internals.

Check In – If the server sent a ping back to RedLine13, this implies we can communicate with the load agent.

IP Address – The AWS EC2 Supplied IP Address

Status – Current status of the EC2 Instance. This helps to see when the instance was terminated.

Size/Max Price – This is from your selection of the load agent request

Test Plans Running – During a test if you refresh page you can see this data update to represent how many tests (Threads, Virtual Users) are running on this load agent.

Test Plans Completed – How many virtual users have fully completed.

Avg Ptest Plan Time – The average time specific to this load agent. This information is only for quick view, actual performance data is located in the response time graphs.

Download Size – How much data was downloaded by the load test on this server. This information is only for a quick view, you can see request graphs for download size over time.

Failed Pages – How many requests failed on this load agent.

Output Files

In order to obtain the output files you have to select this option before your test starts.

After enabling this you will be able to see and download the output files. After downloading the files you can import the test results into the Jmeter Dashboard Report and you can see the test results.

Metrics Section

This covers realtime and post-test results.

Percentiles

Percentiles for Average Response Time

This section is generated after the test completes. The average of all the response times of a whole transaction by multiple threads (users) is the Average Response Time. And also the time it took to process successful requests over the course of your test for the different percentiles. Sometimes it may be hard to understand and analyze the percentiles response time. Let’s look at the example below. We selected 90% percent and see that the overall response time is 37.41 seconds. This means that 90% of the HTTP requests were processed in 37.41 seconds or less.

Overview Metrics

JMeter Threads Per Second

This report simply shows the number of threads initiated by the time.

One of the two columns shown in the table shows the number of jmeter threads started(y axis), and the other shows instant clock data (x axis). Blue line meaning is the JMeter Threads started at graph.

When we select a point on the graph, for example in the screenshot below, this means at 7:18:27 pm 1 Jmeter Thread is started.

When we click on the “Toggle Cumulative “ button, the graph shows the cumulative started, completed and error thread by the time. Using this graph, we can easily understand when the run test reached its peak thread and understand when the completed the threads.

When we click on the “Users Running” button, it excludes the Completed and Errors thread and shows the only cumulative number of threads by the time.

Average JMeter Thread Elapsed Time

JMeter measures the elapsed time from just before sending the request to just after the last response has been received.this reports shows the overall elapsed duration over a time.

Request Table and Filter

This is perhaps one of the most important and efficient summary reports to look at after the test execution. It shows the average response time for all pages and also shows the total number of success and error count. You can inspect the specific page easily with this report. You can also easily search for the desired page using the search box.

Request Metrics

Request Per Second

Using this report, you can understand how many requests are sent at any moment. According to the below report we can understand that at 7:18:28:PM, 2 “updateQuote1” requests are sent.

When we click on the “Toggle average” button, we can see the number of average, max and sum of requests at that moment.

Also you can get the cumulative average, max and sum values if you click on the “Toggle Cumulative” button below.

Request Average Response Time

This graph shows the average response times from the pages. For our example we see that our highlighted request at 7:18:36 pm took 1.66 seconds response time. We can look for and analyze the response times of our request using this graph. However it might be complex if you want to search a specific page response time because this graph shows the response times of all of our requests. So it is better to look to Request Table and Filter Graph.

KB Per Request

This graph shows the downloaded data by time.

Error Metrics

Error Per Second

This shows the number of errors at a time.

Error Average Response Time

This shows the average response times of the errors received from requests by time. It could be a useful metric for resolving the error. Some errors may happen for setting lower timeout value. You can increase the timeout if applicable.

Agent Metrics

Load Agent CPU Usage

The most important thing to consider when planning a test to get the actual response time is determining the load agent. So we should make sure that our Load Agents are properly sized. In the below chart the load agent increases to 80% in a short period and later continues at 55%. This is a properly sized load agent.

Free Memory

In addition to paying attention to CPU usage in Load Agents, you should track the memory usage of the load agents in order to get more realistic results. With this report, you can see instant memory usage by a time .

Network Bytes Received & Transmitted

In some cases it may be necessary to check network traffic and throughput during tests. With this report you can see total bytes of data downloaded and uploaded from the server.

In many cases although the application is very well optimized downloaded files(css,images,js files..) from the server may be too large and this causes the bottleneck of the network. So using this graph is helpful to tracking the network throughput

Error Table

An efficient report that allows us to see the details of the errors and which pages were received during the test .