Selenium Load Testing: When and How

People often ask if they can use Selenium for load testing. The Selenium website says:

Performance testing using Selenium and WebDriver is generally not advised. Not because it is incapable, but because it is not optimised for the job and you are unlikely to get good results.

The site goes on to recommend JMeter. We would say that Selenium can be used for load testing with limitations as many RedLine13 customers do.

RedLine13 is a cloud-based platform extending an easy-to-use load testing infrastructure.  Users of RedLine13 routinely run large scale scripted tests numbering in the tens and even hundreds of millions of requests.  However, browser based testing presents several unique challenges, the most significant of which is a high resource overhead requirement.

Here is how we ran a 500 virtual user Selenium WebDriver test on RedLine13.  In this particular case, the target test application was taken to the failure point.

Selenium Load Testing Test Script

We created a simple test script for this demonstration.  Your test script can be as simple or as complicated as necessary.  The concepts used in this demonstration should carry regardless of this. Here is the script that was used:

const redline = require('redline13-webdriver');
const By = redline.webdriver.By;
const until = redline.webdriver.until;
const Condition = redline.webdriver.Condition;
const BROWSER = 'chrome';
const URL_1 = 'https://demoqa.com/';
const URL_2 = 'https://demoqa.com/webtables';
const DELAYED_CAPTURE = 14500;
const DELAYED_INTERATION = 20000;
const ITERATIONS = 5;
const main = () => {
  const driver = redline.loadBrowser(BROWSER);  
  for(var i=0; i < ITERATIONS; i++){
     driver.get(URL_1).then(_ => {
     driver.manage().timeouts().implicitlyWait(DELAYED_CAPTURE);
     redline.snap('capture1.png');
     setTimeout(function timedDelay() {console.log("...");}, DELAYED_INTERATION);
     driver.get(URL_2).then(_ => {
        driver.manage().timeouts().implicitlyWait(DELAYED_CAPTURE);
        redline.snap('capture2.png');      
     });  
     driver.manage().timeouts().implicitlyWait(DELAYED_INTERATION);
     });
  }
  driver.quit();
};
main();

Selenium Load Testing Test Plan

Initially we started with a single-user test.  As we ran through several iterations of this test we made small changes until arriving at the behavior we wanted.  Then to scale the test up to my target number of 500 virtual users, we simply increased the user count in RedLine13 and ran it on both more powerful and a larger number of servers.  Scaling any load test on RedLine13 is as easy as a few clicks to generate tens or even thousands of times more load.

Below is the configuration for this test.  Custom tests on RedLine13 can be run in a number of different languages, but here we have chosen Node.js.  Therefore we must set the Load Agent Language. We also need to select “WebDriver Node.js” to enable Selenium WebDriver support from Node.js.  For this test we have selected “Chrome” as the browser. Since our target test application resides on “demoqa.com”, we have specified this in the “Domains” field and ticked “Enable Hard Filter”.  This prevents web page dependencies from being considered in the results.

When preparing to scale our test, the clone feature of RedLine13 is particularly useful.  We can start with a small proof-of-concept test and increase the capacity of our test only when we know that it works as intended.  For instance with this test, we we able to do our proof-of-concept runs using only 10 virtual users, and once satisfied with the execution, quickly scaled to 500 users simply by changing the number of and size of the servers used.

Selenium Load Testing Results

Due to the fact we are simulating the entire client experience with Selenium WebDriver tests, they tend to be resource-intensive.  Unlike scripted tests using JMeter, tests based on WebDriver need to run a full web browser for each simulated virtual user.

One of the most compelling advantages of running a Selenium WebDriver load test on RedLine13 is the ability to view detailed aggregated charts across all load agents. In the above example, we can see graphed percentiles for response times.  Similarly in the graph below, we can view plotted request counts per second that each resource on our target pages produces.

With output files enabled, your Selenium WebDriver test has the option to capture the rendered browser pages to obtain visual confirmation of your target test application.  This is useful to see how the various elements of your web application are rendered as load is scaled up.  For our example, we can see that despite stressing our application to the point of failure, the pages continued to render appropriately.

Conclusion

It is possible to run load tests with Selenium WebDriver tests on RedLine13 with limitations. Under those limitations, it is actually quite easy.  The ability to scale your test is constrained only by the number of AWS resources that you wish to provision. As we have demonstrated, you can design your test with a small-scale, cost effective proof-of-concept and effortlessly scale that test up to achieve any practical number of virtual users.  Bearing in mind that Selenium WebDriver tests are more resource-intensive than scripted tests, the size of your test is dependent only on the quantity of AWS resources you are willing to provision.

---

Did you know that RedLine13 offers a full-featured free trial?  Sign up now and run your own large-scale Selenium WebDriver tests today!