Troubleshooting Common Issues with Selenium Tests on RedLine13

Many RedLine13 customers have a need for full-browser testing. This is something we support with Selenium WebDriver. In this brief post we will cover common issues encountered with Selenium WebDriver tests and when scaling load testing on RedLine13.

Test Never Completes

Occasionally we encounter a Selenium WebDriver test that otherwise appears to run as expected, however never completes. One possible explanation is a simple one, where the driver.quit() statement is omitted.

const main = () => {
    const driver = redline.loadBrowser(BROWSER);
    try {
        ...
        driver.get(URL);
        ...
    }
    driver.quit();
}

To avoid this, always remember to add a call to driver.quit() before the end of your main() method.

Test Terminates without Running Script

Another issue that can be perplexing to debug is the case where a Selenium WebDriver test appears to complete and terminate before even running the desired test plan script. From the load test results page in RedLine13, the test will appear to complete after a few seconds without posting any meaningful results. One culprit to this is using the async/await model without any synchronous entry point defined. Consider the following script:

const async main = () => {
    const driver = redline.loadBrowser(BROWSER);
    try {
        ...
        driver.get(URL);
        ...
    }
    ...
}

Since the entry point on the main() method is async, execution will complete without waiting for the thread on which the test runs to complete. This is easily remedied by ensuring that the main() method is defined as a synchronous function:

const main = () => {
    const driver = redline.loadBrowser(BROWSER);
    try {
        ...
        driver.get(URL);
        ...
    }
    ...
}

Scripted Delays not Working

An issue that is not uncommon that we see our customers face is trouble defining a delay within their Selenium WebDriver scripts. To understand why a simple sleep() call with a predetermined millisecond delay does not have the intended effect, we must understand how the test executes. The following code example illustrates the problem:

const main = () => {
    const driver = redline.loadBrowser(BROWSER);
    try {
        ...
        sleep(1000);
        driver.get(URL);
        ...
    }
    ...
}

Selenium scripts work on the basis of constructing a queue that is then applied to a browser session once instantiated. Each instruction is added to the queue sequentially to be run when the get() method (or similar) is called. Adding an ordinary sleep() call simply causes a delay to be placed when building the script actions, and has no effect on the test when run. To correct the above code sample, we must use driver.sleep() instead. This will cause the sleep instruction to be run at the time when the script actions are performed, thereby executing the delay as a browser action:

const main = () => {
    const driver = redline.loadBrowser(BROWSER);
    try {
        ...
        driver.sleep(1000);
        driver.get(URL);
        ...
    }
    ...
}

High Resource Utilization at Beginning of Test

As discussed in a previous blog post, Selenium WebDriver tests demand high resource utilization. When scaling your Selenium test to generate a load against a target test application, your load generator instances can easily be stressed beyond their capabilities. In particular, CPU utilization tends to spike sharply at the beginning of the test when multiple browser instances are stated simultaneously. When CPU utilization approaches or even pegs at 100% it becomes problematic for performance testing purposes. We want to avoid conditions such as the following:

To remedy this, one easy approach is to spread out instantiating multiple browser instances over time. In contrast to when we want to programmatically invoke a delay within our test script as discussed above, this is a case for using an ordinary sleep() statement. In the interest of keeping in simple, we can program a random delay in milliseconds:

const main = () => {
    Random randomGenerator = new Random();
    sleep(randomGenerator.nextInt(30000));
    const driver = redline.loadBrowser(BROWSER);
    try {
        ...
        driver.get(URL);
        ...
    }
    ...
}

Note that this delay is structured to occur before the browser instance is created. When running multiple concurrent browser instances on the same load generator server, the above code example will spread out creating those instances randomly over a 30-second time frame.

JMeter as an Alternative to Selenium

Even though Selenium WebDriver most closely simulates the end user experience recreating it with high accuracy, it is resource intensive.  If we are more concerned with stress testing our target test application we can do this more efficiently with a scripted framework such as JMeter.  This is especially true when massively scaled load testing is indicated into the thousands or millions of virtual users and beyond.  In these cases, JMeter will almost always be the superior choice.

Conclusion

Though JMeter remains our most popular framework for our customers, there are cases that are best served using full-browser based testing.  We hope that your experience transitioning your Selenium WebDriver to RedLine13 is made easier by outlining these common scenarios for issues and solutions. If you are new to running WebDriver tests on our platform, be sure to also check out our post covering Selenium Basics for Load Testing.

---

Did you know that RedLine13 offers a full-featured free trial? Sign up now and move your Selenium WebDriver testing to the cloud today.