How to use RedLine13’s (almost) Free Load Testing with RUM methodology:

Real User Monitoring (RUM) references a methodology that records every interaction of real users with a web site or web application. The capacity to monitor the way a website interacts in tests with real users is important to the development of online content so operators can see where the fault lines are and what remediation is necessary to make the software operate properly when it is really deployed.

RUM using real human users:

The original model for the use of RUM is to engage human subjects for usability tests. The RUM software is embedded as small JavaScript executables in the application or web page to monitor how the subjects use them. Often, the tests are treated like surveys and the human subjects provide details about how they use the applications and some details about their personal backgrounds so that differences in the way the applications perform for different subject groups can be assessed.

Limitations of RUM in real world applications:

RUM testing does have some important limitations, that could make the method impractical for many applications.

  • The use of a large human subject pool can be prohibitively expensive.
  • Since differences in regional internet systems can affect website performance in unpredictable ways, monitoring the way web applications perform in different national, regional, and linguistic environments is vital in today’s online marketplace.
  • The administration of RUM testing makes global testing impractical.
  • The introduction of the small JavaScript elements may itself distort the measurements by slowing the application slightly.

Synthetic Performance Monitoring, sometimes called proactive monitoring, which approximates real user testing, involves having artificial external agents placed in the cloud around the world. These external agents run scripts on the web applications. The scripts simulate the various ways different real users are expected to use the applications.

Simulations of Real Users:

Unlike RUM, the synthetic systems do not track the details of each keystroke. Since the scripts execute a known set of steps at regular intervals from known locations, performance should be uniform and predictable. Synthetic Performance Monitoring is based on a quality control model, looking for imperfections or large variations from the norm that can be seen as trouble spots.

Many companies treat simulation testing as a quality control step before the production is completed. They perform integration tests using Selenium to automate the human simulation browsers. The computers that simulate real users are themselves simulated in the cloud at specified global locations around the world.

  • Selenium WebDriver creates browser-based automations and tests. The software scales and distributes scripts across many environments.
  • Selenium IDE can create bugs to test the effects. This software creates scripts to be used in automated exploratory testing.
  • Selenium runs within browsers in many operating systems. Controls can be issued in a wide range of programming languages and testing frameworks.

Simulation on a global cloud make massive real user testing simple, inexpensive and practical. Using available software, 5,000 user loads tests can be run for $10 per hour. The pre-programmed Selenium web driver is distributed through the cloud to a range of locations to simulate real users from real physical locations. These simulated users can challenge any web application to test its performance and report variation from normal performance.

Selenium systems:

Redline13’s (almost) Free Load Testing is a delivery system, fully integratable with Selenium that includes the programming of the Selenium simulation and the distribution of the simulation across the world. It includes systems to capture and monitor the results of the tests in real-time using Apache jMeter running in Firefox. The complete system of monitoring will enable the developer to find areas of substandard performance which could then be remedied before final production.

Redline13’s (almost) Free Load Testing is the perfect tool to implement RUM methodology while testing automated user-paths.  Due to the prohibitive costs of large-scale RUM testing, once RUM methodology has been employed to verify transaction path function via real users, proof of concept can be established and any changes necessary can be implemented.  Once having completed these steps in due diligence, RedLine13’s (almost) Free Load Testing can be used to scale tests to simulate any user-base, from anywhere in the world.  Following these steps in the development of web applications and cloud based services can prove invaluable prior to the load testing stage.