We are very excited to launch JMeter + Selenium WebDriverbig-logo Plugin support (in Beta) on AWS. This little update makes it seamless to launch web driver tests at scale.  Create real user tests and generate images of the results as you would a normal Selenium tests.

Tested  on our Linux (Ubuntu) images

  • Chrome
  • Firefox

Key things to know 

  • We do not provide support for Windows/IE support
    • If folks find this feature useful we will continue to spend time providing further support.
  • output/ – if you want to retrieve screenshots or other files written out during the test, they must be written to output/[FILENAME]
    • if we find .png, .jpg, .gif files we will extract and show at the end of the test run
    • Pro feature
    • Screen Shot 2015-11-10 at 6.39.56 PM
  • Chrome – we override the “Path to chromedriver” provided in the JMX file.
  • You are limited by the AWS image in terms of capabilities of the size of your test.
  • You must ‘check’ support for WebDriver in the plugins configuration for the test.
  • Screen Shot 2015-11-10 at 6.28.33 PM
  • You can modify the ‘Screen’ parameters, remember they will impact the performance of the machine.

Scaling

We have been asking ourselves how big can we scale a WebDriver JMeter test and what will it cost?   Sorry we don’t have the answer but we continue to investigate.  As in all load tests the performance of agents is also dependent on the details of the load test.  To give a sample here is the start of what can be accomplished.

Screen Shot 2015-11-10 at 6.26.13 PM

Simulated 100 real users on 2 servers (m4.10xlarge) which were requested as spot instances.  I had run numerous tests, looks like I used it for 7 hours!

  • 2.74 / 11 hours = .25c per hour.
  • .25c / hour to generate 50 concurrent ‘real users’
  • m4.10x large is a 40 cpu machine.
    • Considering real-user tests are not typically hammering like a load test this could be achievable.
  • Screen Shot 2015-11-10 at 6.36.34 PM

 

Test results were very promising, with CPU toping out around 20% used.  Once again this was a very light test and some test-cases completed as others where running.

Screen Shot 2015-11-10 at 6.43.12 PM

If these numbers hold up, get better, or come close we are talking about realistic mechanism to cost effectively to get to a 10,000 real user test.

Example Tests

Summary

Looking forward to RedLine13 users taking this new feature for a test drive and pushing it to the limits.

Comments are closed.