Capturing API Performance Data from Firefox or Chrome during Webdriver Test

In support of running selenium webdriver tests at scale, we built a simple wrapper for webdriver.

We injected javascript into PhantomJS to listen to onResourceRequested and onResourceReceived.  We could also track errors and timeouts.  You can see here how we tracked and summarized this data.  As well as emitting performance data to report back or write to Apache JMeter JTL Performance file.

Chrome’s performance logs will give fine grain details of all events, however, accessing the log was easier when the test completed.  This gave us full access to the log details.  Enabling performance logs via webdriver is simple, just set the capabilities.builder.getCapabilities().setLoggingPrefs({'performance': 'ALL'});   Read details of enabling Performance Log here.    Our code processes the log to summarize the API performance and convert to Apache JMeter JTL format.

Firefox does not have a built in mechanism to collect performance metrics.  You would need to write a plugin which monitored the streams.   We have a plugin to do that https://github.com/redline13/selenium-jmeter but it was written to be a recorder that worked with Selenium IDE plugin.

However, we had not looked at how to use Browser Performance APIs to collect metrics during a test.  We realize that by executing a script in the browser at any given point we can return back all the performance data and process for reporting in real-time or writing to Apache JMeter JTL.

Here is how we did it (source here)

RedLineWebDriver.driver.executeScript(`
  // PhantomJS does not support so don't execute if not available.
  if ( ! window.performance || !window.performance.timeOrigin )
    return null;

  var data = {
    // We need origin for timestamps
    start : Math.round(window.performance.timeOrigin),
    // The page we are on
    navigation: window.performance.getEntriesByType('navigation'),
    // All metrics available
    resources: window.performance.getEntriesByType('resource')
  };
  // Clear so next time we get new data
  window.performance.clearResourceTimings();
  return data;
`).then( function(metrics){ ... do things ... } );

We have further simplified this in our module

To record metrics at any time in Chrome or Chrome-Headless

// include redline
var redline = require( 'redline13-webdriver' );

// load your driver via redline13
// redline13 library uses standard mechanism but presets some capabilities
// browsers available - chrome, chrome-headless, firefox, firefox-headless, phantomjs
var browser = redline.loadBrowser('chrome');

// You can require webdriver yourself or use redline13.webdriver
var By = redline.webdriver.By;
var until = redline.webdriver.until;

// Start your script
browser.get( "http://example.com" );
// Do your normal operations

// Record what is currently available
browser.recordrecordMetrics( "example" );

Firefox and Firefox headless support

Since firefox had no metrics support we manipulate webdriver .get to automatically collect performance data.  The call to browser.recordMetrics can still be used to collect metrics at other points of your script.  See how we override the function at the source of firefox-headless.js.

There are limitations to using this approach

These limitations prohibit collecting a full picture into missing or cached data.  However, we can get accurate performance data for the API calls being made without having to build more tests.  Huge win!  We do recommend still using PhantomJS or Chrome Headless for a complete performance analysis.

You can run this locally or add into all of your Selenium tests to capture API performance metrics.