Custom Tests can be written in PHP or Node.js or Python.

Basics

  • There are two ways to submit a custom load test
    • Submit the Load Test File ( [ANY_NAME].py ) in the language specified
    • Submit a compressed file (.tar, .tar.gz, .tgz ) with the custom file specifically named
      • Python – custom_test.py
  • Custom Load Tests support CSV, Compressed, and Extra files
    • CSV files can be split across servers
    • Compressed files can be marked to be expanded
    • All files are placed in the root of the load test

A language specific load test is

  • ANY code you want to execute
  • With some redline13 API calls to report test execution and timing

Each Language supports a packaging mechanism to install more packages

  • Python – Not available yet, waiting for customer direction and feedback

A Test Harness is available for all languages, including examples

Python Test

Special Thanks to Artem Fliunt for porting the test harness fro PHP/Node to Python allowing us to incorporate into RedLine13 core.

To write a custom load test in Python, you need to create a single Python file that contains a class that extends class LoadTestingTest.

from load_testing_test import LoadTestingTest

class CustomTest(LoadTestingTest):

There are several reporting functions that are available for your use, to access these  a few import statements are required

# Used to import the required functions
import record_helpers

# Used to pass through time
import time

The constructor for CustomTest will be called with 2 parameters, a test number and a random string, both of which should be passed to the constructor for LoadTestingTest. An instance of this class will be started for each user specified in the “Number of User” form field.

def __init__(self, test_num, rand):
    """Constructor

    :param test_num: int Test number
    :param rand: string Random token for test
    """

    # Call parent constructor
    super(ExampleTest, self).__init__(test_num, rand)

An instance of this class will be started for each user specified in the “Number of User” form field.

You must override the start_test() method of LoadTestingTest. This function is called to start the load test.

def start_test(self):
    """ Start test """

Extending  LoadTestingTest Gives access to a few functions, including access to configuration data.

    # Example getting information from Config Dict
    config = self.get_ini_settings()
    iterations = 5
    if config.has_key( 'iterations' ):
         iterations = int(config.get( 'iterations', 1 ))

Here is an example running some custom code within the test and capturing the time for each iteration

    for x in range(1, iterations+1):
        start_time = time.time()
        # Here is your Code - do anything, measure anything
        rand_time = random.randint(2, 5)
        time.sleep(rand_time)
        diff = time.time() - start_time
        # Record your measurement and capture if it was success or fail
        record_helpers.record_load( "Iteration", start_time, diff)

Recording methods available

  • def record_load(name, ts, time, error=False, kb=0):
    – This records and aggregates the total load time for any activity marked as ‘name’, with ts being the UNIX timestamp when the request started and time being the time to complete the request. This function is used in the UI to report the per page average response time, but does not affect the overall response time that is calculated. The go_to_url and loadResources methods of LoadTestingSession calls this for you.

    • name – string representation of the end point, does not have to be a URL can be any string
    • ts – can be specified via time.time()
    • time – is an integer
    • error – boolean to record if your result is fail or success
    • kb – Size of response.
  • def record_error(error,ts): – This records an error that will be displayed in the UI.  If you are recording consider recording also with record_load to show the error count and time errors created.
    • Can be used to record general errors
    • error – must be a string
    • ts – can be specified via time.time()
  • def record_progress(test_num, percent): – Optionally records the progress (between 0 and 100) of a single test. This allows you to enable more accurate progress than the default, which is based only on the number completed vs. the test size.  Not a required API call.

If a test fails, you can raise an Exception and the test will be marked as a failure and recorded as an error for the user being simulated.

Tips

  • To simulate the iterations parameter of a simple test, use a loop around your code within start_test().
  • You can use the random string that is passed in as the second parameter to your CustomTest constructor to append to URLs. This can help you to easily identify load testing requests in your server logs.
  • If you need additional resources for your test, you can download a file (e.g. a .tgz) with the resources in the CustomTest constructor. You should include code to ensure that only one test per server downloads the files. If you know the number of tests per server, you can use the test number to decide which test should download the files. Alternatively, you can attempt to create a lock file, with the first process successfully creating the lock file doing the file download.

Examples

  • CustomLoadTest, custom_load_test.py
    • Simple URL which does echo
  • ExampleTest, example_test.py
    • Loops 100 times with random wait, each loop is a recorded as URL and overall a PAGE
  • FormLoadTest, form_load_test.py
    • Gets data from a webpage, finds a form, and submits that form with data
  • LoginLoadTest, login_load_test.py
    • Submits a login form (to use see code, need to add in data)
  • SimpleLoadTest, simple_load_test.py
    • Hits a URL, parses for other endpoints (JS, CSS) and times those as well