You can now create an (Almost) free load test based on an Apache or NGINX WebServer log files.

Why? Customers asked for it!

A customer asked us for the following: extract from say 5 minutes of traffic on the current web server logs the list of the urls which are called and then try to replicate the exact list of calls on our infrastructure and see what happens.

Limitations

  • We support GET requests only.
  • POST data is not supported as it is not standard in your LOG files and typically bad practice.
  • Have your own question or feature request – email info @ redline13.com

How to use it.

Start Log Replay for Apache NGINX Load Test

  1. Specify a BASE url.  Your log file does not tell us the site URL.
  2. Upload your Access Log.  We do have size limitation for uploads. This should be a sample of your log file.
  3. Log File Pattern.  The ‘Log File Template’ just prepopulates the Log File Pattern with the default for Apache or NGINX.  The real key is making sure you get the correct pattern.
  4. TEST!  Described in detail further down, but this allows you to test your pattern against a sample.
  5. Setup the basics.  How many users to simulate (a user will basically emulate your log file), how many times to simulate the log per user, and how much delay between each request (this will randomize the wait time between the min and max).
  6. Setup the cloud options (like how many servers, size, which regions to run in, etc.).
  7. ‘Start Test’!  Launches your cloud load test.

Fields Description

Field  Description
Base URL The base URL to preprend to all requests found in the log file.
LOG files do not tell us the sites URL
Access Log The Apache or NGINX log file to parse.  We use all the GET requests to generate the traffic.
Log File Template This is not a required field, however selecting Apache or NGNIX will populate the pattern with the default for Apache or NGINX.
Log File Pattern The pattern to use to parse the request.  The patterns available are defined below and in the open source log file parser.
Number of Users The number of users to simulate in total across your cloud servers.  Each user will simulate the entire log file, at least once.  You can set iterations as well.
Iterations Per User The number of iterations per user to simulate the entire log file.
Ramp Up Time The time to ramp up the users, otherwise they all start at the same time.  We evenly ramp up the users based on the time specified per cloud server.
Minimum Delay For each request within the log file what is the minimum time to wait before making the http(s) request.
Maximum Delay For each request within the log file what is the maximum time to wait before making the http(s) request.
Together Min and Max define the random time to delay between each request
Save Response and Percentiles
  • Save Response Output : We capture the content from the request.
  • Percentiles : After test completes we calculate percentiles performance.

Testing a Log Format

We provide a tool to simulate parsing the log file based on the pattern provided.  This makes validating that your test will work a bit simpler.

Start Test Log File Pattern

Once the dialog launches

Test Apache and NGINX Log Patterns

  • Paste in sample of your log file.
  • Validate : We will validate the log file by confirming each line. A lined shaded red will indicate an error with a message of why the line could not be parsed.
  • Close : Close dialog
  • Save : If you alter the log file pattern, then Save will copy that pattern back into the main start test form.

Log Format

Default NGINX Format

  • %h %l %u %t “%r” %>s %O “%{Referer}i” “%{User-Agent}i” 
    • 192.168.99.1 - - [02/Apr/2016:23:38:56 +0000] "GET /favicon.ico HTTP/1.1" 404 209 "https://192.168.99.101/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2695.1 Safari/537.36"
  • %h : HOST
  • %l : Log name
  • %u : User
  • %t : Request time
  • “%r” : Request which is wrapped in quotes
  • %>s : Status, but it can read multiple status at this point.
  • %O : Sent bytes
  • “%{Referer}i” : %{…}i will be a header that was written to the log file, in this case the Referer
  • “%{User-Agent}i” : The user-agent header is expected in the log file.

Default Apache Format

  • %h %l %u %t “%r” %>s %b
  • %h : Host
  • %l : Log name
  • %u: user
  • %t: Request time
  • “%r” : Request which is wrapped in quotes
  • %>s : Status, but it can read multiple status at this point.
  • %b: Response bytes

Open Source

  • Besides the many standard open source projects we used a Log Parser written by Rafael Kassner
    • https://github.com/kassner/log-parser

 

 

Comments are closed.