Practical Advice for Fast Growth Companies and Load Testing

When companies are growing fast, their main concern is not scaling. Until it’s an issue.  This is part of maturing devops (development operations). They need, or will need, load testing. Then the questions come up. This blog post should be seen as practical load testing advice, along with the right questions, for any company that is grappling with the need to load test.

Load testing is based on user requirements. For example, load testing is determining if your systems can handle X throughput and maintain performance at Y ms for Z% percentile.

The process in its simplest form: 1. finding bottlenecks, 2. fix bottlenecks, 3. test again.

This is not something you repeat with every code push.  This is something you might do a few hundred times when load testing to see if you can support 50,000 registrations in minutes, or doubling the traffic on a streaming video service when merging user bases, or something else specific to your business. (Note: these are real scenarios for RedLine13 customers.) Each case can require running hundreds of load tests to work through the bottlenecks. These are typical RedLine13 customers. Why? Because the cost to do each incremental test with RedLine13 is (almost) free.

Should you do this kind of load testing day one?  No, you should be realistic about your requirements. In the first case, companies did not test for 50,000 registrations when they launched, but as the company grew they realized that they were serving bigger and bigger customers.  In the case of streaming, the company already had millions of users when they realized the need.

The question is whether this change will somehow impact the performance of your baseline.  This is the process that could run daily or with every push.  It could run against your dev, staging or production environment.  We have seen this in the real world. A customer has SLA guarantees to their customers, either internal or external.  They run small performance tests daily or hourly and trigger an alert if out of band by X %.

We have a RedLine13 customer that has run 1,391 tests as of today and has been using RedLine13 since April 2015 – so over the past 1,060 days they sometimes runs extra tests but typically run tests daily.

Should you do baseline testing day one? Here are the questions that you need to answer. If you have an SLA – then yes.  If you do not have an SLA – then is it a requirement?  Are you already measuring performance on production?  Are you ok with launching and waiting for peak to see if your code broke something?

Hopefully this is common sense load testing advice, and the right questions, which you can apply to your situation.

We look forward to your feedback.