From the Open Source Load Testing presentation http://www.slideshare.net/richardfriedman/open-source-load-testing
True story, but leaving out a few company specific details. One of my first entries into managing larger teams was a project which had 40-50 developers and the middleware needed to support thousands of transactions per second. The system was built with the work of a great team and the expertise of one amazing project manager, but we needed to start load testing.
Realize this was circa 2004, so pre-cloud. It was a large company and we had a testing environment, however, that was shared. Want to run tests? Get your test ready, get time scheduled, run your tests, and go back and analyze results. No matter when you run your load tests during your software development lifecycle, it is never one and done. You need to iterate over and over, so sharing a testing environment was a painful experience.
As a resourceful team and having been deep into the java community, we knew of two projects at the time Grinder, now called, ‘The Grinder” and JMeter. I don’t remember why, but we selected Grinder. We built the scripts to test our middleware API and we tested from a single machine. Next step was scaling the load test.
Since we had a team of 40+, we commandeered every desktop, distributed the load test script to everyone via email with instructions on setup and execution. When it was time to load test, everyone got to their command line (cmd – it was all Windows desktops) and started the java command to launch the test. As this project was ages ago, some of the details of how we aggregated the results or analyzed data escapes me, however, the process allowed us to iterate our load testing. .
- The Grinder. Open source load testing tool used to run the tests, deployed to each developers machine
- User. While the grinder tests ran a few people where examining the response time from their terminal
- Minions. To launch the grinder tests we had to communicate with everyone to get the test ready
- Servers. Our staging environment, which was similar, but not a mirror of production
When it was time to validate our results on the larger, more expensive environment, we still found some issues, but we reduced the cycles we needed to spend in a shared environment
