Now that the first week of 2017 is officially over, let’s take a look back at some of 2016’s biggest load testing blunders. The story is familiar: a site experiences heavy load, they forgot to do performance testing, and then their site goes down. Here are a few examples to show you exactly why you should load test as we look at the biggest load testing blunders of 2016.
8. The Hamilton Ticket Lottery
The popular Broadway musical Hamilton ran a campaign offering front-row seats for just $10, provided you won an online auction. Registration for the auction goes live, thousands hit the site at once, the website crashed, and people missed out on the lottery. It was so bad that the developers had to take the site completely offline to fix the issue and many were unable to register for the auction.
7. Beyoncé’s Super Bowl Ad & Ivy Park Website Issues
Beyoncé both performed and aired a commercial to millions during the Super Bow, announcing her now tour, immediately causing her official website to go down. Later in the year, Beyoncé’s clothing line, Ivy Park, was finally available in stores/online. The mad rush of prospective buyers caused participating retailer’s websites to crash.
6. Dwyane Wade & The Miami Heat Merch Site
The Miami Heat ran a special promotion where all Dwyane Wade branded merchandise was to be sold for $13 in celebration for his 13 seasons with the team. Cue massive increase in traffic, the website quickly crashes, purchases cannot be completed, and fans take to social media to complain.
5. The Oculus Rift Pre-Order Crash
People could finally pre-order the Oculus Rift, the highly anticipated VR device, after many years of waiting. Pre-orders went live on Jan 7th, and soon after, the pre-order page crashed. People started to complain on social media, and the CEO announced that their site was “experiencing extremely high load”.
4. U.K. Voter Registration Website
The U.K. was about to vote on one of their most important issues in 2016: whether or not they would withdraw from the EU. Naturally, many waited until the last minute to register to vote, and upon registering, the registration website went down. Turns out the reg. application was only hosted on one instance, in one data center.
3. Pokémon Go Launch
The beloved Pokémon franchise finally came to mobile devices via an augmented-reality game allowing you to explore the real world while capturing virtually overlaid Pokémon. Upon the games launch, Niantic’s (the devs) servers were absolutely destroyed by intense load (7.5 million U.S. downloads in first week). On day one, users experienced slow downloads, connectivity issue, crashes, only those that registered with a Gmail address could initially play, and a host of other issues that continued for the next two weeks or so.
2. The Multiple Times the Canadian Immigration Website Went Down
The polarizing political race in the United States caused many to be uncertain. When Donald Trump won Super Tuesday, the Canadian immigration website went down as Google searches for “How Can I Move to Canada” increased by a factor of 1,500%. Then when Trump was elected President, the immigration website again crashed.
1. The Macy’s Black Friday Crash
Macy’s did not prepare their websites and mobile apps for Black Friday, and as expected, disaster ensued. Arguably the biggest weekend for retailers, Macy’s either neglected to load test or didn’t properly scale out their test pans. Either way, their website and app crashed on Black Friday, causing them to loose who knows how much money while also angering shoppers, creating a social media onslaught.
Determine Max Number of Users
Before you begin your performance tests, it’s important to determine how many users you are expecting to hit your site/app. Consult your development team and have everyone agree upon a number. Keep this number in mind with what you’re testing, and adjust accordingly.
Allow Time For Testing
It’s important to examine your Project Plan and understand how much time you have for testing. Ask yourself “what kind of testing is the most important for a high-profile release?”. Tackle the most important projects first that require extensive testing, then move onto the smaller projects.
Be Prepared
If you’re running a major sale, don’t load test, and then you’re website crashes, you’re probably to blame. However, it’s the unexpected that we must be prepared for. Perhaps your site only sees 10,000 page views a month, but suddenly your last blog post went viral, and now some 600K+ people are accessing your site. It’s hard to prepare for these kind of events, which is all the more reason to load test just in case.
Reflect on Past Incidents
Has your website/app ever crashed before in the past? Did it slow down? Do you have any existing website metrics that could show behavior under load? If so, reflect upon these past incidents and determine what caused the issues in the first place. It’s important to learn from our mistakes.
Start Small & Establish Baseline
When you’re ready to begin testing, start out small. Begin by creating a simple load test with 100 users that will hit your website/app over and over again. Smaller tests like these are ideal for establishing baseline levels of performance.
Scale Out Your Tests
Once a baseline level of performance has been established with smaller load tests, you can now scale out your tests in a more realistic manner. Begin by growing your test (250 users, 500 users, 1,000 users, 5,000 users, 15,000 users, etc.) until you can pinpoint the highest level of load your website can take before crashing. Scale the tests back down again to be even more thorough.