Case Study: iCIMS, The Talent Acquisition Software Experts

iCIMS Case Study

Introduction

Here at RedLine13 we always look forward to hearing from our clients about how they specifically use our services. This not only provides other users with insights into how people incorporate and use RedLine13 into their workflow, but it also allows us to improve upon what we do as a company. In an effort to accomplish these goals, we have spoken with some of our clients that regularly use our service. We touched base with one of these clients, iCIMS, about how they incorporate load testing into their work/performance cycle and found some valuable information.

iCIMS: The Talent Acquisition Software Experts

Located in Matawan, New Jersey and with offices across North America and the United Kingdom, iCIMS is a software company that provides a cloud-based software suite which allows businesses to improve their employment process; everything from finding and screening new employees to inevitably hiring them. Their Talent Acquisition Software Suite is an in-depth and feature-rich solution that allows various businesses and organizations to both monitor and manage the lifecycle of a potential candidates application process. They are the second largest ATS provider in the United States with over 3,200 contracted customers and are growing at an exponential rate [1. iCIMS Company Overview]. Some of iCIMS’s more notable clients include Sirius Satellite Radio, Dole, Sony Music, Capcom, New Balance, Banner Health, GNC, and L.L. Bean [2. iCIMS Customer List]. Recently we spoke with Mahadevan Subramani, one of the QA managers at iCIMS, and asked him a range of questions with relations to load testing and cloud-based solutions.

Our communications with iCIMS has provided us with a deeper insight into their workflow and their processes when it comes to testing, why they run these tests, the various tools/integrations that they use, and the value that load testing provides them. We wanted to share some of this information with the community so they can also see how others are using and incorporating load testing.

Challenges: Manage Growth; Successfully Implement New Technologies

Not only does iCIMS have a large and ever-expanding user base, they also implement many new features and updates on a regular basis. According to them, they are growing so fast that they add new customers “every two business hours” [3. iCIMS Customer List]. Naturally when you are receiving new customers that quickly, and on a consistent basis, it can be difficult to manage your growth. iCIMS has to be able to accommodate these new clients and be able to give the same amount of attention and focus to their existing user base simultaneously. This obviously can be a challenge when your users are both small businesses and large corporations alike.

Additionally, iCIMS needs to be able to implement new technologies and features seamlessly into their existing software suite. iCIMS is constantly working towards adding new features, improving old ones, and migrating to new technologies. The ongoing need to successfully implement new technologies is another challenge facing iCIMS: they want to ensure they are able to perform for their users without hindering growth.

Their Testing Process 

As is the case with many companies, iCIMS has a very specific and exact process when it comes to load and performance testing. Before any testing can be done, they first need to create a new feature, program, or line of code. This first starts with the developers who have been tasked with a project. Once the developers have completed the project, they are then responsible for taking charge of the testing. Unlike other companies that rely on QA for this part of the process, iCIMS depends on their developers for performance testing.

From start to finish, the standard performance cycle and workflow process for a project takes a few weeks for iCIMS to complete. In terms of testing, iCIMS begins with running small tests on the code level, like profiling. Once that is completed and everything seemingly checks out, they then tackle a full performance sprint cycle. They first write these tests with Apache JMeter and then run these tests in-house, on smaller scales. After that step, iCIMS loads up the test(s) with Neustar, a tool they use for running the core cases. They will then use a cloud-based load testing solution to run the same tests, only on a much larger scale. Once they run their scalable tests through a load testing service, iCIMS then collects a detailed breakdown of all the data. After all of these steps have been completed, iCIMS is then free to safely implement the subject of their testing.

Solution: Use Cloud-Based Testing Solutions and Other Tools

iCIMS uses a wide range of different tools, integrations, DIY platforms, and other services to meet their challenges. For iCIMS, arguably the most valuable one is JMeter. iCIMS has just started writing and creating tests with JMeter and they’ve quickly come to prefer it over other tools for a few reasons, as Mahadevan told us. First and foremost, JMeter is an open source tool. Due to this fact JMeter is not only free to use, it is also routinely supported by a passionate and tight-knit community of users and developers. Additionally, JMeter is easy to use, it offers many listeners, it provides iCIMS with the most amount of practical features, many developers are creating various plugins for it, and most other major companies that do testing also use JMeter.

Unsurprisingly, that is not the only tool or integration that they use. iCIMS runs core cases continuously in Neustar and will then monitor the performance with New Relic, in addition to incorporating Zabbix and Java Mission Control. iCIMS incorporates other monitoring tools into their production environment and are looking to utilize tools like Splunk and Graphite/Grafana, among others, for their performance testing in the future. Another RedLine13 customer has similarly used Grafana to visualize their test metrics.

iCIMS also uses profiling tools like JProfiler. Once they have incorporated and used all of these tools/integrations, iCIMS then uploads all of this data to a load testing service (RedLine13) where they then do the heavy-lifting when in comes to testing, in particular, scalable testing. For example, iCIMS recently tested their social-distro product using RedLine13 within their own dedicated performance environment. The use of cloud-based solutions with the combination of the various tools that they use allows iCIMS to adequately meet their goals, ensuring that everything runs smoothly no matter what the circumstances are.

Value Added

Given what iCIMS does, load and performance testing is invaluable for them. iCIMS incorporates load and performance tests into their various performance sprints, running the tests as needed. Since iCIMS offers a feature-rich and expansive software suite (one that is constantly receiving new features/updates), they need to ensure that every bit of it works, down to the code level. Any time iCIMS needs to add a new feature or make changes to some existing code, they run scalable load and performance tests. Since this is something that they do every few weeks, they are almost constantly performing these tests.

iCIMS utilizes RedLine13 as their cloud-based testing solution to face the challenges that have been outlined above. The company has recently experienced high levels of growth as they have been bringing in many new clients (including many major, global corporations) and the testing solutions RedLine13 offers allows them to scale accordingly and to meet the needs of these clients. iCIMS wants to ensure that performance on their end does not hinder growth, which is why they run load, stress, and performance tests.

As iCIMS continually moves towards using new technologies and implementing new features, they need to constantly run tests to ensure that this is a smooth process. When testing is as important as it is to iCIMS, having a service that offers unlimited amounts of simulated users and the ability to run unlimited tests is important, for example. iCIMS is then able to meet the needs of their users, scale accordingly, safely implement new technologies, and then bring them to market faster thanks to load and performance testing.

Conclusion

Whether iCIMS is launching a new feature, altering some code, or incorporating new technologies into their existing solutions, load and performance testing is almost always a crucial and important part of the process. Apart from other tools, iCIMS relies on RedLine13 to be able to consistently deliver their cloud-based testing solutions as iCIMS runs load and performance tests on a frequent basis. Their developers first write their tests with JMeter and then run these small tests in-house. When iCIMS wants to run more tests, scalable tests, or soak tests, they will then use RedLine13 for their needs. iCIMS will upload these tests to RedLine13 all the while incorporating various tools, plugins, and integrations along the way. This inside look into iCIMS’s testing process gives us a better understanding of just how companies out there are using load testing to their advantage, which in turn, helps us improve upon what we do.