An old metaphor, intended to explain the concept of “infinity,” states that an infinite number of monkeys, banging away at an infinite number of keyboards, would write code just as well as we humans can, with better commenting.
No, wait, that’s not it. It’s that an infinite number of monkeys banging away for eternity on an infinite number of keyboards would at some point reproduce the entire works of Shakespeare. The idea is that a random process, with an infinitesimally small chance of producing a specific outcome, will succeed if left to run for an infinitely long time.
An infinite (or just a sufficiently large number) of monkeys could be useful in another way, and it wouldn’t take as long: Sit each one at a computer and let them load-test software!
The Importance of Load Testing
Load testing (sometimes called stress testing) is a type of software testing that is often overlooked but is quite important. It is especially useful in gauging the performance (speed, response time, page load time, throughput, latency, and other parameters) of client-server systems, in which multiple clients make demands on a server, such as a website. It’s also useful for evaluating the ability of a system to process large data sets—a critical consideration in this era of big data.
What makes load testing difficult? For client-server systems, you need to coordinate enough users to put a meaningful amount of stress on a system, and even if you do, the results won’t necessarily tell you where the bottlenecks are (client hardware, client software, network, server hardware, server software…).
And even in places where monkeys are plentiful (certain legislative bodies come to mind), it’s hard to train them and get them to sit still long enough to do meaningful load testing. What’s a QA lead to do?
The answer: Automated load-testing tools. These tools simulate multiple users making requests on the system under test. Two of the best-known examples are JMeter and Gatling.
How JMeter and Gatling Compare
JMeter and Gatling are intended to serve the same purpose, although they approach the task in fundamentally different ways:
- JMeter is an open-source tool that’s been around for over 20 years and has had numerous versions and enhancements in that time. Gatling is much newer, with its first release in 2011. It comes in two flavors: an open-source version freely available on GitHub, and a licensed “enterprise” version. Gatling is supported and distributed by the French company Gatling Corp.
- JMeter sports a graphical user interface (GUI), whereas the open-source version of Gatling is intended to be implemented in code (the enterprise version adds a web-based administration console).
- JMeter test configurations are stored as XML files, Gatling as code classes.
Each tool has its strengths and limitations:
- Gatling supports load testing using a small number of protocols, including HTTP, HTTPS, REST, and JMS, among a few others. JMeter supports a much larger number of protocols, including SOAP, JDBC, FTP, and TCP.
- Gatling is intended for use by developers; JMeter, with its GUI, can be used by anyone, thereby extending the reach of its user base.
- JMeter supports scaling, where multiple computers can be coordinated to increase the simulated users and thus the load on the system under test. Gatling does not support coordination among multiple computers.
- Both tools support the recording of user actions as an input to a simulation scenario. Similar to recording a macro in Microsoft Word, the tools record the user’s actions, and then automatically execute these actions on a larger scale to stress the system.
- JMeter enables running the tests from within the GUI or from the command line, whereas Gatling has only a command-line interface to run tests. Both tools generate data files with the results, which can be presented using third-party analysis tools.
An important consideration is how efficiently each tool uses resources on the computer(s) on which the testing tool is running. You can imagine that simulating hundreds or thousands of users on a single computer, all sending requests to a server at the same time, stresses the client computer quite a bit. Some comparison tests show that Gatling has a slight edge in this area, using less memory, disk space, and CPU than JMeter.
Load Testing and AndPlus
At AndPlus, we understand the importance of load testing and always incorporate it, when appropriate, in our clients’ software development projects. We also love automated tools, so we’re pretty excited about the availability of both JMeter and Gatling. We’d be happy to load-test your application, too. Call us today to get started.
