Writing a load testing tool in Go

Motivation :

In one of my previous organizations we had a requirement of load testing Python and Scala based micro-services which interacted over RabbitMQ (using protocol AMQP 0.9 which is pretty different from AMQP 1.0)

We were already using Jmeter as our preferred load testing tool for Rest APIs. So I had few options to work with

1. Jmeter with some AMQP based plugin
2. RabbitMQ itself provides a crude way of loading queues via its PerfTest tool
3. Last option was to build something of my own

Jmeter and RabbitMQ tool could have served the purpose to some extent but they were more suitable for async messages. Some of our micro-services used rpc (Remote Procedure Call) functionality of RabbitMQ which is more of a synchronous communication between Consumer and Publisher over queues.

Now whether the usage of RPC over message queues was a correct design choice or not is a topic of discussion for some other time

Conclusion was to not use Jmeter plugin or RabbitMQ Perf Test tool but to build something of my own

I was learning Go around that time and was super impressed with Go’s concurrency model. I thought it was a good time and opportunity to convert my Go learning into implementing something useful so I decided to build load generator tool in Go.

BTW there is a nice load generator tool written in Go called Vegeta but it’s mainly for HTTP

Before starting to build the tool, I wanted to confirm few things

1. There was a RabbitMQ client library available in Go because RabbitMQ doesn’t provide one, fortunately there was one open-source client library
2. Monitoring RabbitMQ in terms of CPU, Memory, Disk IO of host that runs the RabbitMQ node and then the metrics of individual queues itself like total connections, total consumers, total messages etc. at given point of time. There are multiple ways to do this as documented here

Design:

Once the above things were confirmed, I started thinking about the bare minimum functionality this tool should have and I decided to follow Jmeter’s design for the same. I needed to control

1. No of Threads/Users
2. Ramp up time for this users
3. Execution time of test
4. Way to define samplers/tests

That’s it. If I could manage these 4 things my tool should be ready for use.

Code:

First thing was to accept user input for No of User, Ramp Up time and Execution time. This was pretty straight forward, I could set it with command line flags

Go encourages having shorter variable names. If anyone is more interested in knowing more, here is one presentation

Second thing was to calculate how much time should Go wait before it starts next user thread, which was also straightforward as I had the total no. of users to load and time in which all user threads should be up and running, so wait time in between initiation of two consecutive user threads was

So for example if my ramp up time was 30 secs and I had total of 10 user I would spin off a new user thread every 3 secs.

Now I wanted something which could increment the user thread count as per the ramp up calculations. Best way was to initiate a goroutine for that job.

Goroutine is a lightweight thread created by Go’s runtime when any named or anonymous function is invoked with a prefixed keyword go

Below goroutine pushes user count into the channel every time its incremented but won’t initiate the actual user thread.

Channel object in Go is a communication medium for goroutines to interact with each other by sending and receiving messages and is defined by keyword chan

Next thing was to get a time channel which would help in indicating the end of execution duration

After this the final bit was to use the two channels defined in above code for controlling the flow and executing the required tests
This piece of code brings together all the core features of Go which makes concurrency so easy

select is a special statement in Go which blocks the execution until any one of the channels in case statements is ready for send/receive operation

And that was all I needed !! I just added all required test calls in the above for loop and was able to bombard RabbitMQ message queues with hundreds of users that to with extremely low memory footprint on my load generator machine !

For monitoring and reporting I pushed all metrics into InfluxDB and used Grafana for real time visualization

Go is an extremely powerful language when it comes to concurrency. I would recommend every Go enthusiast to read the book Concurrency in Go written by Katherine Cox-Buday

You can find the full gist of the above code here