Introduction
Nearly two decades ago, the idea of a REST API was conceptualised by Roy Fielding. The idea quickly became very popular. Compared to the existing methods, such as SOAP and RPC, which allowed users to programmatically interact with applications from across the internet, REST provided a pattern that's well-structured and easy-to-reason-about, and could be implemented to solve a wide variety of needs. Similar to object oriented programming (OOP) and relational databases, REST allowed programmers to think about a connected set of resources — similar to objects from OOP or database tables — which could be modified using a limited set of standard HTTP verbs, which could be thought of as a parallel to methods in OOP or CRUD (Create, Read, Update, Delete) operations in databases.
The resource oriented design of REST APIs is as popular as ever today, but there are limitations and points where it’s easy to trip up. In this post, we're going to look specifically at the idea of batch or bulk operations on a REST API, why they're usually necessary, and compare different ways to implement them.
It should be noted that some people, such as John Apostolidis, make a hard distinction between bulk processing (applying the same operation to multiple entries) and batch processing (applying potentially different operations to different entries). While it makes sense to have this distinction, in reality, the two ideas are often conflated and used interchangeably. "Batch" is often regarded as the more general term (processing batches of requests or batches of data), and "bulk" as a subset of batch (batching data, but not operations).
For this tutorial, you should already know at least the basics of REST API design as we won't explaining it in detail. We'll be focusing on why batch endpoints can be useful and different ways to add them to your existing REST API.
Looking at a toy REST API example
Let's imagine a very simple REST API which is a subset of Stripe's payment processing API. We'll consider only the /customers endpoint, which is used to retrieve existing customers or create new ones. The documentation lets us know that the following options are available.
POST /v1/customers
GET /v1/customers/:id
POST /v1/customers/:id
DELETE /v1/customers/:id
GET /v1/customers
We can immediately see one of the core advantages of a REST API. If you've used a REST API before, even without the Stripe-specific documentation, you could probably have guessed these.
To create a customer, we do a POST request to the /v1/customers and to retrieve customers, we use the same endpoint but use a GET request instead.
To retrieve, modify or delete an existing customer, we still use the /customers endpoint, but we add the :id of the specific customer we're interested in at the same time.
One detail that is slightly counter-intuitive is the thinking about singular and plural. The endpoint is called "/customers", and calling GET on it does indeed return many customers as an array of dictionaries (e.g. [{customer1...}, {customer2...}]). However, to create customers, we POST a single customer (e.g. {customer1...}. This is the kind of detail that you'll probably need to look up across different REST APIs, as it is not always implemented consistently.
Why would we need batch endpoints?
From the examples above, we can see an asymmetry when we are dealing with multiple customers at the same time. If we want to retrieve information from all of our customers at once, we can simply call GET /v1/customers and we'll have all the data we need. If we have an existing set of thousands of customers stored outside of Stripe, and we want to add each of them to Stripe, we'd need to create them one at a time, making one call to POST /v1/customers for each one.
Networking, or more specifically, the number of calls we need to make, is often the bottleneck in modern applications. Adoption of public clouds such as AWS has made it easy to scale up the processing power, RAM, or storage of our applications, but each networking call still needs to negotiate a complicated and unreliable global network of computers, routers, switches, and protocols, such as TCP, adding a lot of overhead for each call. Therefore, it's usually better to make fewer requests with more data (e.g. multiple customers) as opposed to making more requests with less data (e.g. one customer) in each request.
Imagine if we had created the simple customers API shown above and now we wanted to allow API users to create multiple customers at once. How would we go about doing this? We could simply modify POST /customers to accept an array of customers instead of a single customer. Doing this would match what GET /customers endpoint returns. However, it would be a breaking change to our API and be annoying for users who only want to add a single customer each time. They would always need to remember to add the customer or array before sending it over, and to process a returned array of created IDs. It is likely that, in a majority of cases, our users want to add only one user at a time.
Stripe doesn't implement a way to create multiple customers at once, but let's look at some other APIs that do have a way to batch requests, and see how they deal with it.
Looking at real-world examples of batch APIs
When you come across a design problem you're not sure how to solve, the best first step is often to look at how others have solved the problem. Luckily, there is no shortage of REST APIs with public documentation. Most APIs choose to either
implement an endpoint that can batch different requests into a single call, or a bulk version of some (or all) endpoints that can accept multiple resources in a single call.
We’ll start by looking at Google Drive, an example of the first option, before looking at ZenDesk, an example of the latter option.
Google Drive - batching requests
Google Drive is best known for being a cloud storage service, similar to Dropbox, but it also includes a powerful API to create, modify, and retrieve documents of various kinds.
Google has implemented a complicated but flexible batch endpoint. Instead of having an endpoint that accepts multiple resources, there's an endpoint that accepts multiple requests. These are essentially "meta" HTTP requests, where the main request contains different sub-requests.
The advantage of this approach is that it's very flexible. Google can accept different kinds of POST requests, each one containing different data, in a single networking call, and process them in parallel server-side, reducing the number of network calls that their API proxies need to handle.
The main disadvantage of this approach is that it's quite difficult to build up POST requests that look like this. It's easy enough for users to manipulate the data that they pass through with a request (e.g. to start using an array of customers if they already know how to pass through a single customer), but it's a lot more complicated for users to batch different API requests together and send them to a new endpoint. Even in the case where API wrappers are provided in a high level language, the users still need to think more about when it makes sense to batch different requests, the best way to combine them, and how the requests would interact with each other.
In Google's example, as copied below, we sent a batch POST request that contains two sub POST requests. Each of the inner POST requests creates permissions on a specific file: the first gives a specific email address to write a specific file, and the second gives permission to an entire domain to read a specific file.
For each embedded request, there is an --END_OF_PART marker. Note how there is some repetition, for example, the Authorization and Content-Type fields are repeated for each sub request, even though these are unlikely to be different. Once this larger multi-part batch request is delivered to Google, their servers will simply split it apart and process each section as if it had arrived individually.
POST https://www.googleapis.com/batch/drive/v3
Accept-Encoding: gzip
User-Agent: Google-HTTP-Java-Client/1.20.0 (gzip)
Content-Type: multipart/mixed; boundary=END_OF_PART
Content-Length: 963
--END_OF_PART
Content-Length: 337
Content-Type: application/http
content-id: 1
content-transfer-encoding: binary
POST https://www.googleapis.com/drive/v3/files/fileId/permissions?fields=id
Authorization: Bearer authorization_token
Content-Length: 70
Content-Type: application/json; charset=UTF-8
{
"emailAddress":"example@appsrocks.com",
"role":"writer",
"type":"user"
}
--END_OF_PART
Content-Length: 353
Content-Type: application/http
content-id: 2
content-transfer-encoding: binary
POST https://www.googleapis.com/drive/v3/files/fileId/permissions?fields=id&sendNotificationEmail=false
Authorization: Bearer authorization_token
Content-Length: 58
Content-Type: application/json; charset=UTF-8
{
"domain":"appsrocks.com",
"role":"reader",
"type":"domain"
}
--END_OF_PART--
ZenDesk - batching resources
ZenDesk is a customer support and ticketing system. They took a slightly different approach on implementing batch APIs. You can read their blog post about why they added batch endpoints and look at their API documentation to see exactly how the implementation works.
Similar to the Google Drive API we looked at before, ZenDesk has a different endpoint to handle batch requests, but unlike Google, there is one batch endpoint per resource type instead of a single batch endpoint for all resources. For example, you would use the following endpoint to create many Users:
POST /api/v2/users/create_or_update_many.json
You’ll see that users is still included in the endpoint above, as opposed to Google Drive, where we had to specify each endpoint we wanted in the data of the POST request's subsections. The example that they used to show how to pass through several users at once is as expected: we create an array of users, specifying the information for each one. Note that the example from the ZenDesk documentation assumes that you are using one of their API libraries, instead of creating the raw POST request, so it looks a bit neater than Google's example. The JSON data would still eventually be encoded into the body of the POST, and the Content-Length, Content-Type, and other headers would be added before sending.
'{"users": [{"name": "Roger Wilco", "email": "roge@example.org", "role": "agent"}, {"external_id": "account_54321", "name": "Woger Rilco", "email": "woge@example.org", "role": "admin"}]}'
The advantage of this approach is that it's simple to use. If users of our API know how to create a single resource and are having performance problems from needing to create multiple resources at once, they can easily modify their existing code to pass through an array, instead of a single resource.
The disadvantage is that it is less flexible than the generic batch endpoint. If we want to create multiple Users, Organisations, and Tickets at the same time, we would still need to make at least three network calls. Whereas with Google's design, we would only need to make one batch request by batching these different endpoints.
Finding other approaches
We only looked at two examples, but you’ll see the two patterns used by multiple companies. It’s easy enough to find more examples by searching the internet for “API Documentation” followed with a keyword of a large technology company. You’ll notice a wide range of quality in API documentation. Stripe, for example, is well known for investing substantial time and money into making sure that their API documentation is well designed, accurate, and easy to use. Older, more corporate companies, such as Salesforce and Oracle, generally have documentation that is less complete and more difficult to interpret.
It is interesting to see how different places implement bulk and batch requests slightly differently (if at all). It's also interesting that many of the bulk or batch endpoints that do exist are labelled as "experimental" or were added at a much later stage than the core endpoints. Clearly, batch and bulk processing is not something that fits naturally into the core design principles of REST or Resource Oriented APIs. That said, it’s a complicated enough topic that it’s worthwhile putting some thought into the different options before blindly adding endpoints to your API the moment you realise you need them.
Other considerations for batch processing
We looked at a few examples of batch API processing, and made a distinction between batch and bulk endpoints. Other than the issues we've mentioned, you'll also need to consider these when implementing batch or bulk endpoints:
- Error handling: What happens if a single request or resource in a batch request fails? Should the entire batch fail, or should it process as many requests as possible?
- Limiting of batch sizes: Many endpoints specify a rate limit of how many times each endpoint can be called within a specific time window. If you add batch endpoints, your users might send huge amounts of data for processing and it might be difficult to keep up.
It is clear from the examples above that batch and bulk processing is often added to REST APIs as an afterthought, when networking bottlenecks are discovered.
Whether you are just starting out with the design of your API, or you have identified the need for batch processing after scaling to real users, it’s good to understand the different ways batch processing in REST APIs can be implemented, and the advantages and disadvantages at play.
