Building a Concurrent Promise Queue with JavaScript

Spent hours looking for a cheesy quote to start with🧐, couldn't find one that went with promises😫.

Well, let's hop in🚶🏿‍♂️.

Promises!!!🚦 If you work with JavaScript, you will encounter them every day. If you work on a large product, you would have come across the problem of having to serve multiple requests to a third-party API. As a Backend developer working in the fintech space, one recurring issue I keep encountering is, serving and processing customer's bulk requests. Most people want to be able to upload a CSV file of a 1000 account (debit/credit) requests and have them executed promptly.

Although there are more optimal methods to solving this kind of problems, JavaScript non-blocking Asynchronous pattern provides a fairly naive approach that helps you process multiple I/O tasks in a manageable manner that doesn't overwhelm the event loop.
Let's do a quick run-through on the basics of Promises.

What is a Promise?

Simply put, a Promise is an object that can be used to represent the eventual completion of an asynchronous process. Not my quote and if this doesn't explain Promises to like you are a kid, this might though🙃. The official doc does a good job to explain promises.

Resolving Promises

const transactionStatus = (seconds) => new Promise((resolve, reject) => {
  setTimeout(() => {
    	resolve('Your transaction is successful');
    }, seconds * 500); //second argument is in milliseconds
})

Invoking our promise

transactionStatus(2)
  .then((result) => console.log(result));

result: Your transaction is successful

PS: There is a nifty little trick we can do above. Our .then() block takes in a function that takes an argument and logs it. Since console.log does the same thing, I can replace the entire function in the .then() method with console.log

transactionStatus(2)
  .then(console.log);

Result will be the same

result: Your transaction is successful

Promises are chainable. Invoking our promise above can include multiple .then() methods.

transactionStatus(2)
  .then(console.log)
  .then(() => 200)
  .then((status) => console.log('Transaction fetched', status));

result: Your transaction is successful
result: Transaction fetched 200

Rejecting Promises

If something goes wrong within the promise, we can chain a .catch() method to handle it (errors that occur within our codes).
We can also pass an error to the .catch() method if we evoke the reject() method.

const transactionStatus = (seconds) => new Promise((resolve, reject) => {
    if(seconds > 25){
        reject(new Error('Request timed out'));
    }
    setTimeout(() => {
        resolve('Your transaction is successful');
    }, seconds * 1000); 
 });
 
transactionStatus(26)
    .then(console.log)
    .then((status) => console.log('Transaction fetched', status))
    .catch((error) => {
         console.log(error.message);
    });

result: Request timed out

The ability to chain multiple .then() offer us the opportunity to execute Promises sequentially.

const fetchTransactionStatus = () => Promise.resolve()
  .then(() => console.log('Begin request')
  .then(() => transactionStatus(5))
  .then(() => console.log)
  .then(() => console.log('End Request'))

Each of the .then() executes sequentially and produce the following result.

result: Begin request
result: Your transaction is successful
result: End Request

Parallel Execution

Executing multiple tasks at the same time is referred to as parallel execution.
JavaScript provides the Promise.all() method that enables us to implement parallel promise execution.

Promise.all([
  transactionStatus(15),
    transactionStatus(5),
    transactionStatus(20)
]).then(() => console.log('All transaction requests resolved'))

result: All transaction requests resolved //prints after 20seconds

The code above will wait for all three promises to resolve before logging the message in the .then() block.That means it will take 20seconds to completely resolve.

If we want our .then() block to log our message after the first promise with less time is fulfilled, wrap the promises in a Promise.race().

Promise.race([
  transactionStatus(15),
    transactionStatus(5),
    transactionStatus(20)
]).then(() => console.log('transaction requests resolved'))

result: transaction requests resolved (logs after 5seconds)

Concurrent Promise Queue

Handling multiple asynchronous tasks is something NodeJS has the mechanism to handle well. Although there are better ways to handling large aysnchronous tasks such as implementing a proper queuing system with RabbitMQ, Amazon SQS, custom SQL Queue etc, a naive approach can be implemented with a promise queue.

I have talked about executing promises both sequentially and in parallel. Running multiple asynchronous tasks sequentially will take longer and if you run them in parallel, it will consume more resources. Yet, we can combine both methods to create a promise queue that will be faster and consume fewer resources.

Step 1: Create an array of promises.

const tasks = [
  transactionStatus(5),
  transactionStatus(15),
  transactionStatus(2),
  transactionStatus(4),
  transactionStatus(6),
  transactionStatus(9),
  transactionStatus(11),
  transactionStatus(12),
  transactionStatus(10)
];

Step 2: Create a constructor function that takes in two parameters, A tasks array and a concurrentCount integer to set the maximum number of promises that should be in the running state.
We also create an array that holds the promises currently running and another array that holds the promises that are completed.

Creating our promise queue constructor function

function PromiseQueue(tasks = [], concurrentCount = 1) {
 this.total = tasks.length;
 this.todo = tasks;
 this.running = [];
 this.complete = [];
 this.count = concurrentCount;
}

Step 3: Create a method that tracks the state of the tasks queue. This will determine if the runner should keep executing or quit. It also ensures we don't have more than the specified concurrent count in the task runner.

PromiseQueue.prototype.runNext = function(){
  return ((this.running.length < this.count) && this.todo.length);
}

Step 4 : Create a task runner. We would move tasks from the todo list to the running list, after execution, we move them to the completed list.

PromiseQueue.prototype.run = function () {
  while (this.runNext()) {
    const promise = this.todo.shift();
    promise.then(() => {
      this.complete.push(this.running.shift());
      this.run();
    });
    this.running.push(promise);
  }

}

Step 5 : Go ahead and instantiate our Promise queue function. we can call the .run() method to run our task queue.

const taskQueue = new PromiseQueue(tasks, 3); //tasks = an array of aysnc functions, 3 = number of tasks to run in parallel
taskQueue.run();

Conclusion

One reason why this naive approach is not scalable is, it becomes very complex to track failures and initiate retries.
What about if requests grow to 20, 000 tasks, it becomes extremely unmanageable. That is why a more robust queuing system that makes queue management easier to implement is the preferred approach.
Nonetheless, learning how to create concurrent promise queue gives us a better understanding of how JavaScript promises work.