Architecture foresight: Put a queue on that

If you build any kind of non trivial system, one of the absolutely best things that you can do for the long term health of your system is to move all significant processing to sit behind a queue.  That is one of those things that is going to pay massive dividends down the line as your system grows. Basically, any time that you want to do something that isn’t “pull data and show it to the user” or “store the data immediately”, throwing a queue at the problem is going to make things easier in the long run.

Yes, this is a bold statement, and I’m sure that you can see that this may be abused. Nevertheless, if you’ll follow this one rule, even if you misuse it, you are likely going to be better off than if you don’t. I should probably explain.

When I’m talking about using a queue, I’m talking about moving actual processing of an operation from the request handler (controller / action / web sever ) to popping a message from a queue and processing that. The actual queue implementation (SQS, Kafka, MSMQ, ring buffer) doesn’t actually matter. It also doesn’t matter if you are writing to the queue in the same process and machine or a distributed system. What matter is that you can created a break in the system between three very important aspects of command processing:

1. Accepting a request.
2. Processing the request.
3. Sending the result of the request back.

A system without a queue will do all of that inline, in this manner:

What is the problem here? If the processing of the request is complex or takes some time, you have an inherent clock here. At some point, the client is going to timeout on the request, which lead to things like this:

On the other hand, if you put a queue in the middle, this looks like this:

Note that there is a separation in the processing of the request and sending the accepted answer to the customer.

What is the impact of this change?

Well, it is a bit more complex to manage in the user interface. Instead of getting the response for the request immediately, we have to fetch it in a separate operation. I’m typically really strict on policing the number of remote calls, why am I advocating for an architectural pattern that requires more remote calls?

The answer is that we build, from the first step, the ability of the system to delay processing. The user interface no longer attempts to pretend that the system reacts instantly, and have far more freedom to change what we do behind the scenes.

Just putting the operation on a queue gives us the ability to shift the processing, which means that we can:

• Maintain speedy responses and responsive system to the users.
• Can easily bridge spikes in the system by having the queue flatten them.
• Scale up the processing of the operations without needing to do anything in the front end.
• Go from a local to a distribute mechanism without changing the overall architecture (that holds even if you previously held the queue in memory and processed that with a separate thread).
• Monitor the size of the queue to get a really good indication about where we are at in terms of load.
• Gain the ability to push updates to the backend seamlessly.

At more advanced levels:

• Can copy message to an audit log, which gives great debugging abilities.
• Can retry messages that failed.

There are whole patterns of message based operations that are available to you, but the things that I list above are what you get almost for free. The reason that I think you should do that upfront is that your entire system would already be used to that. Your UI (and users’ expectations) would already be set to handle potential delays. That gives you a far better system down the line. And you can play games on the front end to present the illusion that operations are accepted on the server (in pending status) without compromising the health of the system.

In short, for the health of your system, put a queue on that, your future self will thank you later.

One final word of warning, this apply to operations, not queries. Don’t bother putting queries through the queue unless they are intended to be very long lived / complex ones.