Phusion Blog

I’m not sure what I should write down here seeing as David Heinemeier Hansson pretty much sums it all up:

Indeed, Passenger now supports the ability to choose for global queing at the request of 37signals, which has the potential to be the first of many exciting collaborations with the company responsible for applications such as Basecamp, Campfire and of course, the Rails framework.

For more info on this feature as well as on Phusion Passenger’s architecture, we’d kindly like to refer you to Passenger’s user guide and architectural overview.

For completness sake, I’ve included the relevant information with regards to this feature below for your reading pleasure.

What does this option (Global Queuing) do?

Recall that Phusion Passenger spawns multiple backend processes (e.g. multiple Ruby on Rails processes), each which processes HTTP requests serially. One of Phusion Passenger’s jobs is to forward HTTP requests to a suitable backend process. A backend process may take an arbitrary amount of time to process a specific HTTP request. If the websites are (temporarily) under high load, and the backend processes cannot process the requests fast enough, then some requests may have to be queued.

If global queuing is turned off, then Phusion Passenger will use fair load balancing. This means that each backend process will have its own private queue. Phusion Passenger will forward an HTTP request to the backend process that has the least amount of requests in its queue.

If global queuing is turned on, then Phusion Passenger will use a global queue that’s shared between all backend processes. If an HTTP request comes in, and all the backend processes are still busy, then Phusion Passenger will wait until at least one backend process is done, and will then forward the request to that process.

When to turn on global queuing?

You should turn on global queuing if one of your web applications may have long-running requests.

For example suppose that:

• global queuing is turned off.
• we’re currently in a state where all backend processes have 3 requests in their queue, except for a single backend process, which has 1 request in its queue.

The situation looks like this:

Backend process A:  [*     ]  (1 request in queue)
Backend process B:  [***   ]  (3 requests in queue)
Backend process C:  [***   ]  (3 requests in queue)
Backend process D:  [***   ]  (3 requests in queue)

Each process is currently serving short-running requests.

Phusion Passenger will forward the next request to backend process A. A will now have 2 items in its queue. We’ll mark this new request with an X:

Backend process A:  [*X    ]  (2 request in queue)
Backend process B:  [***   ]  (3 requests in queue)
Backend process C:  [***   ]  (3 requests in queue)
Backend process D:  [***   ]  (3 requests in queue)

Assuming that B, C and D still aren’t done with their current request, the next HTTP request – let’s call this Y – will be forwarded to backend process A as well, because it has the least number of items in its queue:

Backend process A:  [*XY   ]  (3 requests in queue)
Backend process B:  [***   ]  (3 requests in queue)
Backend process C:  [***   ]  (3 requests in queue)
Backend process D:  [***   ]  (3 requests in queue)

But if request X happens to be a long-running request that needs 60 seconds to complete, then we’ll have a problem. Y won’t be processed for at least 60 seconds. It would have been a better idea if Y was forward to processes B, C or D instead, because they only have short-living requests in their queues.

This problem will be avoided entirely if you turn global queuing on. With global queuing, all backend processes will share the same queue. The first backend process that becomes available will take from the queue, and so this “queuing-behind-long-running-request” problem will never occur.

Turning global queuing off will yield a minor performance improvement (about 5%, depending on how fast/slow your web application is), which is why it’s off by default.