Phusion Blog

Phusion Passenger is an Apache and Nginx module for deploying Ruby and Python web applications. It has a strong focus on ease of use, stability and performance. Phusion Passenger is built on top of tried-and-true, battle-hardened Unix technologies, yet at the same time introduces innovations not found in most traditional Unix servers. Since mid-2012, it aims to be the ultimate polyglot application server.

Development of the Phusion Passenger 4.x series is progressing steadily. The 4.x series is a huge improvement over the 3.x series: in the announcement for Phusion Passenger 4.0.0 beta 1, we introduced a myriad of changes such as support for multiple Ruby versions, Python WSGI support, multithreading (Enterprise only), improved zero-copy architecture, better error diagnostics and more. That was just the beginning, because soon after we announced JRuby and Rubinius support. Today we are announcing another cool feature.

Out-of-Band Work

The Out-of-Band Work feature allows one to perform arbitrary long-running work outside request cycles without blocking HTTP clients. The primary use case is to run the garbage collector in between request cycles so that your requests will finish faster because they will be interrupted less by the garbage collector.

Normally the garbage collector fires up as soon as the Ruby interpreter thinks it needs to, which possibly results in hundreds of milliseconds of latency. With the Out-of-Band Work feature, you can run the garbage collector outside the request cycles so that garbage collection runs inside cycles are much less expensive. While out-of-band work is running, Phusion Passenger will not route any requests to said process.

Cool properties of this feature:

• If the process that triggered Out-Of-Band Work is the only process for that application, then Phusion Passenger will first spawn up a new process before performing the out-of-band work. When the out-of-band work has finished, the process will be eligible for idle timeout cleaning. Thus you can use this feature in any scenario, and Phusion Passenger will do the right thing for you.
• It even works in multithreaded setups (an Enterprise-only feature). Normally the Ruby garbage collector will block all threads while doing work. So before performing the out-of-band work, Phusion Passenger will let all existing requests to the application process finish.

This awesome feature has been contributed by AppFolio. They’ve been running it in production for a while now with quite some success. Average response time has gone down by 100 ms.

Compared to Unicorn’s OOBGC

Users who are familiar with Unicorn’s Out-of-Band GC (OOBGC) might notice the similarities. Our Out-of-Band Work feature (OOBW) is more general and more flexible:

• Unicorn’s OOBGC requires a static number of single-threaded workers. OOBW is designed to be able to handle a dynamic number of workers that may even be multithreaded.
• OOBW is designed to be able to perform arbitrary long-running work, including work that may block all threads. Unicorn’s OOBGC only works with garbage collection.

Using Out-Of-Band Work

Note January 31 2014: the following code snippet is outdated and no longer works on the latest version of Phusion Passenger. Please refer to the Phusion Passenger manual for the latest, correct, code snippet.

Phusion Passenger 4.0 beta 2 provides a simple Rack middleware that you can use to enable out-of-band GC:

if defined?(PhusionPassenger)
  require 'phusion_passenger/rack/out_of_band_gc'
  # Trigger out-of-band GC every 5 requests.
  use PhusionPassenger::Rack::OutOfBandGc, 5
  ## Optional: disable normal GC triggers and only GC outside
  ## request cycles. Not recommended though, see section
  ## "What Ruby can do to improve out-of-band garbage collection"
  # GC.disable
end

It also provides a simple API to perform Out-Of-Band Work. For example out-of-band GC may be implemented as follows without using the Rack middleware:

# Somewhere in a controller method:
# Tell Phusion Passenger we want to perform OOB work.
response.headers["X-Passenger-Request-OOB-Work"] = "true"

# Somewhere during application initialization:
if defined?(PhusionPassenger)
  PhusionPassenger.on_event(:oob_work) do
    # Phusion Passenger has told us that we're ready to perform OOB work.
    t0 = Time.now
    GC.start
    Rails.logger.info "Out-Of-Bound GC finished in #{Time.now - t0} sec"
  end
end    

## Optional: disable normal GC triggers and only GC outside
## request cycles. Not recommended though, see section
## "What Ruby can do to improve out-of-band garbage collection"
# GC.disable

Inside Out-Of-Band Work: a more general mechanism

The Out-Of-Band Work feature is actually built on top of an even more general mechanism: the enable/disable process feature. This is a new feature in Phusion Passenger 4 and is, for now, internal only. Internal Phusion Passenger code can mark a process as disabled, so that Phusion Passenger will no longer route requests to it. But the actual process is kept alive. Internal code can reenable the process later, making it eligible again for processing requests.

This feature is simple to use and simple to understand, but was tricky to implement. Phusion Passenger works in a heavily concurrent environment so it may not be able to disable a process immediately. The process might be handling requests, it might be restarting, another process might be spawning, etcetera. The entire API follows an asynchronous design. If the to-be-disabled process is the only process for that application, then Phusion Passenger will spawn another process. Disabling the original process will complete when the new process has been spawned, and the original process is done processing all its requests.

Once the enable/disable feature was in place, implementing Out-Of-Band Work was almost trivial. When an application wants to perform Out-Of-Band Work, it sends a signal to Phusion Passenger. We currently use the X-Passenger-OOB-Work header to do this, which is filtered out by Phusion Passenger and will never reach the client. Phusion Passenger will then try to disable the process. Once disabled, Phusion Passenger will send a signal to the application, telling it that it may proceed to perform out-of-band work. At this point the process is guaranteed not to be processing any requests, so it can freely do whatever it wants. Once the out-of-band work has finished, Phusion Passenger will reenable the process.

This simple mechanism opens the door to many other possibilities that are currently not implemented:

• In the future we can add an admin command to access the API, so that the administrator can disable/enable processes. That way the administrator can temporarily isolate a process for debugging without disrupting production traffic.
• Phusion Passenger Enterprise’s live IRB console feature can optionally disable the process before attaching itself, so that the administrator can debug the process without him being disrupted by traffic.

What Ruby can do to improve out-of-band garbage collection

The currently recommended mode is to run the out-of-band garbage collection with the normal Ruby garbage collector turned on. This significantly reduces the latency of normal garbage collection runs, but does not eliminate them. It is possible to completely eliminate normal garbage collection latency by disabling the garbage collector so that garbage collection is only performed out-of-band, but this will result in high peak memory usage because:

• There’s currently no way to find out when the Ruby garbage collector needs to be run. The “every x requests” option is a suboptimal heuristic.
• The MRI Ruby interpreter does not support heap compaction, so even when memory has been reclaimed by the garbage collector, Ruby may not be able to return that memory to the operating system.

It would be great if Ruby can address both issues. This will improve the usefulness of out-of-band garbage collection significantly.

Conclusion

Out-of-Band Work will become part of Phusion Passenger 4.0 beta 2, which will be released very soon. Phusion Passenger Enterprise customers can already test and enjoy this feature by downloading the “3.9.2 preview (4.0.0 beta 2)” file from the Customer Area.

Please stay tuned for further announcements on Phusion Passenger 4. If you like, you can subscribe to our newsletters and we’ll keep you up to date.