Phusion Blog

Phusion Passenger is a fast and robust web server and application server for Ruby, Python, Node.js and Meteor. Passenger takes a lot of complexity out of deploying web apps, and adds powerful enterprise-grade features that are useful in production. High-profile companies such as Apple, New York Times, AirBnB, Juniper, American Express, etc are already using it, as well as over 350.000 websites.

Yesterday, Aman Gupta announced an Out-Of-Band garbage collector for Ruby 2.1. What is an “Out-Of-Band garbage collector”? It means that the garbage collector is run in between requests, instead of during a request. While the garbage collector is running, other processes can serve requests. This way visitors will never experience the slight delay that is caused by the garbage collector, much improving response times. We just implemented support for this in Phusion Passenger.

Classical out-of-band garbage collection

Out-of-band garbage collection is not a new idea. The idea was first introduced in Unicorn several years ago. Its mechanism was simple: disable automatic garbage collection, and manually run the garbage collector at the end of every N requests.

Last year, we introduced a much improved version of Unicorn’s out-of-band garbage collection mechanism in Phusion Passenger 4. This feature was originally contributed by AppFolio, and since then many further improvements have been made.

• Unicorn’s out-of-band garbage collection doesn’t work well with multithreaded servers, because a garbage collection is a stop-the-world action that blocks all threads. Phusion Passenger’s mechanism lifts this limit by intelligently coordinating the garbage collector and threads.
• Unicorn’s mechanism is susceptible to occasional pathological behavior. It is possible for many, or all, Unicorn processes to be simultaneously running the out-of-band garbage collector. During that short period of time, no requests can be served, and visitors will experience a delay. Phusion Passenger’s mechanism ensures that only 1 process can run the garbage collector at a time, thereby eliminating this problem.
• Phusion Passenger’s mechanism is more flexible, and can be used for work other than just garbage collection.

The results were impressive. AppFolio observed an average response time reduction of 100 ms.

Unfortunately, this Out-Of-Band GC mechanism (running the GC every few requests) is still suboptimal, as we had pointed out last year:

• If a request generates a lot of garbage, then delaying the GC for a few more request can result in high peak memory usage.
• If the garbage collector doesn’t actually need to be run yet, then running it anyway wastes CPU cycles.

The new Ruby 2.1 Out-Of-Band GC

Fortunately, Ruby has recently introduced many useful garbage collector improvements. Ruby 2.1 introduced a semi-generational garbage collector. Aman Gupta built an Out-Of-Band GC mechanism on top of the recent improvements, which is almost exactly what we needed in Phusion Passenger.

Today, we’ve submitted a pull request to make his Out-Of-Band GC mechanism work well with Phusion Passenger, and we’ve introduced support in Phusion Passenger for his Out-Of-Band GC mechanism. Using it is very easy. First, add our patched gctools to your Gemfile:

gem "gctools", :github => "FooBarWidget/gctools", :require => false

Next, put this in your config.ru:

require "gctools/oobgc"
GC::OOB.setup

Now you can enable the Out-Of-Band GC feature in Phusion Passenger by using the :gctools_oobgc strategy:

PhusionPassenger.require_passenger_lib 'rack/out_of_band_gc'
use PhusionPassenger::Rack::OutOfBandGc, :strategy => :gctools_oobgc

Aman Gupta’s improvements are very impressive indeed. With Ruby 2.1, his average OOBGC pause time (oobgc.mean) “went from 125ms to 50ms thanks to RGenGC”. And “the number of out-of-band collections (oobgc.count) also went down, since the new OOBGC only runs when necessary”.

From Aman Gupta’s blog: “The overall result is much less CPU time (oobgc.sum) spent doing GC work between requests”:

Conclusion

Support for Aman Gupta’s Ruby 2.1 Out-Of-Band GC mechanism will be included in the next release of Phusion Passenger. It’s not completely finished yet: as seen in our pull request, there are still some issues to be fleshed out. But the results are very promising indeed. Although the Ruby hype is over, there are still plenty of improvements going on.

Discuss this on Hacker News.