Phusion Blog

Phusion Passenger version 5.0.0 beta 3 was released today, with a number of changes to the turbocache. The turbocache is a component in Passenger 5 that automatically caches HTTP responses in an effort to speed up the application. The turbocache is not a feature-rich HTTP cache like Varnish, but instead it’s more like a “CPU L1 cache for the web” — small and fast, requires little configuration, but has fewer features by design.

The turbocache was implemented with HTTP caching standards in mind, namely RFC 7234 (HTTP 1.1 Caching) and RFC 2109 (HTTP State Management Mechanism). Our initial mindset in implementing the turbocache was like that of compiler implementors: if something is allowed by the standards, then we’ll implement it in order pursue maximum possible performance.

But it turns out that following the standards strictly may raise security concerns. A while ago, we were contacted by Chris Heald, who provided convincing cases on why the turbocache’s behavior is problematic from a security standpoint. Chris is a veteran Varnish user and has a lot of experience on the subject of HTTP caching.

The gist is that the turbocache made it too easy to accidentally cache responses which should not be cached. Imagine that a certain response contains sensitive information and is only meant to be served to one user. Such sensitive information may include security credentials or session cookies. If the application sets HTTP caching headers incorrectly, then the turbocache may serve that response to other users, resulting in an information leak.

Such problems are technically bugs in the application. After all, Passenger is just following the standards. But Chris asserted that such mistakes are very easily made. Indeed, he has seen quite a number of applications that send out incorrect HTTP caching headers. For this reason, he believes that the turbocache should be more conservative by default.

Cookies also deserve special attention. RFC 2109 mentions that cookies may only be cached when they are intended to be shared by multiple users. But it is impossible for Passenger to know the intention of the cookies without configuration from the application developer. At the same time, providing such configuration goes against the spirit of the turbocache, namely that it should be easy and automatic.

It was clear that something had to be done.

Turbocache changes

More conservative

After contemplating the issues, we’ve decided to make the turbocache more conservative in order to avoid the most common security issues.

In beta 1 and beta 2, the turbocache caches all “default cacheable responses”, as defined by RFC 7234. These are responses to GET requests; with status code 200, 203, 204, 300, 301, 400, 405, 410, 414 or 501; and for which no headers are set that prevent caching, e.g. “Cache-Control: no-store”. This means that a GET request which yields a 200 response with no caching headers, is actually cacheable per the standards, and so it was cached by the turbocache.

In beta 3, responses are no longer cached unless the application explicitly sends caching headers. That means that a GET request which yields a 200 response is only cached if the application sends an “Expires” or a “Cache-Control” header (which must not contain “private”, “no-store”, etc).

While it is still possible for some application responses to be unintentionally cached, this change solves the majority of the issues. The only way to avoid these issues out of the box is by disabling caching completely, which has the downside of not being able to leverage said caching features. Instead, we believe it is reasonable that we only cache requests in case the application asks for this explicitly. A potential caveat is that even though the application and Passenger respect the specifications, an application developer may not have a full understanding of how caching behaves according to said specs. A developer should choose to disable caching entirely in such cases.

Bugs fixed

While investigating Chris’s case, we also uncovered some bugs in the turbocache, such as the incorrect handling of certain headers. These bugs have been fixed. The full details can be found in the Passenger 5 beta 3 release notes.

Using the turbocache and speeding up applications

Now that the turbocache has changed in behavior, here are some practical tips on what you can do to make good use of the turbocache.

Learn about HTTP caching headers

The first thing you should do is to learn how to use HTTP caching headers. It’s pretty simple and straightforward. Since the turbocache is just a normal HTTP shared cache, it respects all the HTTP caching rules.

Set an Expires or Cache-Control header

To activate the turbocache, the response must contain either an “Expires” header or a “Cache-Control” header.

The “Expires” header tells the turbocache how long to cache a response. Its value is an HTTP timestamp, e.g. “Thu, 01 Dec 1994 16:00:00 GMT”.

The Cache-Control header is a more advanced header that not only allows you to set the caching time, but also how the cache should behave. The easiest way to use it is to set the max-age flag, which has the same effect as setting “Expires”. For example, this tells the turbocache that the response is cacheable for at most 60 seconds:

Cache-Control: max-age=60

As you can see, a “Cache-Control” header is much easier to generate than an “Expires” header. Furthermore, “Expires” doesn’t work if the visitor’s computer’s clock is wrongly configured, while “Cache-Control” does. This is why we recommend using “Cache-Control”.

Another flag to be aware of is the private flag. This flag tells any shared caches — caches which are meant to store responses for many users — not to cache the response. The turbocache is a shared cache. However, the browser’s cache is not, so the browser can still cache the response. You should set the “private” flag on responses which are meant for a single user, as you will learn later in this article.

And finally, there is the no-store flag, which tells all caches — even the browser’s — not to cache the response.

Here is an example of a response which is cacheable for 60 seconds by the browser’s cache, but not by the turbocache:

Cache-Control: max-age=60,private

The HTTP specification specifies a bunch of other flags, but they’re not relevant for the turbocache.

Only GET requests are cacheable

The turbocache currently only caches GET requests. POST, PUT, DELETE and other requests are never cached. If you want your response to be cacheable by the turbocache, be sure to use GET requests, but also be sure that your request is idempotent.

Avoid using the “Vary” header

The “Vary” header is used to tell caches that the response depends on one or more request headers. But the turbocache does not implement support for the “Vary” header, so if you output a “Vary” header then the turbocache will not cache your response at all. Avoid using the “Vary” header where possible.

Common application caching bugs

Even though the turbocache has become more conservative now, it is still possible for application responses to be unintentionally cached if the application outputs incorrect caching headers. Here are a few tips on preventing common caching bugs.

Varying response by Ajax

A common pattern is to return a different response depending on whether or not it was an Ajax call. For example, consider this Rails app, which returns a JSON response for Ajax calls, HTML response otherwise:

def show
  headers["Cache-Control"] = "max-age: 600"
  if request.xhr?
    render json: "show"
  else
    render
  end
end

This can cause the turbocache to return the JSON response for non-Ajax calls, or to return the HTML response for Ajax calls.

There are two ways to solve this problem:

1. Set the “Vary: X-Requested-With” header so that the cache knows the response depends on this header. Rails’s request.xhr? method checks this header. Note that the turbocache currently disables caching altogether upon encountering a “Vary” header.
2. Do not vary the response based on whether or not it is an Ajax call. Instead, vary the response based on the URI. For example, you can return JSON responses only if the URI ends with “.json”.

Be careful with caching when setting cookies

If your application outputs cookies then you should be careful with your caching headers. You should only allow the turbocache to cache the response if all cookies may be cacheable by multiple users. If any of your cookies contain user-specific information, or if you’re not sure, then you should set the “private” flag in “Cache-Control” to prevent the turbocache from caching it.

We realize that lots of responses can output cookies, so we’re working on ways to improve the cacheability of responses with cookies.

Set “Cache-Control: private” when working with sessions

If your application works with sessions then you must ensure that all your “Cache-Control” headers set the “private” flag so that the turbocache does not cache it.

Note that your app may work with sessions indirectly. Any Rails page which outputs a form will result in Rails setting a CSRF token in the session. Luckily Rails sets “Cache-Control: private” by default.

Performance considerations

The turbocache plays a major role in the performance of Passenger 5. The performance claims we made are with turbocaching enabled. Now that the turbocache has become more conservative, the performance claims still stand, but may require more application modifications than before.

By strictly following the caching standards, we had hoped that we’d be able to deliver performance for most apps out-of-the-box without modifications. But we also value security, and we value that more so than performance, which is why we made the turbocache more conservative.

But this raises a question: suppose that we can’t rely on the turbocache anymore as a major factor in performance, what else can we do to improve Passenger 5’s performance, and is it worth it?

There is in fact one more thing we can do. and that is by introducing a new operational mode which bypasses a layer. In the current Passenger 5 architecture, all requests go through a process called the HelperAgent. This extra process introduces some context switching overhead and processing overhead. The overhead is negligible in most real-world scenarios, but the overhead is very apparent in hello world benchmarks. It is possible to bypass this process, allowing clients to directly communicate with application processes. But doing so comes at a cost:

• Some features — e.g. load balancing, multitenancy, out-of-band garbage collection, memory checking and statistics collection — are best implemented in the HelperAgent. Reimplementing them inside application processes is either difficult or less efficient.
• Rearchitecting Passenger this way requires a non-trivial amount of development time.

So it remains to be seen whether bypassing the HelperAgent is worth it. Investing time in this means that we’ll have less time available to pursue other goals on the roadmap. What do you think about this? Just post a comment and let us know.

Conclusion

Passenger 5 beta 3 is the first “more or less usable in production” release of the Passenger 5 series. Previous releases were not ready for production, but with beta 3 we are confident enough that the most important issues are solved. The turbocache issue as described in this article is one of them. If you were on a previous Passenger 5 release, then we strongly recommend you to upgrade.

We would like to thank Chris Heald for his excellent feedback. We couldn’t have done this without him.

We expect the Passenger 5 final stable version to be released in February. Beta 3 is supposed to be the last beta. Next up will be Release Candidate 1, followed by the final stable release.

But the story doesn’t end there. We will publish a roadmap in the near future, which describes all the ambitious plans we have in store for Passenger. Passenger is constantly improving and evolving, so please stay tuned for updates.

We’ve just released version 5.0.0 beta 3 of the Phusion Passenger application server for Ruby, Python and Node.js. The 5.x series is also unofficially known under the codename “Raptor”, and introduces many major improvements such as performance enhancements, better tools for application-level visibility and a new HTTP JSON API for accessing Passenger’s internals.

So far, we’ve discouraged using 5.0 in production because it’s still in beta and because it was known to be unstable. But this changes with beta 3, which we consider “more or less ready for production”. This means that we’re confident that most of the major issues have been solved, but you should still exercise caution if you roll it out to production.

Final stable

In February, we will release the first Release Candidate version. If everything goes well, 5.0 final — which is officially ready for production — will also be released in February.

Changes in this version

Turbocaching updates

Main article: Turbocaching security changes

One of the major features in Phusion Passenger 5 is the turbocache, an integrated and high-performance HTTP response cache. It is responsible for a large part of the performance improvements in version 5. In beta 3, we’ve given the turbocache a few major updates:

We’ve been researching ways to improve the turbocache. Based on community feedback on the turbocache, we’ve found that the turbocache in its previous form wasn’t so useful. So we’ve come up with a few ways to allow apps to be better cacheable. These techniques are well-established and have been extensively used in advanced Varnish setups.

We’ve made the turbocache more secure and more conservative, based on excellent feedback from Chris Heald and the community. In previous versions, default cacheable responses (as defined by RFC 7234) were cached unless caching headers tell us not to. Now, default cacheable responses are only cached if caching headers explicitly tell us to. This change was introduced because there are many applications that set incorrect caching headers on private responses. This new behavior is currently not configurable, but there are plans to make it configurable in 5.0.0 release candidate 1.

Learn more about this change here.

Heroku support restored

Beta 1 and beta 2 didn’t work on Heroku because of bugs. With beta 3, Heroku support has been fixed. This closes GH-1329.

Nginx < 1.6 support dropped

Nginx versions earlier than 1.6 are no longer supported.

Bug fixes

Many major bugs have been fixed. These bugs include crashers, cookie handling issues, connection handling issues, etc:

• Fixed various turbocache header handling issues.
• Fixed cookie handling issues. Closes GH-1310.
• Fixed various WebSocket issues. Closes GH-1306.
• Fixed some crashes caused by race conditions. Closes GH-1326.
• Fixed issues with handling POST data. Closes GH-1331.
• Fixed some integer overflows. Fix contributed by Go Maeda. Closes GH-1357.
• Fixed the passenger-status --show=union_station command. Closes GH-1336.

Miscellaneous

A new configuration option, passenger_response_buffer_high_watermark (Nginx) and PassengerResponseBufferHighWatermark (Apache), has been introduced. This allows for configuring the behavior of the response buffering system. Closes GH-1300.

State introspection has been improved. This means that passenger-status --show=requests shows better and more detailed output now.

Installing or upgrading

Here’s a quickstart:

gem install passenger --pre -v 5.0.0.beta3
cd /path-to-your-app
passenger start

The above is a very short excerpt of how to install or upgrade Phusion Passenger. For detailed instructions (which, for example, take users and permissions into account), please refer to the “RubyGems” section of the installation manuals:

• Standalone mode
• Nginx mode
• Apache mode

Please note:

• 5.0.0 beta 3 is a beta release. There are no major bugs left, but please exercise caution and report any issues.
• There are no Homebrew or Debian packages for this release, because this release is still in beta!
• There is also a 5.0.0 beta 3 release of Phusion Passenger Enterprise available. Please refer to the Customer Area.

Phusion Passenger is a fast and robust web server and application server for Ruby, Python, Node.js and Meteor. Passenger takes a lot of complexity out of deploying web apps, and adds powerful enterprise-grade features that are useful in production. High-profile companies such as Apple, New York Times, AirBnB, Juniper, American Express, etc are already using it, as well as over 350.000 websites.

Yesterday, Aman Gupta announced an Out-Of-Band garbage collector for Ruby 2.1. What is an “Out-Of-Band garbage collector”? It means that the garbage collector is run in between requests, instead of during a request. While the garbage collector is running, other processes can serve requests. This way visitors will never experience the slight delay that is caused by the garbage collector, much improving response times. We just implemented support for this in Phusion Passenger.

Classical out-of-band garbage collection

Out-of-band garbage collection is not a new idea. The idea was first introduced in Unicorn several years ago. Its mechanism was simple: disable automatic garbage collection, and manually run the garbage collector at the end of every N requests.

Last year, we introduced a much improved version of Unicorn’s out-of-band garbage collection mechanism in Phusion Passenger 4. This feature was originally contributed by AppFolio, and since then many further improvements have been made.

• Unicorn’s out-of-band garbage collection doesn’t work well with multithreaded servers, because a garbage collection is a stop-the-world action that blocks all threads. Phusion Passenger’s mechanism lifts this limit by intelligently coordinating the garbage collector and threads.
• Unicorn’s mechanism is susceptible to occasional pathological behavior. It is possible for many, or all, Unicorn processes to be simultaneously running the out-of-band garbage collector. During that short period of time, no requests can be served, and visitors will experience a delay. Phusion Passenger’s mechanism ensures that only 1 process can run the garbage collector at a time, thereby eliminating this problem.
• Phusion Passenger’s mechanism is more flexible, and can be used for work other than just garbage collection.

The results were impressive. AppFolio observed an average response time reduction of 100 ms.

Before and after applying out of band GC at AppFolio

Unfortunately, this Out-Of-Band GC mechanism (running the GC every few requests) is still suboptimal, as we had pointed out last year:

• If a request generates a lot of garbage, then delaying the GC for a few more request can result in high peak memory usage.
• If the garbage collector doesn’t actually need to be run yet, then running it anyway wastes CPU cycles.

The new Ruby 2.1 Out-Of-Band GC

Fortunately, Ruby has recently introduced many useful garbage collector improvements. Ruby 2.1 introduced a semi-generational garbage collector. Aman Gupta built an Out-Of-Band GC mechanism on top of the recent improvements, which is almost exactly what we needed in Phusion Passenger.

Today, we’ve submitted a pull request to make his Out-Of-Band GC mechanism work well with Phusion Passenger, and we’ve introduced support in Phusion Passenger for his Out-Of-Band GC mechanism. Using it is very easy. First, add our patched gctools to your Gemfile:

gem "gctools", :github => "FooBarWidget/gctools", :require => false

Next, put this in your config.ru:

require "gctools/oobgc"
GC::OOB.setup

Now you can enable the Out-Of-Band GC feature in Phusion Passenger by using the :gctools_oobgc strategy:

PhusionPassenger.require_passenger_lib 'rack/out_of_band_gc'
use PhusionPassenger::Rack::OutOfBandGc, :strategy => :gctools_oobgc

Aman Gupta’s improvements are very impressive indeed. With Ruby 2.1, his average OOBGC pause time (oobgc.mean) “went from 125ms to 50ms thanks to RGenGC”. And “the number of out-of-band collections (oobgc.count) also went down, since the new OOBGC only runs when necessary”.

From Aman Gupta’s blog: “The overall result is much less CPU time (oobgc.sum) spent doing GC work between requests”:

Conclusion

Support for Aman Gupta’s Ruby 2.1 Out-Of-Band GC mechanism will be included in the next release of Phusion Passenger. It’s not completely finished yet: as seen in our pull request, there are still some issues to be fleshed out. But the results are very promising indeed. Although the Ruby hype is over, there are still plenty of improvements going on.

Discuss this on Hacker News.