Phusion Blog

Executive summary

Phusion Passenger is an app server that supports Ruby. We have released version 5 beta 1, codename “Raptor”. This new version is much faster, helps you better identify and solve problems, and has a ton of other improvements. If you’ve followed the Raptor campaign then you may wonder why we held the campaign like this. Read on if you’re interested. If you just want to try it, scroll all the way down for the changelog and the installation and upgrade instructions.

Learn more about Phusion Passenger

Introduction

A month ago, we released a website in which we announced “Raptor”, supposedly a new Ruby app server that’s much faster than others. It has immediately received a lot of community attention. It was covered by Fabio Akita and by RubyInside’s Peter Cooper. From the beginning, “Raptor” was Phusion Passenger 5, a new major version with major internal overhauls. In the weeks that followed, we blogged about how we made “Raptor” fast.

Even though “Raptor” is Phusion Passenger, it doesn’t make the impact any less powerful. The performance improvements that we claim are real, and they are open source. Because “Raptor” is Phusion Passenger 5, it means that it automatically has a mature set of features:

• Handle more traffic
  Phusion Passenger 5 is up to 4x faster than other Ruby app servers, allowing you to handle more traffic with the same hardware.
• Reduce maintenance
  Automates more system tasks than other app servers. Spend less time micromanaging software, and more time building your business.
• Identify & fix problems quickly
  Why is your app behaving the way it does? What is it doing? Phusion Passenger 5 provides tools that give you the insights you need.
• Keep bugs & issues in check
  Limit the impact of bugs and issues, making downtime and user dissatisfaction less likely. Reduce pressure on developers while the root problem is being fixed.
• Excellent support
  We have excellent documentation and a vibrant community discussion forum. And with our professional and enterprise support contracts, you can consult our team of experts directly.
• Improve security
  Provides extra layers of defense, reduces attack surfaces on applications and limits the impact of security vulnerabilities.

However, the authors behind “Raptor” remained unknown — until today. The reason why we ran the campaign like this is explained in this article.

A brief history

It is perhaps hard to fathom now, but in the early days of Ruby, getting an app into a production environment was a painful task in itself. Many hours were spent by developers on tedious tasks such as manually managing ports and performing other error-prone configuration sit-ups. The status quo of deployment back then wasn’t exactly in line with what Rails advocates through its “convention over configuration” mantra. Far from it in fact.

When we first introduced Phusion Passenger back in 2008, we wanted to “fix” this. We wanted Ruby deployment to be as easy as PHP so that developers could focus on their apps and lower the barrier of entry for newcomers.

Even though we have been able to help power some of the largest sites on the Internet over the past few years through Phusion Passenger, we have always remained vigilant as to not become complacent: we have been eagerly listening to the community as to what they expect the next big thing to be.

The observations we have made over the years have eventually culminated into Phusion Passenger 5, which was codenamed Raptor for a number of reasons.

A greater focus on performance and efficiency

Whether you are deploying a small web app on a VPS or spinning up tens of thousands of instances to power your e-commerce business, we all want the most bang for our buck. Being able to reduce the number of required servers would be beneficial in reducing costs and it is for this reason that developers seek to employ the most efficient software stack currently available. When it comes to making that choice, benchmarks from third parties often seem to play an important part in the decision making process. Even though they are convenient to consult, it is easy to overlook a couple of important things that we would like to underline.

When it comes to performance benchmarks for example, it does not always become clear how the results have been obtained and how they will affect the reader. This is mostly due to the fact that benchmarks are often performed on synthetic applications in synthetic environments that don’t take into consideration real world workloads and latencies. This often leads to skewed results when compared to real time workloads.

A good example of this is the “Hello World” benchmark, where people tend to benchmark app servers against a so-called “Hello World” application: an app that basically returns “Hello World”. Needless to say, this is hardly a real world application.

Anyone who has ever deployed a real world application will know that these kinds of benchmarks don’t really say anything useful as they effectively measure how fast an app server is at “doing nothing”.

In real world Ruby applications on the other hand, processing time quickly gets overtaken by the app itself, plus network overhead, rather than the app server: the differences in performance between the app servers basically become insignificant when compared to the time spent in the hosted app itself.

Despite this, benchmarks remain a popular source to consult when trying to figure out what the “best” software solution is. The danger here lies in the possibility that a developer might be tempted to base their decision solely on what bar chart sticks out the most, without so much as looking at what the other solutions all bring to the table.

There is a reason for example why Phusion Passenger — even though hot on the heels of its competitors — has not been leading these kinds of Hello World benchmarks in the past: we do much more than the competition does when it comes to ease of use, memory efficiency, security and features. All these things are not free, and this is basically what is being measured with said benchmarks.

When benchmarked against real world applications however, the differences in performance between app servers becomes almost indistinguishable. The focus should then be put on what feature-set is most suitable for your production app. We believe that on that front, Phusion Passenger is leading the pack.

We have tried many times to explain this in the comment sections of benchmarks, but have unfortunately had to infer that such explanations often fall on deaf ears. We think that’s a shame, but rather than continue to fight it, we have decided to try to beat our competitors on performance as well. This has led to a series of internal optimizations and innovations which we have documented in the Raptor articles. Not only do we believe we are now able to win these kinds of benchmarks, we believe we have been able to do so with mechanisms that are incredibly useful in real world scenarios too (e.g. Turbocaching).

A greater focus on showcasing the technology

Software that is easy to use runs the risk of being considered “boring” to hackers, or worse, “simple”. In the latter case, the underlying technology facilitating the ease of use gets taken granted for. Over the years, we felt this was happening to Phusion Passenger to the extent that we wanted to set the record straight.

A lot of thought went into using the right algorithms and applying the right optimizations to allow Phusion Passenger to do what it does best: being able to deliver an “upload-and-go” deployment experience second to none in a secure and performant manner is by no means a trivial task. We chose to abstract these implementation details however from the end-user as we wanted them to be able to focus more on their app and business rather than the nitty gritty when it came down to how “the soup was made”. Who cares right?

Well, as it turned out, a lot of hackers do. Articles about “Unicorn being Unix” sparked a lot of interest from hackers allowing it to quickly garner a following. We thought articles such as these were great, but felt somewhat disappointed that people seemed to forget that Phusion Passenger was already doing the majority of what was written in such articles a full year earlier. It then dawned on us that we were no longer being considered to be the new shiny thing, but rather considered being part of the establishment.

In hindsight, it was perhaps also an error of judgement of us to focus our marketing efforts mostly on businesses rather than the grassroots hacker community we originated from ourselves: they are not mutually exclusive and instead of mostly underlining the business advantages, we should have underlined the technological advantages much more as well.

In an effort to set this straight, we chose to document Raptor’s underlying technology in great detail which was incredibly well received by the Hacker News community. It was in fact on the front page for most of the day with close to 200 upvotes and sparked a lot of discussions on Twitter too.

Besides the new optimizations and features found in Raptor, a lot of technology discussed in those articles was already available in Phusion Passenger 4 and its precursors. If there is a lesson to take from all this, it is that marketing is indeed the art of repetition. And that it probably helps to present it as the new kid on the block to get rid of any preconceived notions/misconceptions people might have had about Phusion Passenger.

Smoke and mirrors

Whether or not people would be just as excited if they knew that it was Phusion Passenger 5 all along is perhaps another discussion to be had: some actually found out ahead of time due to the similar writing style of our tech articles and / or through nslookups (a fedora hat-tip goes out to you folks! May your sense of scrutiny live long and prosper!).

What we do however know is that our Raptor approach over the past month has produced more subscribers to our newsletter than we have been able to accomplish over the past 6 years through the Phusion Passenger moniker. We still have a hard time comprehending this, but there is no denying the numbers: we — the community — seem to like shiny new things.

Truth be told, we didn’t really bother trying to cover up the fact that it was in fact Phusion all along that was behind Raptor. We kind of left it as an exercise to the reader to figure this out amidst all the “hype” and claims of “vaporware” to see if people still remembered Phusion Passenger’s fortes.

We were not disappointed when it came to that and felt incredibly proud that a lot of people questioned why Phusion Passenger was not included within the Raptor benchmarks and requested it to be included. Needless to say, this was something we were unable to do because it already was included all along as Raptor itself 😉 We were also happy to see that some even pointed out that some of the features look like they came straight out of Phusion Passenger.

What’s in a name?

You might be wondering why we chose to market Phusion Passenger 5 under the Raptor code name and went through so many hoops in doing so. To quote Shakespeare: “Conceal me what I am, and be my aid, for a disguise as haply shall become, the form of my intent”.

With all the new improvements and features pertaining to performance and introspection, we felt Phusion Passenger deserved new consideration from its audience in an objective manner. To circumvent any preconceived notions and/or misconceptions people may have had about Phusion Passenger over the years, we decided to market it as Raptor. We felt the codename was particularly appropriate for our renewed commitment to performance.

Just to be clear, Phusion Passenger will not be renamed to Raptor. Raptor is just a codename that has served its purpose by the time of this writing for Phusion Passenger 5. We will drop this name starting from today: from now on, “Raptor” is Phusion Passenger 5.

With a little help from our friends

The success of the Raptor campaign would not have been possible without the help and support of our friends. In particular, we would like to thank Peter Cooper and Fabio Akita for their in-depth write-ups on Phusion Passenger 5 / Raptor and its precursors. Their articles carried the necessary weight to allow us to focus on explaining and improving the technology itself rather than having to spend time on trying to debunk “vaporware” claims. In a similar manner, we would also like to thank David Heinemeier Hansson for helping us out via his tweets and feedback.

Lastly, we would like to thank the community and our customers for being so supportive. At the end of the day, it is you folks who make this all possible and we can’t wait to show you what we have in store for the future.

Final words before upgrade instructions

If you would like to stay up to date with Phusion news, please fill in your name and email address below and sign up for our newsletter. We won’t spam you, we promise.

Changelog

5.0.0 beta 1 is a beta release. It may still have bugs and we do not recommend using it in production yet.

Version 5.0.0 beta 1 contains major changes. It is mostly compatible with version 4, but there are a few minor breakages, which are described below. Major changes and notable breakages are:

• Performance has been much improved. This is thanks to months of optimization work. You can learn more at www.rubyraptor.org.
• We have published a server optimization guide for those who are interested in tuning Phusion Passenger.
• Support for Rails 1.2 – 2.2 has been removed, for performance reasons. Rails 2.3 is still supported.
• Phusion Passenger now supports integrated HTTP caching, which we call turbocaching. If your app sets the right HTTP headers then Phusion Passenger can tremendously accelerate your app. It is enabled by default, but you can disable it with --disable-turbocaching (Standalone), PassengerTurbocaching off (Apache), or passenger_turbocaching off (Nginx).
• Touching restart.txt will no longer restart your app immediately. This is because, for performance reasons, the stat throttle rate now defaults to 10. You can still get back the old behavior by setting PassengerStatThrottleRate 0 (Apache) or passenger_stat_throttle_rate 0 (Nginx), but this is not encouraged. Instead, we encourage you to use the passenger-config restart-app tool to initiate restarts, which has immediate effect.
• Websockets are now properly disconnected on application restarts.
• The Phusion Passenger log levels have been completely revamped. If you were setting a log level before (e.g. through passenger_log_level), please read the latest documentation to learn about the new log levels.
• If you use out-of-band garbage collection, beware that the X-Passenger-Request-OOB-Work header has now been renamed to !~Request-OOB-Work.
• When using Rack’s full socket hijacking, you must now output an HTTP status line.
• [Nginx] The passenger_set_cgi_param option has been removed and replaced by passenger_set_header and passenger_env_var.
• [Nginx] passenger_show_version_in_header is now only valid in the http context.
• [Apache] The PassengerStatThrottleRate option is now global.

Minor changes:

• The minimum required Nginx version is now 1.6.0.
• The instance directory is now touched every hour instead of every 6 hours. This should hopefully prevent more problems with /tmp cleaner daemons.
• Applications are not grouped not only on the application root path, but also on the environment. For example, this allows you to run the same app in both production and staging mode, with only a single directory, without further configuration. Closes GH-664.
• The passenger_temp_dir option (Nginx) and the PassengerTempDir option (Apache) have been replaced by two config options. On Nginx they are passenger_instance_registry_dir and passenger_data_buffer_dir. On Apache they are PassengerInstanceRegistryDir and PassengerDataBufferDir. On Apache, PassengerUploadBufferDir has been replaced by PassengerDataBufferDir.
• Command line tools no longer respect the PASSENGER_TEMP_DIR environment variable. Use PASSENGER_INSTANCE_REGISTRY_DIR instead.
• passenger-status --show=requests has been deprecated in favor of passenger-status --show=connections.
• Using the SIGUSR1 signal to restart a Ruby app without dropping connections, is no longer supported. Instead, use passenger-config detach-process.
• Introduced the passenger-config reopen-logs command, which instructs all Phusion Passenger agent processes to reopen their log files. You should call this after having rotated the web server logs.
• [Standalone] The Phusion Passenger Standalone config template has changed. Users are encouraged to update it.
• [Standalone] passenger-standalone.json has been renamed to Passengerfile.json.
• [Standalone] passenger-standalone.json/Passengerfile.json no longer overrides command line options. Instead, command line options now have the highest priority.

Installing or upgrading

Here’s a quickstart:

gem install passenger --pre -v 5.0.0.beta1
cd /path-to-your-app
passenger start

The above is a very short excerpt of how to install or upgrade Phusion Passenger. For detailed instructions (which, for example, take users and permissions into account), please refer to the “RubyGems” section of the installation manuals:

• Standalone mode
• Nginx mode
• Apache mode

Please note:

• 5.0.0 beta 1 is a beta release. It may still have bugs and we do not recommend using it in production yet.
• There are no Homebrew or Debian packages for this release, because this release is still in beta!
• There is also a 5.0.0 beta 1 release of Phusion Passenger Enterprise available. Please refer to the Customer Area.