Phusion white papers Phusion overview

Phusion Blog

Introducing Phusion Passenger 5 beta 1, codename “Raptor”

By Hongli Lai on November 25th, 2014

Phusion Passenger Raptor

Executive summary

Phusion Passenger is an app server that supports Ruby. We have released version 5 beta 1, codename “Raptor”. This new version is much faster, helps you better identify and solve problems, and has a ton of other improvements. If you’ve followed the Raptor campaign then you may wonder why we held the campaign like this. Read on if you’re interested. If you just want to try it, scroll all the way down for the changelog and the installation and upgrade instructions.

Learn more about Phusion Passenger

Introduction

A month ago, we released a website in which we announced “Raptor”, supposedly a new Ruby app server that’s much faster than others. It has immediately received a lot of community attention. It was covered by Fabio Akita and by RubyInside’s Peter Cooper. From the beginning, “Raptor” was Phusion Passenger 5, a new major version with major internal overhauls. In the weeks that followed, we blogged about how we made “Raptor” fast.

Even though “Raptor” is Phusion Passenger, it doesn’t make the impact any less powerful. The performance improvements that we claim are real, and they are open source. Because “Raptor” is Phusion Passenger 5, it means that it automatically has a mature set of features:

  • Handle more traffic
    Phusion Passenger 5 is up to 4x faster than other Ruby app servers, allowing you to handle more traffic with the same hardware.
  • Reduce maintenance
    Automates more system tasks than other app servers. Spend less time micromanaging software, and more time building your business.
  • Identify & fix problems quickly
    Why is your app behaving the way it does? What is it doing? Phusion Passenger 5 provides tools that give you the insights you need.
  • Keep bugs & issues in check
    Limit the impact of bugs and issues, making downtime and user dissatisfaction less likely. Reduce pressure on developers while the root problem is being fixed.
  • Excellent support
    We have excellent documentation and a vibrant community discussion forum. And with our professional and enterprise support contracts, you can consult our team of experts directly.
  • Improve security
    Provides extra layers of defense, reduces attack surfaces on applications and limits the impact of security vulnerabilities.

However, the authors behind “Raptor” remained unknown — until today. The reason why we ran the campaign like this is explained in this article.

A brief history

It is perhaps hard to fathom now, but in the early days of Ruby, getting an app into a production environment was a painful task in itself. Many hours were spent by developers on tedious tasks such as manually managing ports and performing other error-prone configuration sit-ups. The status quo of deployment back then wasn’t exactly in line with what Rails advocates through its “convention over configuration” mantra. Far from it in fact.

When we first introduced Phusion Passenger back in 2008, we wanted to “fix” this. We wanted Ruby deployment to be as easy as PHP so that developers could focus on their apps and lower the barrier of entry for newcomers.

Even though we have been able to help power some of the largest sites on the Internet over the past few years through Phusion Passenger, we have always remained vigilant as to not become complacent: we have been eagerly listening to the community as to what they expect the next big thing to be.

The observations we have made over the years have eventually culminated into Phusion Passenger 5, which was codenamed Raptor for a number of reasons.

A greater focus on performance and efficiency

Whether you are deploying a small web app on a VPS or spinning up tens of thousands of instances to power your e-commerce business, we all want the most bang for our buck. Being able to reduce the number of required servers would be beneficial in reducing costs and it is for this reason that developers seek to employ the most efficient software stack currently available. When it comes to making that choice, benchmarks from third parties often seem to play an important part in the decision making process. Even though they are convenient to consult, it is easy to overlook a couple of important things that we would like to underline.

When it comes to performance benchmarks for example, it does not always become clear how the results have been obtained and how they will affect the reader. This is mostly due to the fact that benchmarks are often performed on synthetic applications in synthetic environments that don’t take into consideration real world workloads and latencies. This often leads to skewed results when compared to real time workloads.

A good example of this is the “Hello World” benchmark, where people tend to benchmark app servers against a so-called “Hello World” application: an app that basically returns “Hello World”. Needless to say, this is hardly a real world application.

Anyone who has ever deployed a real world application will know that these kinds of benchmarks don’t really say anything useful as they effectively measure how fast an app server is at “doing nothing”.

In real world Ruby applications on the other hand, processing time quickly gets overtaken by the app itself, plus network overhead, rather than the app server: the differences in performance between the app servers basically become insignificant when compared to the time spent in the hosted app itself.

Despite this, benchmarks remain a popular source to consult when trying to figure out what the “best” software solution is. The danger here lies in the possibility that a developer might be tempted to base their decision solely on what bar chart sticks out the most, without so much as looking at what the other solutions all bring to the table.

There is a reason for example why Phusion Passenger — even though hot on the heels of its competitors — has not been leading these kinds of Hello World benchmarks in the past: we do much more than the competition does when it comes to ease of use, memory efficiency, security and features. All these things are not free, and this is basically what is being measured with said benchmarks.

When benchmarked against real world applications however, the differences in performance between app servers becomes almost indistinguishable. The focus should then be put on what feature-set is most suitable for your production app. We believe that on that front, Phusion Passenger is leading the pack.

We have tried many times to explain this in the comment sections of benchmarks, but have unfortunately had to infer that such explanations often fall on deaf ears. We think that’s a shame, but rather than continue to fight it, we have decided to try to beat our competitors on performance as well. This has led to a series of internal optimizations and innovations which we have documented in the Raptor articles. Not only do we believe we are now able to win these kinds of benchmarks, we believe we have been able to do so with mechanisms that are incredibly useful in real world scenarios too (e.g. Turbocaching).

A greater focus on showcasing the technology

Software that is easy to use runs the risk of being considered “boring” to hackers, or worse, “simple”. In the latter case, the underlying technology facilitating the ease of use gets taken granted for. Over the years, we felt this was happening to Phusion Passenger to the extent that we wanted to set the record straight.

A lot of thought went into using the right algorithms and applying the right optimizations to allow Phusion Passenger to do what it does best: being able to deliver an “upload-and-go” deployment experience second to none in a secure and performant manner is by no means a trivial task. We chose to abstract these implementation details however from the end-user as we wanted them to be able to focus more on their app and business rather than the nitty gritty when it came down to how “the soup was made”. Who cares right?

Well, as it turned out, a lot of hackers do. Articles about “Unicorn being Unix” sparked a lot of interest from hackers allowing it to quickly garner a following. We thought articles such as these were great, but felt somewhat disappointed that people seemed to forget that Phusion Passenger was already doing the majority of what was written in such articles a full year earlier. It then dawned on us that we were no longer being considered to be the new shiny thing, but rather considered being part of the establishment.

In hindsight, it was perhaps also an error of judgement of us to focus our marketing efforts mostly on businesses rather than the grassroots hacker community we originated from ourselves: they are not mutually exclusive and instead of mostly underlining the business advantages, we should have underlined the technological advantages much more as well.

In an effort to set this straight, we chose to document Raptor’s underlying technology in great detail which was incredibly well received by the Hacker News community. It was in fact on the front page for most of the day with close to 200 upvotes and sparked a lot of discussions on Twitter too.

Besides the new optimizations and features found in Raptor, a lot of technology discussed in those articles was already available in Phusion Passenger 4 and its precursors. If there is a lesson to take from all this, it is that marketing is indeed the art of repetition. And that it probably helps to present it as the new kid on the block to get rid of any preconceived notions/misconceptions people might have had about Phusion Passenger.

Smoke and mirrors

Whether or not people would be just as excited if they knew that it was Phusion Passenger 5 all along is perhaps another discussion to be had: some actually found out ahead of time due to the similar writing style of our tech articles and / or through nslookups (a fedora hat-tip goes out to you folks! May your sense of scrutiny live long and prosper!).

What we do however know is that our Raptor approach over the past month has produced more subscribers to our newsletter than we have been able to accomplish over the past 6 years through the Phusion Passenger moniker. We still have a hard time comprehending this, but there is no denying the numbers: we — the community — seem to like shiny new things.

Truth be told, we didn’t really bother trying to cover up the fact that it was in fact Phusion all along that was behind Raptor. We kind of left it as an exercise to the reader to figure this out amidst all the “hype” and claims of “vaporware” to see if people still remembered Phusion Passenger’s fortes.

We were not disappointed when it came to that and felt incredibly proud that a lot of people questioned why Phusion Passenger was not included within the Raptor benchmarks and requested it to be included. Needless to say, this was something we were unable to do because it already was included all along as Raptor itself 😉 We were also happy to see that some even pointed out that some of the features look like they came straight out of Phusion Passenger.

What’s in a name?

You might be wondering why we chose to market Phusion Passenger 5 under the Raptor code name and went through so many hoops in doing so. To quote Shakespeare: “Conceal me what I am, and be my aid, for a disguise as haply shall become, the form of my intent”.

With all the new improvements and features pertaining to performance and introspection, we felt Phusion Passenger deserved new consideration from its audience in an objective manner. To circumvent any preconceived notions and/or misconceptions people may have had about Phusion Passenger over the years, we decided to market it as Raptor. We felt the codename was particularly appropriate for our renewed commitment to performance.

Just to be clear, Phusion Passenger will not be renamed to Raptor. Raptor is just a codename that has served its purpose by the time of this writing for Phusion Passenger 5. We will drop this name starting from today: from now on, “Raptor” is Phusion Passenger 5.

With a little help from our friends

The success of the Raptor campaign would not have been possible without the help and support of our friends. In particular, we would like to thank Peter Cooper and Fabio Akita for their in-depth write-ups on Phusion Passenger 5 / Raptor and its precursors. Their articles carried the necessary weight to allow us to focus on explaining and improving the technology itself rather than having to spend time on trying to debunk “vaporware” claims. In a similar manner, we would also like to thank David Heinemeier Hansson for helping us out via his tweets and feedback.

Lastly, we would like to thank the community and our customers for being so supportive. At the end of the day, it is you folks who make this all possible and we can’t wait to show you what we have in store for the future.

Final words before upgrade instructions

Phusion Passenger’s core is open source. Learn more about Phusion Passenger or fork or watch us on Github. 🙂

If you would like to stay up to date with Phusion news, please fill in your name and email address below and sign up for our newsletter. We won’t spam you, we promise.



Changelog

5.0.0 beta 1 is a beta release. It may still have bugs and we do not recommend using it in production yet.

Version 5.0.0 beta 1 contains major changes. It is mostly compatible with version 4, but there are a few minor breakages, which are described below. Major changes and notable breakages are:

  • Performance has been much improved. This is thanks to months of optimization work. You can learn more at www.rubyraptor.org.
  • We have published a server optimization guide for those who are interested in tuning Phusion Passenger.
  • Support for Rails 1.2 – 2.2 has been removed, for performance reasons. Rails 2.3 is still supported.
  • Phusion Passenger now supports integrated HTTP caching, which we call turbocaching. If your app sets the right HTTP headers then Phusion Passenger can tremendously accelerate your app. It is enabled by default, but you can disable it with --disable-turbocaching (Standalone), PassengerTurbocaching off (Apache), or passenger_turbocaching off (Nginx).
  • Touching restart.txt will no longer restart your app immediately. This is because, for performance reasons, the stat throttle rate now defaults to 10. You can still get back the old behavior by setting PassengerStatThrottleRate 0 (Apache) or passenger_stat_throttle_rate 0 (Nginx), but this is not encouraged. Instead, we encourage you to use the passenger-config restart-app tool to initiate restarts, which has immediate effect.
  • Websockets are now properly disconnected on application restarts.
  • The Phusion Passenger log levels have been completely revamped. If you were setting a log level before (e.g. through passenger_log_level), please read the latest documentation to learn about the new log levels.
  • If you use out-of-band garbage collection, beware that the X-Passenger-Request-OOB-Work header has now been renamed to !~Request-OOB-Work.
  • When using Rack’s full socket hijacking, you must now output an HTTP status line.
  • [Nginx] The passenger_set_cgi_param option has been removed and replaced by passenger_set_header and passenger_env_var.
  • [Nginx] passenger_show_version_in_header is now only valid in the http context.
  • [Apache] The PassengerStatThrottleRate option is now global.

Minor changes:

  • The minimum required Nginx version is now 1.6.0.
  • The instance directory is now touched every hour instead of every 6 hours. This should hopefully prevent more problems with /tmp cleaner daemons.
  • Applications are not grouped not only on the application root path, but also on the environment. For example, this allows you to run the same app in both production and staging mode, with only a single directory, without further configuration. Closes GH-664.
  • The passenger_temp_dir option (Nginx) and the PassengerTempDir option (Apache) have been replaced by two config options. On Nginx they are passenger_instance_registry_dir and passenger_data_buffer_dir. On Apache they are PassengerInstanceRegistryDir and PassengerDataBufferDir. On Apache, PassengerUploadBufferDir has been replaced by PassengerDataBufferDir.
  • Command line tools no longer respect the PASSENGER_TEMP_DIR environment variable. Use PASSENGER_INSTANCE_REGISTRY_DIR instead.
  • passenger-status --show=requests has been deprecated in favor of passenger-status --show=connections.
  • Using the SIGUSR1 signal to restart a Ruby app without dropping connections, is no longer supported. Instead, use passenger-config detach-process.
  • Introduced the passenger-config reopen-logs command, which instructs all Phusion Passenger agent processes to reopen their log files. You should call this after having rotated the web server logs.
  • [Standalone] The Phusion Passenger Standalone config template has changed. Users are encouraged to update it.
  • [Standalone] passenger-standalone.json has been renamed to Passengerfile.json.
  • [Standalone] passenger-standalone.json/Passengerfile.json no longer overrides command line options. Instead, command line options now have the highest priority.

Installing or upgrading

Here’s a quickstart:

gem install passenger --pre -v 5.0.0.beta1
cd /path-to-your-app
passenger start

The above is a very short excerpt of how to install or upgrade Phusion Passenger. For detailed instructions (which, for example, take users and permissions into account), please refer to the “RubyGems” section of the installation manuals:

Please note:

  • 5.0.0 beta 1 is a beta release. It may still have bugs and we do not recommend using it in production yet.
  • There are no Homebrew or Debian packages for this release, because this release is still in beta!
  • There is also a 5.0.0 beta 1 release of Phusion Passenger Enterprise available. Please refer to the Customer Area.
  • EagleFlight

    No way I’m using this if it’s not called Raptor.

  • Dave Aronson

    Awww, the Codosaurus likes Raptor….

  • Haha, brilliant 🙂
    Maybe now is a good a time as any to rebrand as Raptor tho :p

  • buger

    Does it support EventMachine based servers like Goliath?

  • Devran (Cosmo) Ünal

    I’m with @EagleFlight. Call it Raptor.

  • Devran (Cosmo) Ünal

    +1

  • NathanB

    +1000

  • Khadzhinov

    Could not start the server engine:

    [ 2014-11-25 20:26:22.7033 9036/0x7fff7227d300 agents/Watchdog/Main.cpp:1220 ]: Starting PassengerAgent watchdog…

    [ 2014-11-25 20:26:22.7221 9038/0x7fff7227d300 agents/HelperAgent/Main.cpp:881 ]: Starting PassengerAgent server…

    [ 2014-11-25 20:26:22.7223 9038/0x7fff7227d300 agents/HelperAgent/Main.cpp:254 ]: PassengerAgent server running in single-application mode.

    [ 2014-11-25 20:26:22.7223 9038/0x7fff7227d300 agents/HelperAgent/Main.cpp:255 ]: Serving app : /Users/khadzhinov/RoR/centask-centask-officer.main

    [ 2014-11-25 20:26:22.7223 9038/0x7fff7227d300 agents/HelperAgent/Main.cpp:256 ]: App type : rack

    [ 2014-11-25 20:26:22.7223 9038/0x7fff7227d300 agents/HelperAgent/Main.cpp:257 ]: App startup file: config.ru

    [ 2014-11-25 20:26:22.7225 9038/0x7fff7227d300 agents/HelperAgent/Main.cpp:900 ]: ERROR: Cannot bind a TCP socket on address ‘0.0.0.0’ port 3000: Address already in use (errno=48)

    in ‘int Passenger::createServer(const Passenger::StaticString &, unsigned int, bool)’ (IOUtils.cpp:237)

    in ‘void startListening()’ (Main.cpp:271)

    in ‘int runServer()’ (Main.cpp:884)

    [ 2014-11-25 20:26:22.7232 9037/0x7fff7227d300 agents/Watchdog/Main.cpp:1141 ]: ERROR: Unable to start the Phusion Passenger helper agent: it seems to have crashed during startup for an unknown reason, with exit code 1

    in ‘void startAgents(const WorkingObjectsPtr &, vector &)’ (Main.cpp:1126)

    in ‘int watchdogMain(int, char **)’ (Main.cpp:1260)

    [ 2014-11-25 20:26:23.7284 9037/0x7fff7227d300 agents/Watchdog/Main.cpp:345 ]: Cannot open cleanup PID file /var/folders/8b/vdbfyn957zn3kys74gqq4y040000gn/T/passenger-standalone.1msfo3n/temp_dir_toucher.pid

    Stopping web server… done

  • Could you report problems to the discussion forum?

  • Wow, this was such a clever approach. I definitely needed this to change my perception of Passenger. When I read about Raptor, and how it compared to Unicorn and Puma, I thought it sounded awesome. I’m going to have to take a second look at using Passenger.

  • stephanromanov

    I remember reading that nginx was not needed in Raptor and Passenger uses it. It’s weird to see Raptor is now a Phusion product, was it acquired? I see every Raptor’s page now updated as Phusion’s. Even the 1st benchmark blog post has an update with the name changed. I’m confused.

  • Raptor was not acquired. We have been behind Raptor all along. From the beginning, it was Phusion Passenger 5.

    Phusion Passenger does not *need* Nginx. You may *optionally* use Nginx, but Phusion Passenger has had a standalone mode for over 2 years now. Hence the reason we launched the Raptor campaign: to change old, outdated perspectives on Phusion Passenger.

  • Einstein

    I’ve been using Passenger (mainly) ever since it replaced my mongrels and I have to say that it is a pleasure to admin, unlike that certain magical coned-horse thing. That’s not to put down that coned-horse thing at all, it works great for some apps but for others it is a royal pain in the rear-end. I don’t like to waste time on things that I don’t have to for a negligible performance gain, if any.

    I am completely baffled why anyone would want to add W to X to Y to run and maintain Z (introduce multiple pieces of software to their web stack) when they can just have A,Z to begin with.. please don’t tell me it’s because of the performance gains W,X,Y,Z has over A,Z.. have you even benchmarked your application to see if that is true in your case? I don’t mean run abs/httperf against a static page either, I mean really benchmark it and provide yourself with simulated user behavior. Probably not. I know, don’t be a $&*# I too am guilty of this at times!

    I just wanted to say thanks for helping me sleep at night and I look forward to Passenger 5 being production ready.

    Keep up the great work!

  • Guest

    Props on the marketing gag — very well done. But without rolling restarts, even the fastest server performs like a free Heroku dyno every time you deploy a rails app. Unfortunately, that feature is still not available with Passenger’s free version, and to buy it can cost nearly as much as the VM it’s running on. Meanwhile, the whole concept of per-server licensing appears to be going the way of the raptor. Because Docker.

    I agree with the others — this is a great opportunity to rebrand your product. And the per-server license made a lot of sense when deploying several apps on one server, which Passenger does better than anyone. We could get economies of scale that way. But soon all of this will be containerized and modularized. So unless we can dynamically scale Passenger licenses at a reasonable price, the current license model seems at odds with the coming infrastructure model.

  • Pranas

    But even in the standalone mode Passenger still runs nginx, doesn’t it? The old Passenger used to install it’s runtime first time you start it. Did this change? I never tried it, but is this first run installation work well on Heroku?

  • In version 4, Nginx was used to implement the standalone mode. In version 5, this is no longer true: Phusion Passenger uses its own standalone HTTP server by default. But in version 5 it is still possible to use the Nginx engine if the user so wishes, by setting a configuration option.

    Both version 4 and version 5 work great on Heroku: https://github.com/phusion/passenger-ruby-heroku-demo

  • We’ve already solved those issues. In case of Docker, the licensing is per Docker host, not per container. This should be reasonable. We also have special cloud licenses (pay-as-you-go) which are available on request.

  • It’s possible to run an EventMachine reactor inside a Phusion Passenger-served app.

  • Pranas

    Even though you claim nginx is no longer necessary and off by default it is still installed on the first run. Why? Also the binary download and native support compilation on the first run feels wrong. Can’t you make Passenger install whatever it needs at the gem install phase? Call me paranoid but it just feels brittle. I wish it worked more like Unicorn or Puma regarding this aspect.

  • > Even though you claim nginx is no longer necessary and off by default it is still installed on the first run. Why?

    Because I was too lazy and I didn’t write the code to skip installing Nginx when not necessary. Seeing that I had a deadline and that I had more important things to worry about, it was just easier to *always* install Nginx even when not necessary.

    > Can’t you make Passenger install whatever it needs at the gem install phase?

    It’s certainly possible. But again, I haven’t done that (yet) because I had a deadline. It’s certainly something that can be changed in the future.

  • Pranas Kiziela

    Haha, okay 🙂 I know tackling architecture and performance issues is way more fun than doing the simple stuff. However you should put it on your roadmap. I think it will make Passenger more user friendly. By the way, I switched to Passenger 5 on one of our yet-to-be-released apps. I had some minor issues, but they are already reported on the bug tracker. Thanks for a great read and your contributions to OSS.

  • notinittowinit

    But then it loses its rails connection.. “passenger” “rails” get it? What does raptor go with? Dinosaurs? Prehistoric?

  • nv

    I had a software consulting from ClarityEd http://clarityed.com/ in his topic and they are very good in this field. They are also good in SAT Test Prep http://clarityed.com/test-prep/sat/