Phusion Blog

On Unix systems, compiling software to install it is often the norm. However compiling takes CPU time and memory, and on some systems it is preferable not to have a compiler installed for security reasons. Especially compiling C++ code is memory-intensive, and could require as much as 700 MB of RAM. Since Phusion Passenger is written largely in C++, the need to compile Phusion Passenger could be problematic for some users, e.g. users on small VPSes with little memory.

Another problem is the need to compile Nginx. Unlike Apache, Nginx does not support dynamically loadable modules. So to add Phusion Passenger support, Nginx must be recompiled. Some users do not like doing this.

Today’s solutions

There are some solutions for this problem that you can already use with the current version of Phusion Passenger:

1. You can use our Debian and Ubuntu packages, kindly contributed by John Leach from Brightbox. John Leach has been helping us with Debian packaging for years now, and he often updates his packages within a few days after an official release. Packages for both the Apache mode and the Nginx mode of Phusion Passenger are available.
   

   If you didn’t already know about this, you should. Strongly recommended. It solves nearly every problem users have with installing Phusion Passenger. Just run apt-get install libapache2-mod-passenger or apt-get install nginx-full, and you’re done. The provided Nginx package is the exact same as the one in Debian/Ubuntu, but with Phusion Passenger support added. This means you get to keep your /etc/init.d/nginx init script and other nice things.

   There are just two drawbacks at the moment: Phusion Passenger Standalone (described later in this article) is not packaged, and Phusion Passenger Enterprise is not packaged.

2. You can precompile Phusion Passenger on one of your machines, and copy over the binaries to another machine.
3. Users who do not mind compiling Phusion Passenger but do mind compiling Nginx, can use a reverse proxy setup with Phusion Passenger Standalone.

Tomorrow’s solution: binaries for Phusion Passenger Standalone

While the current Debian packages are great, we haven’t been standing still. There is still room for improvement. One of the things that we’ve been working on is a generic binary building infrastructure for Phusion Passenger. The end goal is to eliminate the need to compile Phusion Passenger, for all users that are using Linux on the x86 and x86_64 architectures, regardless of the distribution that they’re using.

This infrastructure automatically builds generic, multi-distribution-compatible Linux binaries for x86 and x86_64 every time we push commits to our Git repository. The Phusion Passenger installers can then just download binaries from the Phusion server instead of compiling them from source. The installers automatically check whether the downloaded binaries work on the current system, and if not, falls back to compiling. For 95%-99% of the users, this system effectively eliminates the need to compile Phusion Passenger. Furthermore, this infrastructure compiles binaries for both the open source variant and the Enterprise variant.

Rome was not built in one day, so we’re tackling this problem iteratively. Starting from the next version of Phusion Passenger, these automatically built binaries will be used in Phusion Passenger Standalone.

What is Phusion Passenger Standalone, and how is it related to compiling?

For those of you who are not familiar with Phusion Passenger Standalone: it is a third mode of Phusion Passenger, alongside with the Apache mode and the Nginx mode, introduced in 2010. In Standalone mode, Phusion Passenger does not require an external web server, and can be started with just a single command: passenger start. This command starts a fully-featured Phusion Passenger instance. In fact, it is powered by an Nginx core, making it very fast and lightweight, and extremely good at serving static assets. You can even expose Phusion Passenger Standalone immediately on the public Internet without using a reverse proxy, something e.g. Unicorn cannot safely do (“Slow Clients Are Problematic”). Therefore, Phusion Passenger Standalone is the easiest way to get a Ruby web app up and running with minimal hassle and with great performance. It can also be installed in a reverse proxy setup behind Nginx, for those who do not want to or cannot compile Nginx.

The following video, originally created in 2010, demonstrates Phusion Passenger Standalone. Note that back then it was still called “Phusion Passenger Lite”. As you can see, Phusion Passenger Standalone compiles its own runtime when it is run for the first time. Compiling can take a few minutes. Starting from the next version, starting Phusion Passenger Standalone for the first time takes a few seconds.

Day after tomorrow’s solutions: fully-automated binary building and package building

We have two goals for the futher future:

1. Automatically download binaries when installing Phusion Passenger for Apache and Phusion Passenger for Nginx.
2. To fully automate the building of Debian packages. Although the generic binaries should work for most of our users, installing Debian packages is easier still, and integrates better with the OS. We’re working on a system that will publish 100% working Debian packages 15 minutes after we publish the source code for a new release, so that no human intervention is required to publish Debian packages. This will greatly increase ease of installation and upgrade for users. There will also be automatically built Debian packages for Enterprise customers.

   We’ve already begun working on integrating Debian packaging scripts into our main Git repository. We’ve also written an automated set of tests for checking whether packages built with these scripts work correctly. These tests are run by our continuous integration server on every commit, so you could say that we’re employing “continuous packaging”.

Conclusion

We’re working hard on making Phusion Passenger as easy to install as possible. We take usability very seriously, and we invite everyone to review our generic binary building system, our Debian packaging scripts, as well as our packaging tests. All code is open source and available on Github.

If you would like to stay up to date with Phusion news, please fill in your name and email address below and sign up for our newsletter. We won’t spam you, we promise.

Phusion Passenger is software that deploys Ruby and Python web apps, by integrating into Apache and Nginx and turning them into a fully-featured application server. It is very fast, stable and robust and thus used by the likes of New York Times, AirBnB, Symantec, Pixar, etc. It comes with many features that make your life easier and your application perform better.

“Flying Passenger” is a technology which allows one to decouple Phusion Passenger’s life time from the web server’s life time, so that the web server can be independently restarted from Phusion Passenger, and from any of the application processes served by Phusion Passenger.

Normally, Phusion Passenger starts together with the web server, and shuts down together with the web server. The advantages of this default behavior is that it makes Phusion Passenger easy to administer: one only has to deal with the web server process and can expect all relevant processes to be cleaned up after a web server shut down. However this also brings about a disadvantage: every time one restarts the web server (e.g. to make a minor configuration change), Phusion Passenger and all its application processes also get restarted.

This problem is solved by “Flying Passenger”, which is an advanced mode of operation in Phusion Passenger that allows the web server to be indepedently restarted from Phusion Passenger. When this mode is enabled:

• One must start Phusion Passenger separately from the web server, namely by starting the Flying Passenger daemon. This daemon must — to an extent — be separately configured and managed from the web server.
• The web server must be configured to forward requests to the Flying Passenger daemon.
• You should beware of the caveats and limitations, which will be described in this article.

Flying Passenger is fully documented in the Phusion Passenger manual. This article serves as an introduction.

Differences with existing reverse proxy setups

You may be wondering how this differs from existing reverse proxy setups with other application servers, e.g. Nginx+(G)Unicorn, Nginx+Puma, or Nginx+uWSGI. Most other application servers can serve only one application per instance. Flying Passenger works just like Phusion Passenger for Nginx: it can serve multiple applications. Almost all Phusion Passenger for Nginx features are preserved in the Flying Passenger mode: dynamic process scaling, process supervision, user switching, real-time response buffering, etc all work as one is used to in Phusion Passenger for Nginx. However, Flying Passenger is only partially configured through the Nginx configuration file; some configuration is set through command line options.

Python users may see parallels between Flying Passenger’s mode of operation and uWSGI Emperor’s.

Availability

Flying Passenger will become available in the Enterprise variant of the next Phusion Passenger release, namely version 4.0.6.

At this time, this feature is only available in Phusion Passenger for Nginx. You must have Phusion Passenger for Nginx properly installed. Flying Passenger is not available in Phusion Passenger for Apache, and also not in Phusion Passenger Standalone.

Basic usage

Start the Flying Passenger daemon by invoking the flying-passenger command. The only required option is --socket-file. Depending on whether you wish to enable User Switching, you have to start flying-passenger with root privileges or not.

$ sudo flying-passenger --socket-file=/var/run/flying-passenger.sock
I, [2013-06-14T09:10:13.095339 #77179]  INFO -- : Welcome to Flying Passenger 4.1.0
I, [2013-06-14T09:10:13.095339 #77179]  INFO -- : Starting PassengerWatchdog...
I, [2013-06-14T09:10:13.097036 #77179]  INFO -- : PassengerWatchdog started on PID 77181
...
I, [2013-06-14T09:10:13.129017 #77179]  INFO -- : PassengerWatchdog initialized properly
I, [2013-06-14T09:10:13.129127 #77179]  INFO -- : Flying Passenger up and listening on /var/run/flying-passenger.sock!

Now configure Phusion Passenger for Nginx to make use of the Flying Passenger daemon, by setting the passenger_fly_with option to the socket filename:

http {
    ...
    passenger_fly_with /var/run/flying-passenger.sock;
    ...
}

After (re)starting Nginx, Nginx + Flying Passenger is fully operational:

$ sudo /path-to/nginx

You can test it by adding a virtual host for a web app:

http {
    ...

    server {
        listen 80;
        server_name www.foo.local;
        root /webapps/foo/public;
        passenger_enabled on;
    }
}

Verify that it works by making an HTTP request to it:

$ curl http://www.foo.local/

Now let’s verify that restarting the web server does not restart the just-spawned application process. Run passenger-status to obtain the PID of the application process:

$ sudo passenger-status
Version: 4.1.0
Date   : 2013-06-14 09:21:51 -0400
----------- General information -----------
Max pool size : 6
Processes     : 1
Requests in top-level queue : 0

----------- Application groups -----------
/webapps/foo#default:
  App root: /webapps/foo
  Requests in queue: 0
  * PID: 77283   Sessions: 0       Processed: 1       Uptime: 2s
    CPU: 1%      Memory  : 8M      Last used: 2s ago

As you can see, the PID of the application process is 77283. Now let’s see what happens if we restart Nginx:

$ sudo /path-to/nginx -s stop
$ sudo /path-to/nginx
$ sudo passenger-status

The application process should remain there, unchanged:

$ sudo passenger-status
Version: 4.1.0
Date   : 2013-06-14 09:21:51 -0400
----------- General information -----------
Max pool size : 6
Processes     : 1
Requests in top-level queue : 0

----------- Application groups -----------
/webapps/foo#default:
  App root: /webapps/foo
  Requests in queue: 0
  * PID: 77283   Sessions: 0       Processed: 1       Uptime: 18s
    CPU: 1%      Memory  : 8M      Last used: 18s ago

Configuring Flying Passenger

Flying Passenger gets some configuration from the web server, but not all. In particular, most web server directives that are only valid in the http context, e.g. passenger_log_level, have no effect when using Flying Passenger. Instead, you are supposed to pass these configuration directives through command line options to the Flying Passenger daemon. Configuration directives that have no effect on Flying Passenger are documented as such. You can assume that configuration directives that are not documented as such, work fine on Flying Passenger.

For example, to achieve the same effect as setting passenger_log_level to 2, run the Flying Passenger daemon as follows:

$ sudo flying-passenger --socket-file=/var/run/flying-passenger.sock --log-level=2

Currently, not all configuration directives have a Flying Passenger equivalent. Run the following command to see an overview of available options:

$ flying-passenger --help

Managing the Flying Passenger daemon

The Flying Passenger daemon runs in the foreground by default. This is undesirable on server environments. You can make it go into the background by passing --daemonize, --log-file and --pid-file:

$ sudo flying-passenger --socket-file=/var/run/flying-passenger.sock \
    --daemonize --log-file=/var/log/flying-passenger.log \
    --pid-file=/var/run/flying-passenger.pid

You can shut down a Flying Passenger daemon by sending SIGINT or SIGTERM to it:

$ kill `cat /var/run/flying-passenger.pid`

We recommend using daemontools or runit for managing the Flying Passenger daemon. These tools will automatically start the Flying Passenger daemon at boot, and will automatically restart the daemon if it crashes. You can create and enable a daemontools/runit service as folows:

$ sudo mkdir /etc/service/flying-passenger
$ sudo tee /etc/service/flying-passenger/run <<EOF
#!/bin/sh
exec /path-to/flying-passenger \
    --socket-file=/var/run/flying-passenger.sock \
    --log-file=/var/log/flying-passenger.log \
    --pid-file=/var/run/flying-passenger.pid
EOF

Immediately after creating the run file, daemontools/runit automatically runs it to start the daemon. Note that the location (/etc/service) depends on the OS or Linux distros. Sometimes it’s /service. Also note that we start the Flying Passenger daemon without --daemonize.

To shut down a daemontools/runit-managed daemon, you need to use svc -d /etc/service/flying-passenger (daemontools) or sv stop /etc/service/flying-passenger (runit) instead of sending a signal to the process.

Caveats and limitations

Beware of the following caveats and limitations when using Flying Passenger:

• The Nginx executable must be compiled with the same version of Phusion Passenger as the Flying Passenger daemon. Failing to meet this requirement may result in cryptic errors, or may result in certain features not working, until you’ve fixed the situation. When upgrading Phusion Passenger, you must restart both Nginx and the Flying Passenger daemon.
• The passenger_root directive has no effect. When using Flying Passenger, you are not supposed to set passenger_root.
• When you add a new application to the web server configuration, Flying Passenger will automatically pick up the application’s settings and spawn this new application upon the first request to it. However it is not capable of automatically starting the new app before a request has been sent to it (i.e. passenger_pre_start-like behavior is not available in this case). As a workaround, you can send an HTTP request to your application after starting the daemon, which forces it to spawn application processes.
• When you remove an application from the web server configuration, Flying Passenger will not detect the removal and will not shut down the associated application processes. Killing the application processes will also not help, because Flying Passenger will restart them per the (now-removed, but still in the Flying Passenger daemon’s memory) passenger_min_instances settings. At the moment, there are two ways to get rid of those processes:
  • Before removing the application from the web server configuration, explicitly set its passenger_min_instances to 0. Next, send a request to it, which will cause the Flying Passenger daemon to take over the new passenger_min_instances 0 option. You can then proceed with removing the application from the web server configuration, and restarting the web server. Finally, kill the PIDs associated to those application processes and remove the application configuration.
  • Restart the Flying Passenger daemon.

Conclusion

Flying Passenger should especially be useful for those who have to modify Nginx configuration on a regular basis, without modifying application configuration. Flying Passenger is still a young feature, so we encourage users to provide feedback and bug reports.

We at Phusion are continuously developing and improving Phusion Passenger. If you would like to stay up to date with Phusion news, please fill in your name and email address below and sign up for our newsletter. We won’t spam you, we promise.

Phusion Passenger turns Apache and Nginx into a full-featured application server for Ruby and Python web apps. It has a strong focus on ease of use, stability and performance. Phusion Passenger is built on top of tried-and-true, battle-hardened Unix technologies, yet at the same time introduces innovations not found in most traditional Unix servers. Since mid-2012, it aims to be the ultimate polyglot application server.

Today we are pleased to announce Release Candidate 6 of Phusion Passenger 4.0. The 4.x series is a huge improvement over the 3.x series: during the development of 4.0, we’ve introduced a myriad of changes which we’ve covered in past beta preview articles:

• Support for multiple Ruby versions, support for Python WSGI, multithreading, evented core similar to Nginx and Node.js, real-time response buffering, improved zero-copy architecture, better error diagnostics.
• JRuby and Rubinius support.
• Out-of-Band Work.
• Support for the Rack socket hijacking API
• Improved stability and test suite.

Release Candidate 5 was a private interim release for Phusion Passenger Enterprise customers only.

Changes in 4.0 RC 5 and RC 6

The most important changes in RC 5 and RC 6 are as follows:

• The default config snippet for Apache has changed! It must now contain a
  PassengerDefaultRuby option. The installer has been updated to output
  this option. The PassengerRuby option still exists, but it’s only used
  for configuring different Ruby interpreters in different contexts. Please
  refer to the manual for more information.
• We now provide GPG digital signatures for all file releases by Phusion.
  More information can be found in the manual.
• WebSocket support on Nginx. Requires Nginx >= 1.3.15.
• passenger-status now displays process memory usage and time when it
  was last used. The latter fixes issue #853.
• Exceptions in Rack application objects are now caught to prevent
  application processes from exiting.
• The passenger-config tool now supports the --ruby-command argument,
  which helps the user with figuring out the correct Ruby command to use
  in case s/he wants to use multiple Ruby interpreters. The manual has
  also been updated to mention this tool.
• Fixed streaming responses on Apache.
• Worked around an OS X Unix domain socket bug. Fixes issue #854.
• Out-of-Band Garbage Collection now works properly when the application
  has disabled garbage collection. Fixes issue #859.
• Fixed support for /usr/bin/python on OS X. Fixes issue #855.
• Fixed looping-without-sleeping in the ApplicationPool garbage collector
  if PassengerPoolIdleTime is set to 0. Fixes issue #858.
• Fixed some process memory usage measurement bugs.
• Fixed process memory usage measurement on NetBSD. Fixes issue #736.
• Fixed a file descriptor leak in the Out-of-Band Work feature. Fixes issue #864.
• The PassengerPreStart helper script now uses the default Ruby
  interpreter specified in the web server configuration, and no longer
  requires a ruby command to be in $PATH.
• Updated preferred PCRE version to 8.32.
• Worked around some RVM bugs and generally improved RVM support.
• The ngx_http_stub_status_module is now enabled by default.
• Performance optimizations.

Installing and testing 4.0.0 Release Candidate 6

Quick install/upgrade

Phusion Passenger Enterprise users can download the Enterprise version of 4.0 RC 6 from the Customer Area.

Open source users can install the open source version of 4.0 RC 6 with the following commands:

gem install passenger --pre
passenger-install-apache2-module
passenger-install-nginx-module

You can also download the tarball at Google Code.

In-depth instructions

In-depth installation and upgrade instructions can be found in the Installation section of the documentation. The documentation has been updated to cover 4.0 changes, including Enterprise features. You can view them online here:

• Apache manual
• Nginx manual

Final

We are excited about the final release. You can help us by testing RC 6 and reporting any bugs. Please submit bug reports to our bug tracker.