Phusion white papers Phusion overview

Phusion Blog

Phusion Passenger 2.2.3 released – Bug Fix Edition

By Hongli Lai on June 17th, 2009

Phusion Passenger is an Apache and Nginx module for deploying Ruby on Rails web applications, and is mainly focused on ease of use and stability.

Announcing the “Bug Fix Edition”

Phusion Passenger has gained a lot of features lately, with the most prominent one being Nginx support. A lot of software become more bloated and more buggy over time, and their documentation become outdated. But we at Phusion strive to maintain a high level of quality. Thus, after the release of version 2.2.2, we decided to focus our efforts on polishing Phusion Passenger, which means:

  • Fixing as many outstanding bugs as possible, instead of introducing new major features.
  • When introducing new features, only introduce those for which there is a lot of demand and that do not affect stability too much.
  • Spending time on improving the documentation and keeping it up-to-date.
  • Improving error messages. Nothing is more frustrating than seeing that something went wrong, but not knowing that it exactly is.
  • Improving overall usability. Even though Phusion Passenger is server software and our user base is generally knowledgeable, we still think usability is important and we take it very seriously.

The result is version 2.2.3, which we informally call the “Bug Fix Edition”. Over the course of two months, we’ve committed more than 80 change sets. Approximately 7500 lines of code has been changed, and approximately 45 bugs have been fixed.

The summary of changes – which we’ve attempted to sort based on importance – are as follows. Unless marked otherwise, a change affects both Apache and Nginx.

Fixed sub-URI deployments for Rails 2.2+ apps
Sub-URI deployments has been broken on Rails 2.2+ apps for some time now. This is because Rails 2.2 changed the way the sub-URI deployments are handled for the sake of thread-safety. Sub-URI deployments on Rails 2.2 was no longer automatic and required one to change config.action_controller.relative_url_root in environment.rb. We’ve fixed this issue so that sub-URI deployment is once again automatic, without the need to set config.action_controller.relative_url_root manually. Fixes issue #169.
Fixed many compilation problems
The most prominent fix affects GCC 4.4 support. GCC is included in e.g. Fedora 11.
Compilation problems on OpenBSD, Solaris and OpenSolaris have also been fixed.
On OpenBSD, please note however that Phusion Passenger for Apache doesn’t work because of severe limitations in their pthreads implementation. The Nginx version works fine, so we recommend OpenBSD users to use Phusion Passenger for Nginx.
[Apache] Ability to pass environment variables via mod_env directives
If you want to pass environment variables to your applications, then you no longer need to do that via a Ruby wrapper script. Instead, you can just use mod_env directives, just like you intuitively would do:

<VirtualHost *:80>
   ServerName www.foo.com
   DocumentRoot /rails_apps/foo/public
   SetEnv PATH /usr/bin:/opt/ffmpeg/bin:/usr/local/bin
</VirtualHost>

You can then access PATH in your application via ENV['PATH'].

Please note however that:

  • Environment variables are no longer passed through request headers. Previously you would be able to access the PATH value set by mod_env from a controller through request.env['PATH']. You can no longer do that now. Access ENV['PATH'] instead.
  • Environment variables are set after the Ruby interpreter is started, so you cannot set environment variables that are supposed to affect the Ruby interpreter itself. For example, the Ruby Enterprise Edition garbage collector tuning environment variables cannot be set through SetEnv directives. You still need to use a wrapper script for that.
[Apache] Fixed many unintended “Broken pipe” I/O problems
Some people experience a lot of “Broken pipe” I/O problems. This manifests itself on the browsers’ side as a prematurely terminated connection, i.e. the web server closes the connection before the page was fully loaded. We’ve worked extensively with one of our clients to solve this problem, and it was determined that this problem was for a large part caused by bugs in Apache’s I/O routines. We’ve fixed many of these “Broken pipe” problems by rewriting our code to use our own I/O routines instead of Apache’s.

If you’re still experiencing “Broken pipe” problems, then please consider switching to Nginx. Some of these “Broken pipe” problems seem to be caused by bugs in the Apache I/O routines which we are not able to override. It has been reported that 99% of the broken pipe problems went away after switching to Nginx.

Please also note that such “Broken pipe” problems may also be caused by other malfunctioning parts, such as broken routers.

[Nginx] Fixed several Nginx restarting-related bugs
These bugs could cause Phusion Passenger to malfunction after restarting Nginx gracefully with SIGHUP.
[Nginx] If passenger_root is not set, then Nginx will disable Phusion Passenger instead of exiting with an error
This makes it possible to distribute Phusion Passenger-enabled Nginx binaries to users who may not necessarily use Phusion Passenger. Fixes issue #286.
[Nginx] Fixed page cache file serving.
The content type wasn’t being set correctly, so that all page cache files were served as application/octet-stream. Fixes issue #261.
[Nginx] Fixed Nginx 0.7 compatibility
Page caching was broken on Nginx 0.7, for a reason other than the aforementioned.
[Nginx] STDOUT and STDERR are now correctly redirected to the Nginx log file; fixes “Input/Output Error” (EIO) problems
Before, STDOUT and STDERR wouldn’t always be redirected to the Nginx log file, depending on the way Nginx is started. If it didn’t happen, and STDOUT and STDERR still point to the console, and the user disconnects from the console, then all data written to STDOUT and STDERR is silently discarded on most operating systems. This is obviously undesirable because it would mean that error messages are silently discarded. But on Linux, writing to a disconnected console like this would generate “Input/Output Error” (EIO) messages. Fixes issue #307.
Fixed displaying of information written to STDERR by the application during startup.
Such information was being silently discarded, which caused a lot of confusion for some people.
Fixed restarting of the ApplicationSpawner server
If the ApplicationSpawner server was still running, but no Rails processes are running, then touching restart.txt would not result in the ApplicationSpawner being restarted. Fixes issue #300.
[Apache] Improved file upload error messages
Apache’s file upload I/O routines only tell us that something went wrong, but not what went wrong. We rewrote these routines with our own so that we can now tell the administrator the reason why a file upload failed.
[Apache] Improved I/O error messages
Phusion Passenger will now try harder to come up with a descriptive error message, as well as with some suggestions.
Log application startup errors to the web server log file
Before, application startup errors are only shown in the web browser. This poses a problem for people who replace the default Phusion Passenger error page templates: there would be no way to see the error. Fixes issue #210.
Fixed file upload support on Ruby 1.9
It wasn’t setting encoding properties properly.
[Nginx] Fixed some client connection problems by increasing the HelperServer’s connection backlog to the maximum possible value
If you see errors in your Nginx log file along the lines of “helper_server.sock: connect(): Resource temporarily unavailable” or “helper_server.sock: connect(): In progress”, then this probably indicates that the HelperServer’s connection backlog is full, in which case this fix might help.

If it doesn’t help, then you should also increase:

  • your system’s default maximum allowable backlog size. On Linux, you can check the current size with the following command:
    cat /proc/sys/net/core/somaxconn

    Nginx’s backlog size defaults to 511, but Linux’s maximum allowable backlog size defaults to 128. By writing ‘512’ to this file you can increase it.

  • your system’s default TCP receive buffer size. Please consult http://www.psc.edu/networking/projects/tcptune/ for information.
[Apache] TMPDIR is no longer set automatically
This fixes compatibility with PHP sessions, among other things.
[Nginx] Fixed WSGI support
It was looking for passenger_wsgi.py in the wrong directory.
passenger-status no longer shows backtraces by default
We used to show backtraces for debugging purposes, but many people confused these with errors, so now we hide the backtraces in passenger-status by default.
Various Rack specification compliance fixes.
Fixed a potential security problem in the random number generator
The random number generator is now reseeded in every process that’s spawned off by Phusion Passenger, so that the child processes do not generate the same random number sequences. Fixes issue #118.
[Apache] New configuration option: PassengerUploadBufferDir
This option allows you to configure the directory in which upload buffer files are stored, in case you do not want them to be stored in /tmp for whatever reason.
[Apache] Fixed “Socket not connection” (ECONN) errors
These errors are harmless so we silently ignore them now. Fixes issue #158
[Nginx] passenger-install-nginx-module: do not crash if unable to write to the Passenger root directory
It will now report a proper error message, as it should. Fixes issue #287.
[Nginx] passenger-install-nginx-module now downloads Nginx 0.6.37 by default
We don’t download 0.7 by default yet until we’ve confirmed that it’s stable.
Improved test coverage
We’ve spent some time to increase the test coverage in our code base in order to maintain the current level of quality.
Improved documentation
The following chapters in the Apache users guide have been updated or added:

How do I upgrade to 2.2.3?

Via a gem

Please install it with the following command:

gem install passenger

Next, run:

passenger-install-apache2-module

Or, if you’re an Nginx user:

passenger-install-nginx-module

Please don’t forget to copy & paste the Apache/Nginx config snippet that the installer gives you.

(Note that RubyForge might still be distributing the files to their mirrors, so you might temporarily get “404 Not Found” errors until all mirrors are updated.)

Via a native Linux package

John Leach from Brightbox has kindly provided an Ubuntu 8.04 package for Phusion Passenger. The package is available from the Brightbox repository which you can find at:

http://apt.brightbox.net

Add the following line to the Third Party Software Sources:

deb http://apt.brightbox.net hardy main

(The simplest way to do that is to create a file in /etc/apt/sources.list.d/ containing the deb instruction, and then run ‘apt-get update’).

Once you’ve done this then you can install Phusion Passenger by running:

sudo apt-get install libapache2-mod-passenger

-or-

sudo apt-get install nginx-brightbox

(Note that John is currently packaging 2.2.3, so it might take a while before this release shows up in the apt repository.)

Final

Phusion Passenger is provided to the community for free. We’ve put a lot of effort into this release. If you like Phusion Passenger, please consider sending us a donation. Thank you!

Hongli Lai Ninh Bui
  • Weyert

    Thanks! I will update my servers to this new version. Keeping my fingers crossed 😉

  • lolcatz!

    Update was successful, no issues occured.

    Thanks guys!

  • Sean

    Has this corrected the problem with the 404 Not Found issues that appeared with Passenger 2.2x when using a symbolic link to the document root?

    Example in Apache config:
    DocumentRoot /home/user/domains/domain.com/public_html

    Whereas the actual Rails app is stored:
    /home/user/domains/domain.com/public_html -> /home/user/apps/app1/public

    This has always worked in Passenger 2.1x, but not in Passenger 2.2x, so we haven’t been able to upgrade. Our customers have been asking about this.

  • @Sean: That is not a bug, it’s the correct behavior. Please read http://www.modrails.com/documentation/Users%20guide%20Apache.html#_how_phusion_passenger_detects_whether_a_virtual_host_is_a_web_application for an explanation as well as possible solutions.

  • wow that rocks. Thanks!
    also did you tell apache up stream that their I/O stinks?
    Thanks again.
    -=r

  • soph

    Thanks for this update! We have a problem monitoring the passenger-status after this update

    /opt/ruby/bin/passenger-status
    *** ERROR: Cannot query status for Passenger instance 17918:
    No such file or directory – /tmp/passenger.17918/info/status.socket

    status.socket is at this place

    /tmp/passenger.17918/var/passenger.17918/info/status.socket

    any ideas?

  • soph

    ok, thought about it again after posting. I remembered that i did only an “apache2 reload” after updating passenger from 2.2.2 auf 2.2.3.
    after “apache2 restart” everything is normal again.

    Maybe somebody else hits the same problem…

  • Thanks. Updated RailsCluster!

  • Tommy

    This is great. Thanks for all of your hard work on this.

  • Pingback: Double Shot #477 « A Fresh Cup()

  • Juan

    And what about gems installed? Do I need to reinstall them?

    Thanx

  • Juan: Phusion Passenger has got nothing to do with gem management.

  • Pingback: The Knut Hellan Blog()

  • Installing the latest stable Nginx 0.7.6 regardless of the Nginx 0.6.3 by mod_rails defaults
    http://www.railsgeek.com/2009/6/20/installing-the-latest-stable-nginx-0-7-6-regardless-of-the-nginx-0-6-3-by-mod_rails-default

  • Quint

    @mikhailov — There’s a reason that Nginx 0.7.6 isn’t the default version that Passenger installs and that is because it hasn’t yet been verified that that particular version of Nginx is stable alongside Passenger.

  • Mike Hagedon

    “…through request.env[‘PATH’]. You can no longer do that now. Access ENV[‘PATH’] instead.”

    I’ve been modifying a Rails app to work with Shibboleth [http://shibboleth.internet2.edu/], and as far as I can tell this change breaks that. Shibboleth 2, by default, passes data through the Apache environment variables, but Passenger only sets ENV on startup. Since Shibboleth data is authorization-related, it needs to be request-specific. Having that data populated only on startup is fairly useless. 🙂 Shibboleth does have an option to use HTTP headers instead, but those are not recommended because they’re more easily spoofed than environment variables. (Cf. recent IIS vulnerability)

    Is there any way to get the environment variables into request.env? I don’t need the added mod_env functionality. As far as I can tell this all worked with 2.2.2.

    This is the situation as I see it; it’s quite possible I’m confused about all this. 🙂

  • Mike Hagedon: Not at this time, but you gave a valid reason to restore the old behavior. Could you file a bug report so that this issue doesn’t get lost?

  • Mike Hagedon

    hongli: Done. Thanks!

  • Pingback: he1ix' blog » Blog Archive » installing redmine on debian/lenny using mod_passenger()

  • Pingback: Installing phusion passenger with apache2 on Ubuntu « Exploring myself…()