Phusion Blog

At Phusion, we’ve received some comments from Engine Yard customers who have mentioned that upon requesting support for Passenger, they were advised to stick with an Nginx + Mongrel setup instead. Even though Passenger is our flagship product, I also have the utmost respect for both Mongrel and Nginx, and believe that people should choose the solution they deem most suitable and pragmatic for the job. In light of this, I believe Passenger and Apache are just as viable a solution for “the deployment problem” as the aforementioned solutions. Unfortunately, after inquiring a bit further, I was unable to get a clear answer as to why they were advised by Engine Yard to not use Passenger specifically.

After reading some responses on Ezra’s talk on Rails deployment, and in particular, after reading this summary by an attendee of this talk, the impression seemed to be made that Passenger was only, if not more suitable, for small VPS and/or shared hosting environments. Unfortunately, I was unable to attend this talk myself but luckily, Ezra has made his slides publicly available on his blog.

Here, slide 13 states “Passenger(with caveats)” and “State of art now: Passenger for shared hosting/small VPS”. Clearly, we’re unable to infer what is being said here without the proper narrative/annotations which unfortunately isn’t available. This leaves room for interpretation and the slides on their own seem to suggest that Passenger “has caveats” and is the best solution for shared hosting/small VPS. Even though it seems to do a great job in these fields, it’s certainly not the only goal we had in mind when we were designing and engineering Passenger.

Taking this into consideration, I felt inclined to further investigate and asked Ezra for some elaboration in the comments section of the aforementioned blogs. Here it soon became apparent that Ezra seemed to have problems with the memory consumption of Apache and Ruby Enterprise Edition(REE) not giving them memory savings on 64bit platforms but segfaulting instead.

As I’ve pointed out earlier in the comments section of Ezra’s blog, I believe it’s one thing (and very unfortunate) that saying one solution isn’t working out for Engine Yard, but it’s an entirely different matter to give the impression that is therefore more suitable for VPS/small hosting. Especially if no specifics are or have ever been given about this potential problem in order for people like us to try to reproduce it. After asking for these specifics, Ezra commented that he would contact us on this, but Ezra is probably a super busy man so we unfortunately didn’t receive anything on this as of yet. The just recently released Ruby Enterprise Edition however, contains fixes that should increase the support for 64bit and we hope that they will solve any problems people might have with this on their 64bit platform.

Update:
It seems Ezra sent us an email at around the same time this reply was posted which contents now seems to indicate a working REE setup on their 64 bit platform! 🙂 I’d still be interested in learning more about their test suite though which would allow us verify a working setup through reproduction, and I’m sure the community would benefit from having this kind of knowledge as well.

Many thanks to the following people and organizations for making this possible through sponsorship (list sorted in alphabetical order):

• 37signals
• Curve21
• Dr Dispatch Transportation Software
• InfoEther
• Martian Interactive
• New York Times
• Shopify
• Trevor Turk
• Utah Imaging

After releasing both Passenger 2.0.5 and Ruby Enterprise Edition, Tom Mornini from Engine Yard elaborated a bit more on the reasons why Engine Yard is currently favoring Nginx + Mongrel over Passenger + Apache. Seeing as some reasons are given with regards to Passenger and Apache, I’d like to go over these to try to get the entire story across for you to decide whether or not it is suitable for you.

First off, Tom writes:

“If you’re running multiple Rails applications, and are using Passenger’s ability to serve more applications than applications processes, you need to be aware of the performance of requests requiring an application process switch from one application to another. First requests are always slow, as a *lot* of code needs to be heated up, etc.”

Here, I believe the very first thing to note is the first word Tom uses in this paragraph, being “if”. “If” you are running multiple Rails applications and are using Passenger’s ability to serve more applications than application processes, then yes, you need to be aware of the possibility of the performance of requests requiring an application switch from one application to another. The reason why I’ve emphasized possibility here is that you need to have more concurrent requests in progress than application processes in the first place in order to be able to get into such a situation. Having said that, one could just as easily increase the allowed application instances Passenger is allowed to spawn to the number of applications we have and increase the PassengerPoolIdleTime to a sufficiently large number so that they will virtually live as long as you’ll ever need. In git master, we’ve made this solution a bit more elegant by providing a sugar that allows you to assign a negative value to PassengerPoolIdleTime, which will result in it “living infinitely long” and you can expect this to be in 2.0.6. Doing something like this will essentially give you the same behavior as having and keeping multiple mongrel instances alive behind a proxy balancer. The key thing to note here however, is that we actually allow you to choose for an alternative setup as well to accommodate your needs even better. Just as there are situations in which you want the behavior Tom is describing, there are situations where it is perhaps deemed more suitable to have less application processes than actual applications (e.g. where memory is limited). The important thing to remember here however, is that Passenger allows you to do both (very easily). Doing the aforementioned with Passenger requires you to just modify two numeric values in the Apache configuration (PassengerPoolIdleTime and PassengerMaxPoolSize) and doesn’t involve manual port management for example which in general, is necessary for proxy-load-balanced-mongrel setups. Using Passenger to establish the same setup as proxy-load-balanced-mongrel setups involves significantly less steps, making it arguably less tedious and less error prone to maintain. Unfortunately, this part isn’t elaborated on in Tom’s post, which could result in people being left only half informed.

Also, Tom has made mention of Passenger possibly needing to load a “lot” of code. Even though Tom acknowledges that Ruby Enterprise Edition *does* offer the additional advantage of memory savings, there doesn’t seem to be an explicit mention of the fact that Ruby Enterprise Edition negates “this need to load a lot of code” dramatically/entirely by leveraging a technique called Copy-on-Write which is available on a myriad of *nix variants. Copy-on-Write works on an operating systems level so if you already have a running Rails process, by the time you need to spawn a new one, the operating system is able to infer that most –if not all– memory can be pooled and doesn’t need to be replicated. This in particular is the case for identical applications, where not only the Rails framework will be shared among processes but also the application code.

Moving on, Tom writes:

“The issues I’m discussing are generally acceptable in a situation where you have many low traffic applications, and you need to run those applications in a very limited environment. At Engine Yard, this is a very rare need for our customers, so this makes us less anxious to switch to something new -vs- something that is proven and works well.”

Even though just shy of 8 months in production status, a growing number of very large sites (see here and here) are using REE and Passenger to successfully serve millions upon millions of request per day respectively. I’m not sure what one would need to do to get it to a point of being considered “proven and working well”, but what I do however know is that sites by MTV, Yammer, Geni, Shopify and 37signals are powered by Passenger and seem to have no problems dealing with high-volume requests.

Moving on, Tom writes:

“With the additional of Ruby Enterprise Edition, however, it *does* offer the additional advantage of memory savings. Some of those advantages are lost by having to run Apache, which as commonly delivered, is considerably larger and less efficient than nginx. Of course, you can reduce these disadvantages by cuting down a custom build and tweaking the included modules, but once you’re headed down that road, the simplicity advantage has left the building. :-)”

Even though we’ve also inferred that an “out-of-the-box” Apache installation incurs a larger overhead than an “out-of-the-box” Nginx, it begs the question if this is really relevant in this article. After all, I was under the impression that Tom was elaborating about why Engine Yard isn’t using Passenger + Apache, where he also mentions that one can configure Apache to get comparable results to Nginx, effectively giving the answer to Engine Yard’s own “problems” with Apache.

From our experience, configuring an Apache setup to “perform on par with Nginx” isn’t trivial, but it’s very doable. Also note that this initial time one spends on setting up such an Apache installation is being rewarded by not having to maintain a relatively more difficult setup with nginx+mongrel. Here, you need to ask yourself how much time you spend on doing an Apache setup as opposed to maintaining the actual installation, i.e. how many times you update Apache as opposed to maintaining and updating Passenger. In general, from our experience, the former doesn’t take as nearly as much time as the latter. To get a sense of what I’m trying to get at, please take a look at the Apache site where you can see that the previous and most current version were released about 9 months apart from each other by the time of this writing. If we also take into account that in general, your old Apache configuration files will probably still work after doing an Apache upgrade, then I’m not sure if this really is such a big problem as it is made out to be. Especially with the solution we’re currently working on, which I hope to be able to elaborate a bit more about soon.

One thing is for sure however, heavily visited Passenger powered sites like Shopify which processes millions upon millions of requests per day
certainly don’t seem to have a problem with neither memory or performance and benefit greatly from this approach.

In particular, you might find the following quote from Tobias Lütke of Shopify fame
to be of particular interest:

“We switched to enterprise ruby to get the full benefit of the [copy-on-write] memory characteristics and we can absolutely confirm the memory savings of 30% some others have reported. This is many thousand dollars of savings even at today’s hardware prices.”

Moving on, Tom writes:

“Second, our customers choose Engine Yard for experience and expertise in hosting Rails applications. At this moment, we have no experience or expertise in Passenger™.
That is a fact.”

Well fortunately, Phusion offers commercial support for Passenger so this doesn’t necessarily have to be a problem. 😉 More interestingly however, is that even though Engine Yard claims to not have experience or expertise in Passenger, they seem to be perfectly capable of making some strong statements about it anyway as found on Ezra’s slides and Tom’s blog post. This also, is a fact.

In closing, somewhere in the middle of the article, Tom writes:

“The reason that I felt compelled to respond to Pratik’s Twitter post is that it entirely ignores a few issues that prevent Engine Yard, and no doubt other companies from immediately supporting Passenger™ and/or Ruby Enterprise Edition.”

In a similar fashion, I felt inclined to respond to Tom’s article seeing as it completely seemed to ignore a few (easy) solutions as well.

The morale of the story: feel encouraged to try out different solutions to find out which one suits you best, regardless of its current life-span, because that’s the only way of helping a product mature. In any case, please remain critical of information you receive, and yes, that even means this blog post. Seeing is believing right? Whatever the case may be, stay tuned for more exciting news from Phusion. 🙂