From: Ezra Zygmuntowicz <ezmobius@...>
Date: 2006-11-30T07:09:51+09:00
Subject: Re: Two Advanced Ruby Performance Questions


On Nov 28, 2006, at 10:08 PM, Sunny Hirai wrote:

> Max Muermann wrote:
>> I know you don't intend to use Rails, but there's a
>> performance-specific blog dealing with Rails that might be of  
>> interest
>> to you:
>>
>> http://railsexpress.de/blog/
>>
>> Maybe you can get in touch with those guys and get them to share some
>> fo their experiences.
>
> Thanks for the recommendation. The website looks to be some more of  
> what
> I'm looking for.
>
> Hi Ezra,
>
> Could you comment on Xen instances yielding better performance then
> instances of Ruby on a single server. Is this a mild improvement, a
> significant one or for redundancy/easy of deployment?
>
> By the way, the ideas in engineyard are fascinating. Offering a  
> hosting
> platform with scalability built in. Very Nice.
>
> I'd be willing to pay for an early beta of your book. :)
>
> Sunny Hirai

Hey Sunny-

	The whole thing about Xen is involved in the idea of easy  
scalability. The behavior of production apps run on a cluster of  
mongrels that I have seen has a sweet spot. About 3 or 4 mongrels in  
a cluster behind a front webserver like nginx all running in one Xen  
instance can serve quite a bit of traffic. Any more then 5 mongrels  
in a small cluster like this and you start to see diminishing  
returns. And when you add a shared filesystem to the mix like gfs  
then you can add and remove nodes from an app cluster at will. Just  
by cloning the xen instance and having it join the cluster and mount  
the rails applications on a gfs mount. Doing it this way you can have  
up to 16 xen instances all sharing a gfs filesystem all serving the  
same rails application. So when it is time to deploy new code you  
only have to deploy the code to one of these xen nodes and then just  
restart the mongrels on all nodes that share the same gfs mount. Then  
all nodes are serving the new code. All of these nodes run behind  
hardware load balancers so you can bring them down and up in a piggy  
back fashion and never have down time to the website users.

	We have found that Xen only imposes a performance overhead of 5-9%  
compared to non virtualized linux. So going with xen makes it so much  
easier to scale across boxes that the tradeoff is well worth it. The  
only boxes you may not want Xen on is the database box or boxes. If  
you have super heavy database traffic then you may want to go non  
virtualized on that box. But saying that, we still run our mysql  
clusters on xen instances and have not had performance problems with  
it. We use coraid AoE SAN which is block level ATA disk access over  
ethernet but without the overhead of tcp/ip. So none of our servers  
have hard disk drives. They only have 128Mb flash ata chips for the  
main Xen dom0's to boot off of. Then they load all the domU's off of  
the SAN.

	Rails tends to use a lot of memory but the cpu usage is not so bad.  
Our cpu's are sleeping and its always ram that needs to be increased.  
We are going to start buying big 4 processor boxes with 32 or 64gigs  
of ram and virtualizing on top of those. Currently we use boxes  with  
two dual-core amd opterons with 8gigs of ram.

	Using many smaller rails app server nodes behind load balancing with  
all of them sharing state with the database and the shared filesystem  
makes applications feel responsive and distribute the load in a nice  
way.

Cheers-
-- Ezra Zygmuntowicz 
-- Lead Rails Evangelist
-- ez@engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)