From: Greg Lorriman <temp@...>
Date: 2007-03-08T05:20:05+09:00
Subject: Re: Horizontal scaling - advice needed


> Drb would rapidly become a bottleneck there.  

I've not seen any articles on the characteristics and limitations of
Drb but it isn't immediately apparent to me that it should be a
bottleneck. Why do you think it would be a problem?

I can only see that in the longer term there might be a network
traffic problem if I had a lot of machines. I haven't worked it out
yet but perhaps the growth in inter-machine traffic would not be
linear.

> And explicitly federating
> your data like that seems needlessly complicated.

I would agree except that the application is simple while at the same
time such federation is the ultimate in horizontal scaling AFAIK, and
that is the exercise I would like to embark on. I want to find out the
ins and outs, costs and difficulties.

> Put a fast proxy of some sort in front of your backend processes.  When
> you need more throughput, add another machine and some more processes.

Eventually the database will be the bottleneck, hence federation.

> > There must be proxies of this kind, but I'll be blowed if I know what an
> > appropriate one would be, or where to start in making it do what I want.
>
> If I were doing something that required some very specific proxying
> behavior that I couldn't get with an off-the-shelf solution (HAProxy is a
> very nice general purpose proxy with a ton of features), I'd write a
> purpose built proxy.  It's not really too hard to do.

I'll keep that in mind. I hope you are right. But I've always had the
idea that tcp/ip is a somewhat painful affair, and I suspect one would
have to drop to that level to get adequate efficiency/speed to act as
a router/redirector of incoming requests.

thanks for the words.