From: Robert Klemme <bob.news@...>
Date: 2005-12-01T19:07:32+09:00
Subject: Re: Ruby Enterprise App Design Advice

<disclaimer>I have done web applications and I have done enterprise level
apps but not enterprize level web apps of this scale.  I do have quite
some experience with databases though.</disclaimer>

TeslaOMD wrote:

<snip/>

> I'll try to oversimplify this in the interest of this post not being
> more of a book than it already is. The software is actually a new idea
> that I'm not going to reveal for creative reasons, but it is probably
> closest to some of the existing social networking/dating apps. The  app
> must support over a million users (yes, this is realistic if not a
> gross underestimate if all goes well). We expect to have several
> thousand concurrent users. AJAX and querying DB data will play a heavy
> role in the system. Users will have lots of preferences/settings to
> manage/configure, most of which will be stored to disk as xml
> (serialized objects perhaps?) rather than the database.

Did you consider writing user configs using YAML?  It's human readably and
you don't have to fiddle with XML mappings etc.

<snip/>

> Current Specs/Design:
>
> Our design uses MVC/n-tier. Business logic should be able to run
> independently on its own servers and not care about the web. Some of
> our basic design/plans are as follows:
>
> 1. Web Server -- Lighthttpd - seems to work well with FastCGI, very
> quick. Load balanced to send user to best server. Dedicated servers
> for things like serving images.
>
> 2. FastCGI or SCGI - We would like to replace FastCGI with something
> else if possible since we have concerns about all of our processes
> being constantly occupied by AJAX polling back to server code. We're
> not entirely convinced FastCGI is a great architecture for us but if
> we do use it, we would like to scale it accross many servers and use a
> SessionID to bind a user to a server. I worry about running out of
> available FastCGI processes, even with multiple machines.

An architectural idea: since you want to use sessions to pin users to a
certain instance and want to have a single session server (if I understand
you correctly) you could set up your session server as DRB server that
deals with login and logout and assigns an app server.  App servers use
DRB to register with the session server and also are DRB servers for
FastCGI handlers.  The FastCGI client programs are only small stubs that
ask the session server for the app server and then connect to the app
server and probably stream results back to the web client.  There is room
for optimization and tuning: mapping from session to app server can be
cached to save one internal network communication.  You are free to decide
where to do the HTML generation - on the app server or more likely in the
FastCGI program.  To better utilize resources you can have several app
servers per machine (i.e. several ruby processes that use different ports
for DRB communication).

> 3. Database -- MySQL or Postgres - We need transaction support and
> data integrity -- doubts about MySQL in these areas. We also need
> good join performance - database is heavily normalized, may
> denormalize if needed. Separate DBs for things like Logging/Auditing
> vs. Content. Want to cluster DBs. We want connection pooling.
> Considering making some modifications to DBI.  Use stored procedures
> in the database, no sql in the code/dynamic sql. I loathe O/R mappers
> for complex databases and they would likely bring our system to a
> screeching halt. Unfortunately we cannot afford to use Oracle right
> now.

I don't have experience with postgres and mysql.  Given the fact that
mysql is quite new and so are stored procedures and the way it deals with
transactions I'd probably tend to use postgres.

> 4. Caching -- We'll cache user settings, DB data, etc. Need a good
> ruby caching solution. Considering using memcached or something else
> existing.

Difficult to comment on this without further detailing the app
architecture.  Just one remark: I would try to not cache DB content as
databases do this already and they are usually quite good at doing this.

<snip/>

> 7. Sessions - Separate server for managing Sessions if possible. I
> would like to persist things in memory and share if possible. If not
> possible, we'll settle for persisting info to the database. Been
> looking at Session affinity for Ruby some.

see above.

> 8. AJAX - Polling and Queuing system for AJAX interactions in
> Javascript. Queing server side. Likely we will use and possibly extend
> an existing AJAX library such as Mochikit or base it off another
> existing one.
>
> 9. Remote - Looked at Drb for some remote things.

see above.

<snip/>

> 11. Events/Threading - I am worried about Ruby here, especially after
> reading about Myriad's problems with libevent. We will definitely need
> something similar to delegates and events and  some good queueing and
> threading functionality.

If your FastCGI processes are lightweight then ruby threading is probably
good enough.


<snip/>

Kind regards

    robert