Re: fastcgi performance problems and ruby

[e-mail] wrote:
> Hello
> 
> I have recently embarked upon the process of getting FastCGI to work
> with Apache in order to run Ruby via FastCGI.
> 
> It is all working a running but the performance is not what I had
> expected and certainly not what is needed to run a production web
> application.
> 
> Strange because I am using fairly powerful computers.  Stranger because
> this discussion thread
> 
> http://groups.google.com/groups?hl=en&lr=lang_en&safe=off&threadm=20021005174928.A71759%40freeze.org&rnum=9&p
> rev=/groups%3Fnum%3D100%26hl%3Den%26lr%3Dlang_en%26safe%3Doff%26q%3Dfastcgi%2Bruby%26btnG%3DSearch
> 
> -->extract from above thread
> a) 400 request/sec with mod_ruby and cached Ruby script
> b) 200 request/sec with fastcgi.so (the mixed C/Ruby version)
> c) 100 request/sec with FCGI (clean Ruby version)
> d) 70 request/sec with mod_ruby naive.
> 
> 
> The thread above seems to suggest that it would be reasonable to expect
> at least 200 requests/second using
> 
> fastcgi.  I actually would have expected better performance than this
> because I am using a 2.4ghz p4
> 
> I have included results of my tests below.
> 
> Can anyone throw any light for me on what sort performance I should
> expect from ruby under mod_fastcgi?
> 
> Would anyone mind taking the trouble to benchmark their own mod_fastcgi
> performance with Ruby?
> 
> I'm concerned that unless I get the performance up then ruby with
> fastcgi does not represent a production
> 
> level solution from a performance perspective.
> 
> Thanks in advance for any help.
> 
> Andrew Stuart
> andrew dot stuart at xse dot com dot au
> 
> ***********************************************************************
> ***********************************************************************
> ****************load testing application
> http://www.acme.com/software/http_load/
> http_load - multiprocessing http test client
> 
> load testing executed on the web server itself. not from a client over
> the network
> 
> ***********************************************************************
> ***********************************************************************
> ****************server config
> Intel P4 Celeron 2.4Ghz
> 512meg RAM
> Fedora core 2
> Apache
> FastGCI
> Ruby
> Postgres
> 
> ***********************************************************************
> ***********************************************************************
> ****************output of "free" shows plenty of memory available on
> server
> 
> [root@xsecore docs]# free
> total       used       free     shared    buffers
> cached
> Mem:        483724     138532     345192          0      21532
> 47944
> -/+ buffers/cache:      69056     414668
> Swap:       979956          0     979956
> [root@xsecore docs]#
> 
> ***********************************************************************
> ***********************************************************************
> ****************performance test plain HTML file -- 897.629 fetches/sec
> 
> [root@xsecore http_load-04jan2002]# ./http_load -verbose -parallel 100
> -seconds 20 url82testhtml.txt
> 17960 fetches, 20 max parallel, 1.42082e+08 bytes, in 20.0083 seconds
> 7911 mean bytes/connection
> 897.629 fetches/sec, 7.10114e+06 bytes/sec
> msecs/connect: 2.19892 mean, 20.245 max, 0.133 min
> msecs/first-response: 11.1338 mean, 20.245 max, 9.666 min
> HTTP response codes:
> code 200 -- 17960
> [root@xsecore http_load-04jan2002]#
> 
> ***********************************************************************
> ***********************************************************************
> ****************performance test FastCGI simple ruby script dump env --
> 155.699 fetches/sec
> 
> [root@xsecore http_load-04jan2002]# ./http_load -verbose -parallel 100
> -seconds 20 urlfcgienvrb.txt
> 3114 fetches, 100 max parallel, 2.21717e+06 bytes, in 20.0001 seconds
> 712 mean bytes/connection
> 155.699 fetches/sec, 110858 bytes/sec
> msecs/connect: 0.445062 mean, 30.947 max, 0.152 min
> msecs/first-response: 630.785 mean, 883.717 max, 13.366 min
> HTTP response codes:
> code 200 -- 3114
> [root@xsecore http_load-04jan2002]#
> 
> Ruby script source for this test:
> 
> [root@xsecore http_load-04jan2002]# cat /var/www/fcgi-bin/env.rb
> #!/usr/bin/ruby
> require 'cgi'
> require 'fcgi'
> 
> 
> FCGI.each_cgi do |cgi|
> content = ''
> env = []
> cgi.env_table.each do |k,v|
> env << [k,v]
> end
> env.sort!
> env.each do |k,v|
> content << %Q(#{k} => #{v}<br>\n)
> end
> content << "hello from dude"
> cgi.out{content}
> end
> 
> [root@xsecore http_load-04jan2002]#
> 
> ***********************************************************************
> ***********************************************************************
> ****************performance test FastCGI database access ruby script --
> 35.9997 fetches/sec
> 
> [root@xsecore http_load-04jan2002]# ./http_load -verbose -parallel 100
> -seconds 20 url82dbtestrb.txt
> 1639 fetches, 100 max parallel, 908006 bytes, in 20.0001 seconds
> 554 mean bytes/connection
> 81.9498 fetches/sec, 45400.2 bytes/sec
> msecs/connect: 0.429315 mean, 20.021 max, 0.146 min
> msecs/first-response: 1182.09 mean, 1489.77 max, 208.131 min
> HTTP response codes:
> code 200 -- 1639
> [root@xsecore http_load-04jan2002]#
> 
> Ruby script source for this test:
> 
> [root@xsecore http_load-04jan2002]# cat /var/www/fcgi-bin/dbtest.rb
> #!/usr/bin/ruby
> require 'cgi'
> require 'fcgi'
> require 'dbi'
> 
> dbh = DBI.connect('DBI:Pg:catchmail', 'dbusername', 'password')
> 
> FCGI.each_cgi do |cgi|
> 
> content = ''
> dbh.select_all('select msgno, msgtype, msgsubject, msgbody from
> message limit 10 offset 50') do |row|
> content << "subject line:  "
> content << row["msgsubject"]
> content << "<br>"
> end
> 
> cgi.out{content}
> end
> 
> dbh.disconnect
> [root@xsecore http_load-04jan2002]#
> 
> 
> ***********************************************************************
> ***********************************************************************
> ****************performance test FastCGI echo example -- 664.188
> fetches/sec
> 
> 
> [root@xsecore http_load-04jan2002]# ./http_load -verbose -parallel 100
> -seconds 20 url82fcgiecho.txt
> 13290 fetches, 31 max parallel, 1.21736e+07 bytes, in 20.0094 seconds
> 916 mean bytes/connection
> 664.188 fetches/sec, 608396 bytes/sec
> msecs/connect: 2.56759 mean, 30.368 max, 0.153 min
> msecs/first-response: 15.0369 mean, 86.158 max, 12.604 min
> HTTP response codes:
> code 200 -- 13290
> [root@xsecore http_load-04jan2002]#
> 
> Source code for echo is written in C and is included in the examples in
> the fastcgi software
> 
> 
> ***********************************************************************
> ***********************************************************************
> ****************performance test PHP example (not using FastCGI) --
> 234.798 fetches/sec
> 
> [root@xsecore http_load-04jan2002]# ./http_load -verbose -parallel 100
> -seconds 20 url82testphp.txt
> 4699 fetches, 100 max parallel, 1.88669e+08 bytes, in 20.013 seconds
> 40150.9 mean bytes/connection
> 234.798 fetches/sec, 9.42734e+06 bytes/sec
> msecs/connect: 5.28116 mean, 113.121 max, 0.15 min
> msecs/first-response: 161.609 mean, 2363 max, 22.176 min
> HTTP response codes:
> code 200 -- 4699
> [root@xsecore http_load-04jan2002]#
> 
> 
> Source code for test.php
> 
> [root@xsecore html]# cat test.php
> <html>
> <head>
> <title>PHP Test</title>
> </head>
> <body>
> <?php echo '<p>Hello World</p>'; ?>
> </body>
> </html>
> 
> <?php
> phpinfo();
> ?>
> 
> ***********************************************************************
> ***********************************************************************
> ****************performance test FastCGI simple ruby script dump
> username/IP -- 159.399 fetches/sec
> 
> [root@xsecore html]#
> [root@xsecore http_load-04jan2002]# ./http_load -verbose -parallel 100
> -seconds 20 urlfcgiusernamerb.txt
> 3188 fetches, 100 max parallel, 153024 bytes, in 20.0001 seconds
> 48 mean bytes/connection
> 159.399 fetches/sec, 7651.15 bytes/sec
> msecs/connect: 0.462713 mean, 41.912 max, 0.168 min
> msecs/first-response: 615.345 mean, 949.505 max, 160.687 min
> HTTP response codes:
> code 200 -- 3188
> [root@xsecore http_load-04jan2002]#
> 
> Source code for username.rb
> 
> [root@xsecore http_load-04jan2002]# cat /var/www/fcgi-bin/username.rb
> #!/usr/bin/ruby
> 
> require "fcgi"
> 
> FCGI.each_cgi {|cgi|
> puts cgi.header
> puts "You are #{cgi.remote_user} <br>"
> puts "Connecting from #{cgi.remote_addr} <br>"
> }
> 
> [root@xsecore http_load-04jan2002]#
> 
> 
> ***********************************************************************
> ***********************************************************************
> ***************************extract from httpd.conf relating to FastCGI
> 
> LoadModule fastcgi_module modules/mod_fastcgi.so
> 
> <IfModule mod_fastcgi.c>
> 
> # URIs that begin with /fcgi-bin/, are found in /var/www/fcgi-bin/
> Alias /fcgi-bin/ /var/www/fcgi-bin/
> 
> FastCgiIpcDir /var/fcgi/
> #FastCgiConfig -autoUpdate
> 
> # Anything in here is handled as a "dynamic" server if not defined
> as "static" or "external"
> <Directory /var/www/fcgi-bin/>
> Order allow,deny
> Allow from all
> SetHandler fastcgi-script
> Options +ExecCGI
> #Require user andrewstuart
> </Directory>
> 
> # Anything with one of these extensions is handled as a "dynamic"
> server if not defined as
> # "static" or "external". Note: "dynamic" servers require ExecCGI
> to be on in their directory.
> AddHandler fastcgi-script .fcgi .fpl
> 
> # Start a "static" server at httpd initialization inside the scope
> of the SetHandler
> FastCgiServer /var/www/fcgi-bin/examples/echo -processes 5
> 
> # Start a "static" server at httpd initialization inside the scope
> of the AddHandler
> #FastCgiServer /var/www/fcgi-bin/examples/echo
> 
> # Start a "static" server at httpd initialization outside the scope
> of the Set/AddHandler
> #FastCgiServer /var/www/htdocs/some/path/coolapp
> <Directory /var/www/html/ruby>
> SetHandler fastcgi-script
>     </Directory>
> 
> </IfModule>
> 

Sometimes, limiting the number of fastcgi processes that get created can 
help boost performance.  See:

http://www.renegadeinternet.com/kb/faq.php?num=4&f_id=11&s_id=34&q_id=125