From: andrew.stuart@...
Date: 2004-11-02T12:18:51+09:00
Subject: fastcgi performance problems and ruby

Hello

I have recently embarked upon the process of getting FastCGI to work
with Apache in order to run Ruby via FastCGI.

It is all working a running but the performance is not what I had
expected and certainly not what is needed to run a production web
application.

Strange because I am using fairly powerful computers.  Stranger because
this discussion thread

http://groups.google.com/groups?hl=en&lr=lang_en&safe=off&threadm=20021005174928.A71759%40freeze.org&rnum=9&p
rev=/groups%3Fnum%3D100%26hl%3Den%26lr%3Dlang_en%26safe%3Doff%26q%3Dfastcgi%2Bruby%26btnG%3DSearch

-->extract from above thread
a) 400 request/sec with mod_ruby and cached Ruby script
b) 200 request/sec with fastcgi.so (the mixed C/Ruby version)
c) 100 request/sec with FCGI (clean Ruby version)
d) 70 request/sec with mod_ruby naive.


The thread above seems to suggest that it would be reasonable to expect
at least 200 requests/second using

fastcgi.  I actually would have expected better performance than this
because I am using a 2.4ghz p4

I have included results of my tests below.

Can anyone throw any light for me on what sort performance I should
expect from ruby under mod_fastcgi?

Would anyone mind taking the trouble to benchmark their own mod_fastcgi
performance with Ruby?

I'm concerned that unless I get the performance up then ruby with
fastcgi does not represent a production

level solution from a performance perspective.

Thanks in advance for any help.

Andrew Stuart
andrew dot stuart at xse dot com dot au

***********************************************************************
***********************************************************************
****************load testing application
http://www.acme.com/software/http_load/
http_load - multiprocessing http test client

load testing executed on the web server itself. not from a client over
the network

***********************************************************************
***********************************************************************
****************server config
Intel P4 Celeron 2.4Ghz
512meg RAM
Fedora core 2
Apache
FastGCI
Ruby
Postgres

***********************************************************************
***********************************************************************
****************output of "free" shows plenty of memory available on
server

[root@xsecore docs]# free
total       used       free     shared    buffers
cached
Mem:        483724     138532     345192          0      21532
47944
-/+ buffers/cache:      69056     414668
Swap:       979956          0     979956
[root@xsecore docs]#

***********************************************************************
***********************************************************************
****************performance test plain HTML file -- 897.629 fetches/sec

[root@xsecore http_load-04jan2002]# ./http_load -verbose -parallel 100
-seconds 20 url82testhtml.txt
17960 fetches, 20 max parallel, 1.42082e+08 bytes, in 20.0083 seconds
7911 mean bytes/connection
897.629 fetches/sec, 7.10114e+06 bytes/sec
msecs/connect: 2.19892 mean, 20.245 max, 0.133 min
msecs/first-response: 11.1338 mean, 20.245 max, 9.666 min
HTTP response codes:
code 200 -- 17960
[root@xsecore http_load-04jan2002]#

***********************************************************************
***********************************************************************
****************performance test FastCGI simple ruby script dump env --
155.699 fetches/sec

[root@xsecore http_load-04jan2002]# ./http_load -verbose -parallel 100
-seconds 20 urlfcgienvrb.txt
3114 fetches, 100 max parallel, 2.21717e+06 bytes, in 20.0001 seconds
712 mean bytes/connection
155.699 fetches/sec, 110858 bytes/sec
msecs/connect: 0.445062 mean, 30.947 max, 0.152 min
msecs/first-response: 630.785 mean, 883.717 max, 13.366 min
HTTP response codes:
code 200 -- 3114
[root@xsecore http_load-04jan2002]#

Ruby script source for this test:

[root@xsecore http_load-04jan2002]# cat /var/www/fcgi-bin/env.rb
#!/usr/bin/ruby
require 'cgi'
require 'fcgi'


FCGI.each_cgi do |cgi|
content = ''
env = []
cgi.env_table.each do |k,v|
env << [k,v]
end
env.sort!
env.each do |k,v|
content << %Q(#{k} => #{v}<br>\n)
end
content << "hello from dude"
cgi.out{content}
end

[root@xsecore http_load-04jan2002]#

***********************************************************************
***********************************************************************
****************performance test FastCGI database access ruby script --
35.9997 fetches/sec

[root@xsecore http_load-04jan2002]# ./http_load -verbose -parallel 100
-seconds 20 url82dbtestrb.txt
1639 fetches, 100 max parallel, 908006 bytes, in 20.0001 seconds
554 mean bytes/connection
81.9498 fetches/sec, 45400.2 bytes/sec
msecs/connect: 0.429315 mean, 20.021 max, 0.146 min
msecs/first-response: 1182.09 mean, 1489.77 max, 208.131 min
HTTP response codes:
code 200 -- 1639
[root@xsecore http_load-04jan2002]#

Ruby script source for this test:

[root@xsecore http_load-04jan2002]# cat /var/www/fcgi-bin/dbtest.rb
#!/usr/bin/ruby
require 'cgi'
require 'fcgi'
require 'dbi'

dbh = DBI.connect('DBI:Pg:catchmail', 'dbusername', 'password')

FCGI.each_cgi do |cgi|

content = ''
dbh.select_all('select msgno, msgtype, msgsubject, msgbody from
message limit 10 offset 50') do |row|
content << "subject line:  "
content << row["msgsubject"]
content << "<br>"
end

cgi.out{content}
end

dbh.disconnect
[root@xsecore http_load-04jan2002]#


***********************************************************************
***********************************************************************
****************performance test FastCGI echo example -- 664.188
fetches/sec


[root@xsecore http_load-04jan2002]# ./http_load -verbose -parallel 100
-seconds 20 url82fcgiecho.txt
13290 fetches, 31 max parallel, 1.21736e+07 bytes, in 20.0094 seconds
916 mean bytes/connection
664.188 fetches/sec, 608396 bytes/sec
msecs/connect: 2.56759 mean, 30.368 max, 0.153 min
msecs/first-response: 15.0369 mean, 86.158 max, 12.604 min
HTTP response codes:
code 200 -- 13290
[root@xsecore http_load-04jan2002]#

Source code for echo is written in C and is included in the examples in
the fastcgi software


***********************************************************************
***********************************************************************
****************performance test PHP example (not using FastCGI) --
234.798 fetches/sec

[root@xsecore http_load-04jan2002]# ./http_load -verbose -parallel 100
-seconds 20 url82testphp.txt
4699 fetches, 100 max parallel, 1.88669e+08 bytes, in 20.013 seconds
40150.9 mean bytes/connection
234.798 fetches/sec, 9.42734e+06 bytes/sec
msecs/connect: 5.28116 mean, 113.121 max, 0.15 min
msecs/first-response: 161.609 mean, 2363 max, 22.176 min
HTTP response codes:
code 200 -- 4699
[root@xsecore http_load-04jan2002]#


Source code for test.php

[root@xsecore html]# cat test.php
<html>
<head>
<title>PHP Test</title>
</head>
<body>
<?php echo '<p>Hello World</p>'; ?>
</body>
</html>

<?php
phpinfo();
?>

***********************************************************************
***********************************************************************
****************performance test FastCGI simple ruby script dump
username/IP -- 159.399 fetches/sec

[root@xsecore html]#
[root@xsecore http_load-04jan2002]# ./http_load -verbose -parallel 100
-seconds 20 urlfcgiusernamerb.txt
3188 fetches, 100 max parallel, 153024 bytes, in 20.0001 seconds
48 mean bytes/connection
159.399 fetches/sec, 7651.15 bytes/sec
msecs/connect: 0.462713 mean, 41.912 max, 0.168 min
msecs/first-response: 615.345 mean, 949.505 max, 160.687 min
HTTP response codes:
code 200 -- 3188
[root@xsecore http_load-04jan2002]#

Source code for username.rb

[root@xsecore http_load-04jan2002]# cat /var/www/fcgi-bin/username.rb
#!/usr/bin/ruby

require "fcgi"

FCGI.each_cgi {|cgi|
puts cgi.header
puts "You are #{cgi.remote_user} <br>"
puts "Connecting from #{cgi.remote_addr} <br>"
}

[root@xsecore http_load-04jan2002]#


***********************************************************************
***********************************************************************
***************************extract from httpd.conf relating to FastCGI

LoadModule fastcgi_module modules/mod_fastcgi.so

<IfModule mod_fastcgi.c>

# URIs that begin with /fcgi-bin/, are found in /var/www/fcgi-bin/
Alias /fcgi-bin/ /var/www/fcgi-bin/

FastCgiIpcDir /var/fcgi/
#FastCgiConfig -autoUpdate

# Anything in here is handled as a "dynamic" server if not defined
as "static" or "external"
<Directory /var/www/fcgi-bin/>
Order allow,deny
Allow from all
SetHandler fastcgi-script
Options +ExecCGI
#Require user andrewstuart
</Directory>

# Anything with one of these extensions is handled as a "dynamic"
server if not defined as
# "static" or "external". Note: "dynamic" servers require ExecCGI
to be on in their directory.
AddHandler fastcgi-script .fcgi .fpl

# Start a "static" server at httpd initialization inside the scope
of the SetHandler
FastCgiServer /var/www/fcgi-bin/examples/echo -processes 5

# Start a "static" server at httpd initialization inside the scope
of the AddHandler
#FastCgiServer /var/www/fcgi-bin/examples/echo

# Start a "static" server at httpd initialization outside the scope
of the Set/AddHandler
#FastCgiServer /var/www/htdocs/some/path/coolapp
<Directory /var/www/html/ruby>
SetHandler fastcgi-script
    </Directory>

</IfModule>