From: Peter Booth Date: 2009-02-27T03:24:39+09:00 Subject: Re: Load/Stress/Performance Testing via Ruby --Apple-Mail-67--332510312 Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Dav, It really depends on why you want to do this. If you simply want to push a workload that is representative but without any concern about absolute correctness then a JMeter or LoadRunner or Grinder can be good. LoadRunner has a great UI. This allows you to stress test and see where the hotspots are. New Relic is an excellent complement to any of these. If you want a realistic load test and want to actually estimate absolute capacity then you're choosing to do something harder. The best tool is the open source httperf. Its the only tool here that uses an open workload model. You will find that LoadRunner and the other tools I suggested will always overestimate system capacity. That is because in these tools, a virtual agent won't fire a new user request until the prior one completes. This of course is not what happens in real life, where a heavy tailed Pareto distribution of requests is typical. Httperf addresses this issue and allows you to get data that is more realistic in an absolute sense. If you're wanting to tune and improve performance or throughput and not simply measure it that's a whole other ball-game. Be prepared to take some time and learn, or hire some help. Peter Booth 917 445 5663 PS - If anyone needs assistance with tuning Rails apps, website capacity planning, load testing, please ping me. I am available. I work at the application code, web server, DB server, operating system, TCP stack, physical network and hardware layers. On Wed, Feb 25, 2009 at 11:53 AM, Dav Yaginuma wrote: > I've done some research on this and I'm fairly convinced it doesn't > exist, > but I thought I would just ask here in case I missed something. > > I want to be able to full-stack load test a Ruby on Rails site (not > that > rails matters in this context) running dynamic test plans across > multiple > concurrent users in order to better replicate real world usage. > > In other words, I'd like to run say 50 concurrent user sessions > where some > of them are signing up, some are just browsing around, some are > logging in > and doing UserActivity1, some are doing UserActivity2, etc. > > I want to script the test plans in ruby, similar to a seleinum-rc > test plan > where I can use the same DSL to simulate user input and identify > parts of > the response html, and use that to drive further test activity. I'd > like to > use some of the same DSL from my selenium or integration test helper > modules > already developed in these tests as well in order to be DRY. > > I would like the test framework to mimc the browser behaviour: > handling > cookies and even simulating caching to send the appropriate > If-Modified-Since headers. > > Since I would like to replicate actual request load on the server, I > want to > make sure that all page assets, even static ones like images/css/js/ > flash, > are requested during the test, so it would be nice if an HTTP Proxy > could be > used to create the initial test plans by recording actual browser > sessions. > > Some things I've looked at so far: > > JMeter: > - seems difficult to make dynamic test plans here that will modify > requests > based on earlier responses > - test plans are in XML, although I suppose I could use a ruby DSL to > generate the XML, but since the proxy records to XML I'd also have to > generate ruby from XML > > The Grinder: > - Looks awesome, and I may end up going with this, but it would be > nicer to > use ruby instead of Jython in order to leverage my teams existing > code and > skills. > > shell/curl/ruby script mish mash: > - A lot of control, and curl handles cookies, but no real framework > support > and no proxy recording > > Ruby code: > - Doesn't really seem to be any test framework that deals with > concurrent > requests like this > - I assume JRuby would be required (and would be sufficient) to get > real > threading > - I feel like there are probably a lot of pieces that could be tied > together > to make what I want, but unsure of what they are or which ones to > concentrate on. Hpricot? Mechanize? > > Anyone have any ideas? Is anyone doing load testing via ruby? > > -- > Dav Yaginuma > http://AkuAku.org/ > --Apple-Mail-67--332510312--