From: Peter Booth <pbooth@...>
Date: 2009-04-12T13:46:10+09:00
Subject: Re: load testing tool - Please advise?

--Apple-Mail-2--788639809
Content-Type: text/plain;
	charset=US-ASCII;
	format=flowed;
	delsp=yes
Content-Transfer-Encoding: 7bit

My two cents:

Load testing is hard, and often load tests produce results that are  
not credible.
Watir isn't right for load testing, and there are many alternatives to  
choose from.

There are two key questions:

1. What's the goal of the testing? 	
	B:  identify Bottlenecks
	C:  estimate Capacity

2. Are you wanting to
	S: isolate a Single page/user operation
or	G: test a Group of pages / paths

BS  	  --> 	all you need is ApacheBenchmark, ab
BG 	  --> 	JMeter, Grinder and Loadrunner all support remote agents,  
scripts.
		Grinder is Java based, like JMeter, but configured with jython scripts
		Loadrunner is commercial, expensive, and visualizes results in a way  
that
		can sometimes help you to see patterns quickly.
CG,CS  --> 	JMeter, Grinder and Loadrunner produce data that  
overestimates throughput.
		This is because they use a closed system to model workload
		where a "user agent" issues a request, and then won't issue a second
		until first is complete. If you want to estimate absolute capacity  
then you need
		a workload generator that can drive your system to the point of  
overload, and the best
		I know is httperf, an open source product created by HP. httperf  
uses an open model to
		drive traffic probabilistically

The best thing about load testing Ruby is "Ruby is slow and Rails  
doesn't scale" (irony intentional). With Java its more likely that you  
will need five or ten test clients to produce enough traffic to stress  
your application. The HP Tech Report on httperf discusses some of the  
issues that make it difficult do do effective load testing.

http://www.hpl.hp.com/techreports/98/HPL-98-61.pdf

If your application uses Rails then I'd strongly recommend buying a  
New Relic Silver subscription for at least month. It's a great time- 
saver when diagnosing performance issues.

Hope this helps,

Peter Booth



I would strongly recommend using

On Apr 10, 2009, at 9:23 PM, Kyle Schmitt wrote:

> I'd go straight for JMeter.
>
> WATiR works by driving a browser, and you just aren't really going to
> be able to stress test a webapp that way.  Even with several machines
> running the tests.
>
> Jmeter works by simulating user sessions, it isn't a whole browser.
> It handles sessions and cookies, but other than that just sends http
> commands to the server.  Because of it's design, it can easily
> generate a lot of traffic from multiple different sessions on a single
> machine, and really give the server a working over.
>
> It takes a few days to get really good at, but it's free (open
> source), currently maintained, and very _very_ powerful.  It's heavily
> java based, and any scripting done in it is though java-esqe scripting
> languages, not ruby.  That said, it's easy to make it read values from
> csv and xml files for dynamic tests, so you can do a whole lot without
> much scripting.
>
> When I worked as a test-developer it's what I used for stress testing,
> while relying on WATiR for other testing.
>
> --Kyle
>


--Apple-Mail-2--788639809--