From: jason marshall Date: 2012-09-12T05:30:08+09:00 Subject: Re: Looking for advice on how to report performance issues --20cf3079c0bc8313fe04c972f0bd Content-Type: text/plain; charset=ISO-8859-1 Comments inline On Mon, Sep 10, 2012 at 11:27 PM, Robert Klemme wrote: > On Mon, Sep 10, 2012 at 10:16 PM, jason marshall > wrote: > > We have a Rails app running on Ruby 1.9.3, and a under Windows we're > seeing > > a 20x time penalty in our RSpec tests. > > And the hardware is comparable? Yes, good question. On the same model hardware, it's 20x initial, 4.5x current. (2000, 440, vs <100 seconds) > > I'm fairly new to Ruby, but not to performance analysis, so I've > volunteered > > to follow up on this problem. If I want to file/find a bug to associate > > with this problem, what sort of data should I be gathering, and how? > > I have zero experience with Rails profiling, but maybe using the std > lib profiler can give you a first data point. > Yeah my readings from Google say it's not easy to do what I need to do. I'm hoping someone could talk me down from the ledge and point me to a baked solution. > > > Background/random data: > > > > We have 800+ tests, and on a good day they all run in a minute. Some do > no > > I/O (pure 'unit' test), most do light I/O (usually a couple create+reads > to > > MongoDB or a render partial). Only a few do any heavy lifting. > > > > I don't see any glaring hotspots. Most of the tests seem to take several > > times longer to run. Dots just accumulate slower. > > Just to exclude an obvious issue: do you have an AV engine up and > running on the Windows box? If yes, excluding some directories might > help. Some engines then still slow down operation so completely > switching off for the test might also be an option to determine > whether it's AV. > I'll entertain the question, but let's take it as a given that turning off AV on Windows is a very, very unwise action. Those numbers reflect my project directory already being excluded. I'm not sure I'm comfortable having Ruby whitelisted, though it's an interesting data point: With Ruby exclusion, Regular is 1490 seconds, patched is 400 seconds. > To exclude something else: is there concurrent activity that eats CPU > and / or IO bandwidth? > Not much. Or at least, there's not a lot of jitter in the numbers. I just have RubyMine and a web browser (low CPU) open. > > Kind regards > > robert > > -- > remember.guy do |as, often| as.you_can - without end > http://blog.rubybestpractices.com/ > > As a mostly unrelated anecdote, the Subversion team had conversations like this for a long time, and resisted making changes for a number of years. In the 1.7 release they made some pretty substantial changes to how they use the filesystem, and ended up with something that was both simpler (for users) and happened to work a hell of a lot faster on Windows. It sounds like you guys are on that path at least. -- - Jason --20cf3079c0bc8313fe04c972f0bd Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Comments inline

On Mon, Sep 10, 2012 at 1= 1:27 PM, Robert Klemme <shortcutter@googlemail.com>= wrote:
On Mon, Sep 10, 2012 at 10= :16 PM, jason marshall <jdmarsha= ll@gmail.com> wrote:
> We have a Rails app running on Ruby 1.9.3, and a under Windows we'= re seeing
> a 20x time penalty in our RSpec tests.

And the hardware is comparable?

Yes, = good question. =A0On the same model hardware, it's 20x initial, 4.5x cu= rrent. (2000, 440, vs <100 seconds)

=A0
> I'm fairly new to Ruby, but not to performance analysis, so I'= ve volunteered
> to follow up on this problem. =A0If I want to file/find a bug to assoc= iate
> with this problem, what sort of data should I be gathering, and how?
I have zero experience with Rails profiling, but maybe using the std<= br> lib profiler can give you a first data point.

Yeah my readings from Google say it's not easy to do what I need= to do. =A0I'm hoping someone could talk me down from the ledge and poi= nt me to a baked solution.
=A0

> Background/random data:
>
> We have 800+ tests, and on a good day they all run in a minute. =A0Som= e do no
> I/O (pure 'unit' test), most do light I/O (usually a couple cr= eate+reads to
> MongoDB or a render partial). =A0Only a few do any heavy lifting.
>
> I don't see any glaring hotspots. =A0Most of the tests seem to tak= e several
> times longer to run. =A0Dots just accumulate slower.

Just to exclude an obvious issue: do you have an AV engine up and
running on the Windows box? =A0If yes, excluding some directories might
help. =A0Some engines then still slow down operation so completely
switching off for the test might also be an option to determine
whether it's AV.

I'll entertain= the question, but let's take it as a given that turning off AV on Wind= ows is a very, very unwise action.=A0

Those number= s reflect my project directory already being excluded. =A0I'm not sure = I'm comfortable having Ruby whitelisted, though it's an interesting= data point: =A0

With Ruby exclusion, Regular is 1490 seconds, patched i= s 400 seconds. =A0



To exclude something else: is there concurrent activity that eats CPU
and / or IO bandwidth?

Not much. =A0Or = at least, there's not a lot of jitter in the numbers. =A0I just have Ru= byMine and a web browser (low CPU) open.
=A0

Kind regards

robert

--
remember.guy do |as, often| as.you_can - without end
http://blo= g.rubybestpractices.com/


As a mostly unrelated anecdote, the Su= bversion team had conversations like this for a long time, and resisted mak= ing changes for a number of years. =A0In the 1.7 release they made some pre= tty substantial changes to how they use the filesystem, and ended up with s= omething that was both simpler (for users) and happened to work a hell of a= lot faster on Windows.

It sounds like you guys are on that path at least. =A0



--
- Jason
--20cf3079c0bc8313fe04c972f0bd--