From: jason marshall <jdmarshall@...>
Date: 2012-09-12T05:33:02+09:00
Subject: Re: Looking for advice on how to report performance issues

--20cf307d03b0ea222504c972faef
Content-Type: text/plain; charset=ISO-8859-1

Oops, correction: 2000s vs 440s vs ~60s

On Tue, Sep 11, 2012 at 1:30 PM, jason marshall <jdmarshall@gmail.com>wrote:

> Comments inline
>
> On Mon, Sep 10, 2012 at 11:27 PM, Robert Klemme <
> shortcutter@googlemail.com> wrote:
>
>> On Mon, Sep 10, 2012 at 10:16 PM, jason marshall <jdmarshall@gmail.com>
>> wrote:
>> > We have a Rails app running on Ruby 1.9.3, and a under Windows we're
>> seeing
>> > a 20x time penalty in our RSpec tests.
>>
>> And the hardware is comparable?
>
>
> Yes, good question.  On the same model hardware, it's 20x initial, 4.5x
> current. (2000, 440, vs <100 seconds)
>
>
>
>>  > I'm fairly new to Ruby, but not to performance analysis, so I've
>> volunteered
>> > to follow up on this problem.  If I want to file/find a bug to associate
>> > with this problem, what sort of data should I be gathering, and how?
>>
>> I have zero experience with Rails profiling, but maybe using the std
>> lib profiler can give you a first data point.
>>
>
> Yeah my readings from Google say it's not easy to do what I need to do.
>  I'm hoping someone could talk me down from the ledge and point me to a
> baked solution.
>
>
>>
>> > Background/random data:
>> >
>> > We have 800+ tests, and on a good day they all run in a minute.  Some
>> do no
>> > I/O (pure 'unit' test), most do light I/O (usually a couple
>> create+reads to
>> > MongoDB or a render partial).  Only a few do any heavy lifting.
>> >
>> > I don't see any glaring hotspots.  Most of the tests seem to take
>> several
>> > times longer to run.  Dots just accumulate slower.
>>
>> Just to exclude an obvious issue: do you have an AV engine up and
>> running on the Windows box?  If yes, excluding some directories might
>> help.  Some engines then still slow down operation so completely
>> switching off for the test might also be an option to determine
>> whether it's AV.
>>
>
> I'll entertain the question, but let's take it as a given that turning off
> AV on Windows is a very, very unwise action.
>
> Those numbers reflect my project directory already being excluded.  I'm
> not sure I'm comfortable having Ruby whitelisted, though it's an
> interesting data point:
>
> With Ruby exclusion, Regular is 1490 seconds, patched is 400 seconds.
>
>
>
>> To exclude something else: is there concurrent activity that eats CPU
>> and / or IO bandwidth?
>>
>
> Not much.  Or at least, there's not a lot of jitter in the numbers.  I
> just have RubyMine and a web browser (low CPU) open.
>
>
>>
>> Kind regards
>>
>> robert
>>
>> --
>> remember.guy do |as, often| as.you_can - without end
>> http://blog.rubybestpractices.com/
>>
>>
> As a mostly unrelated anecdote, the Subversion team had conversations like
> this for a long time, and resisted making changes for a number of years.
>  In the 1.7 release they made some pretty substantial changes to how they
> use the filesystem, and ended up with something that was both simpler (for
> users) and happened to work a hell of a lot faster on Windows.
>
> It sounds like you guys are on that path at least.
>
>
>
> --
> - Jason
>



-- 
- Jason

--20cf307d03b0ea222504c972faef
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Oops, correction: 2000s vs 440s vs ~60s<br><br><div class=3D"gmail_quote">O=
n Tue, Sep 11, 2012 at 1:30 PM, jason marshall <span dir=3D"ltr">&lt;<a hre=
f=3D"mailto:jdmarshall@gmail.com" target=3D"_blank">jdmarshall@gmail.com</a=
>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">Comments inline<br><br><div class=3D"gmail_q=
uote"><div class=3D"im">On Mon, Sep 10, 2012 at 11:27 PM, Robert Klemme <sp=
an dir=3D"ltr">&lt;<a href=3D"mailto:shortcutter@googlemail.com" target=3D"=
_blank">shortcutter@googlemail.com</a>&gt;</span> wrote:<br>

<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div>On Mon, Sep 10, 2012 at 10:16 PM, jason=
 marshall &lt;<a href=3D"mailto:jdmarshall@gmail.com" target=3D"_blank">jdm=
arshall@gmail.com</a>&gt; wrote:<br>


&gt; We have a Rails app running on Ruby 1.9.3, and a under Windows we&#39;=
re seeing<br>
&gt; a 20x time penalty in our RSpec tests.<br>
<br>
</div>And the hardware is comparable?</blockquote><div><br></div></div><div=
>Yes, good question. =A0On the same model hardware, it&#39;s 20x initial, 4=
.5x current. (2000, 440, vs &lt;100 seconds)</div><div class=3D"im"><div><b=
r>
</div><div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0=
 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
&gt; I&#39;m fairly new to Ruby, but not to performance analysis, so I&#39;=
ve volunteered<br>
&gt; to follow up on this problem. =A0If I want to file/find a bug to assoc=
iate<br>
&gt; with this problem, what sort of data should I be gathering, and how?<b=
r>
<br>
</div>I have zero experience with Rails profiling, but maybe using the std<=
br>
lib profiler can give you a first data point.<br></blockquote><div><br></di=
v></div><div>Yeah my readings from Google say it&#39;s not easy to do what =
I need to do. =A0I&#39;m hoping someone could talk me down from the ledge a=
nd point me to a baked solution.</div>
<div class=3D"im">
<div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;=
border-left:1px #ccc solid;padding-left:1ex">
<div><br>
&gt; Background/random data:<br>
&gt;<br>
&gt; We have 800+ tests, and on a good day they all run in a minute. =A0Som=
e do no<br>
&gt; I/O (pure &#39;unit&#39; test), most do light I/O (usually a couple cr=
eate+reads to<br>
&gt; MongoDB or a render partial). =A0Only a few do any heavy lifting.<br>
&gt;<br>
&gt; I don&#39;t see any glaring hotspots. =A0Most of the tests seem to tak=
e several<br>
&gt; times longer to run. =A0Dots just accumulate slower.<br>
<br>
</div>Just to exclude an obvious issue: do you have an AV engine up and<br>
running on the Windows box? =A0If yes, excluding some directories might<br>
help. =A0Some engines then still slow down operation so completely<br>
switching off for the test might also be an option to determine<br>
whether it&#39;s AV.<br></blockquote><div><br></div></div><div>I&#39;ll ent=
ertain the question, but let&#39;s take it as a given that turning off AV o=
n Windows is a very, very unwise action.=A0</div><div><br></div><div>Those =
numbers reflect my project directory already being excluded. =A0I&#39;m not=
 sure I&#39;m comfortable having Ruby whitelisted, though it&#39;s an inter=
esting data point: =A0</div>

<div><br></div><div>With Ruby exclusion, Regular is 1490 seconds, patched i=
s 400 seconds. =A0</div><div class=3D"im"><div><br></div><div><br></div><bl=
ockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #=
ccc solid;padding-left:1ex">


<br>
To exclude something else: is there concurrent activity that eats CPU<br>
and / or IO bandwidth?<br></blockquote><div><br></div></div><div>Not much. =
=A0Or at least, there&#39;s not a lot of jitter in the numbers. =A0I just h=
ave RubyMine and a web browser (low CPU) open.</div><div class=3D"im"><div>=
=A0</div>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">

<br>
Kind regards<br>
<span><font color=3D"#888888"><br>
robert<br>
<br>
--<br>
remember.guy do |as, often| as.you_can - without end<br>
<a href=3D"http://blog.rubybestpractices.com/" target=3D"_blank">http://blo=
g.rubybestpractices.com/</a><br>
<br>
</font></span></blockquote></div></div><br>As a mostly unrelated anecdote, =
the Subversion team had conversations like this for a long time, and resist=
ed making changes for a number of years. =A0In the 1.7 release they made so=
me pretty substantial changes to how they use the filesystem, and ended up =
with something that was both simpler (for users) and happened to work a hel=
l of a lot faster on Windows.<div>

<br></div><div>It sounds like you guys are on that path at least. =A0<span =
class=3D"HOEnZb"><font color=3D"#888888"><br><div><br></div><div><br clear=
=3D"all"><div><br></div>-- <br>- Jason<br>
</div></font></span></div>
</blockquote></div><br><br clear=3D"all"><div><br></div>-- <br>- Jason<br>

--20cf307d03b0ea222504c972faef--