From: Daniel Schierbeck <daniel.schierbeck@...>
Date: 2007-03-09T20:40:24+09:00
Subject: Re: A better benchmarking syntax (was: Automatic Benchmark	Iterations)

On Fri, 2007-03-09 at 13:43 +0900, M. Edward (Ed) Borasky wrote:
> pat eyler wrote:
> > On 3/8/07, Daniel Schierbeck <daniel.schierbeck@gmail.com> wrote:
> >> I've given this some thought, and it seems like a much bigger project
> >> than I immediately imagined -- i was surprised to find a much greater
> >> need of features than anticipated. I am currently pondering whether to
> >> do this as a Summer of Code project (that is, if I'm lucky enough to get
> >> elected.)
> >
> > I think this would make an awesome summer of code project, and would
> > benefit a lot of other projects downstream.  Please do sign up.
> >
> >
> >>
> >> Unfortunately, I do not have great experience with benchmarking, neither
> >> the underlying math (well, I have some), nor the actual need of the
> >> people who would use an extended framework.
> >
> > The math isn't my strongpoint either (on the statistical side), but 
> > I'm happy
> > to kibbitz (and to try to engage other folks who can help) if you'd like.
> >
> >
> >>
> >> The only thing I have ever used benchmarking for was to compare the
> >> speed of two methods with the same functionality (e.g. sorting).
> >>
> >> So what I initially want to ask you guys is this: what else do you use
> >> benchmarking for? Do you benchmark your entire application, and compare
> >> results from different revisions? I'd really like to know.
> >
> > well, the JRuby/rubinius/etc folks could sure use a common benchmarking
> > suite to look at sets of results from different builds (and different
> > platforms).
> > I do some revision v revision benchmarking of apps as well.
> >
> >>
> >> Second, if there are others who would be interested in contributing to
> >> any project, given that the need is real, I would be delighted to hear
> >> from them!
> >>
> >
> > I'll weigh in some more as scope and plans become more clear (and after
> > I get the other eleventy-six things off my plate).
> >
> >>
> >> Cheers,
> >> Daniel
> >>
> >>
> >>
> >
> >
> I could be persuaded to do a little mentoring in both mathematics and 
> benchmarking. Just don't ask me to come up with clever Ruby tricks. I 
> know a *lot* more about mathematics and benchmarking. :)

Lovely! This really seems like something that could take off!


Cheers,
Daniel