From: eregontp@...
Date: 2019-01-21T11:29:16+00:00
Subject: [ruby-core:91211] [Ruby trunk Bug#15552] app_aobench.rb is broken

Issue #15552 has been updated by Eregon (Benoit Daloze).


@mame Thanks for the quick fix!
With srand(0), I think we could add some verification, e.g., by computing a checksum of the image.

----------------------------------------
Bug #15552: app_aobench.rb is broken
https://bugs.ruby-lang.org/issues/15552#change-76449

* Author: Eregon (Benoit Daloze)
* Status: Closed
* Priority: Normal
* Assignee: mame (Yusuke Endoh)
* Target version: 
* ruby -v: ruby 2.6.0p0 (2018-12-25 revision 66547) [x86_64-linux]
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
Specifically, the image generated is incorrect.
When modifying the footer of the benchmark like this:

```ruby
alias printf_orig printf
def printf *args
  $fp.printf(*args)
end

File.open("ao.ppm", "w") do |fp|
  $fp = fp
  printf("P6\n")
  printf("%d %d\n", IMAGE_WIDTH, IMAGE_HEIGHT)
  printf("255\n")
  Scene.new.render(IMAGE_WIDTH, IMAGE_HEIGHT, NSUBSAMPLES)
end

undef printf
alias printf printf_orig
```

Here is the expected image:
![ao_ref](https://user-images.githubusercontent.com/168854/51442303-dd6f8680-1cdb-11e9-89ac-e88773a384c8.png)
I get this image with MRI 2.6.0:
![ao_mri_2_6_0](https://user-images.githubusercontent.com/168854/51442292-c7fa5c80-1cdb-11e9-9146-2ec3f447d479.png)
And interestingly, TruffleRuby 1.0.0-rc11 renders an image closer to the expected one:
![ao_tr_rc11](https://user-images.githubusercontent.com/168854/51442298-d5174b80-1cdb-11e9-97f6-0b1fc59fcddf.png)

I guess this might be both an interpreter bug, and possibly also a bug in the benchmark code.

I think every benchmark should have some validation, otherwise it's prone to measure something unexpected like this.



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>