[ruby-dev:28305] Re: SEGV with zlib
From:
"H.Yamamoto" <ocean@...2.ccsnet.ne.jp>
Date:
2006-02-06 10:23:53 UTC
List:
ruby-dev #28305
山本です。
>|確保した順番に関係なく、オブジェクトIDの小さいほうから free 関数を
>|呼んでいるように見えます。つまり
>|
>| A(0x03) -> B(0x02) -> C(0x04)
>|
>|のように参照されていると、B が解放されて A の free 関数が解放済みの B
>|にアクセスしてしまうと。
>
>そういうことです。で、それは仕様だと思ってます。
ううむ、仕様ですか。
>|zlib.c では gzfile_writer_end で一応 OBJ_IS_FREED という形でチェックしている
>|のですが、flags に出鱈目な値が入っているので、意図したように働いていません。
>
>開放されてりゃzeroになってるはずなんですがね。デタラメな値が
>入っているってことは再利用されちゃってるのかな。
gc_sweep がこうなっているので、
if (!(p->as.basic.flags & FL_MARK)) {
if (p->as.basic.flags) {
obj_free((VALUE)p);
}
if (need_call_final && FL_TEST(p, FL_FINALIZE)) {
p->as.free.flags = FL_MARK; /* remain marked */
p->as.free.next = final_list;
final_list = p;
}
else {
p->as.free.flags = 0;
p->as.free.next = freelist;
freelist = p;
}
n++;
}
解放された T_DATA も freelist に繋がれて、再利用の対象になっていると思います。
////////////////////////////////////
// 検証用パッチ
Index: gc.c
===================================================================
RCS file: /src/ruby/gc.c,v
retrieving revision 1.232
diff -u -w -b -p -r1.232 gc.c
--- gc.c 6 Feb 2006 02:49:14 -0000 1.232
+++ gc.c 6 Feb 2006 10:12:05 -0000
@@ -1061,11 +1061,19 @@ gc_sweep(void)
obj_free((VALUE)p);
}
if (need_call_final && FL_TEST(p, FL_FINALIZE)) {
+ if ((p->as.basic.flags & T_MASK) == T_DATA) {
+ printf("final_list =============> %p\n", p);
+ fflush(stdout);
+ }
p->as.free.flags = FL_MARK; /* remain marked */
p->as.free.next = final_list;
final_list = p;
}
else {
+ if ((p->as.basic.flags & T_MASK) == T_DATA) {
+ printf("freelist =============> %p\n", p);
+ fflush(stdout);
+ }
p->as.free.flags = 0;
p->as.free.next = freelist;
freelist = p;
Index: ext/zlib/zlib.c
===================================================================
RCS file: /src/ruby/ext/zlib/zlib.c,v
retrieving revision 1.29
diff -u -w -b -p -r1.29 zlib.c
--- ext/zlib/zlib.c 14 Dec 2005 16:37:24 -0000 1.29
+++ ext/zlib/zlib.c 6 Feb 2006 10:07:54 -0000
@@ -2284,6 +2284,8 @@ gzfile_writer_end(gz)
if (ZSTREAM_IS_FINALIZE(&gz->z)) {
if (NIL_P(gz->io)) return;
rb_warn("Zlib::GzipWriter object must be closed explicitly.");
+ printf("zlib -------------> %p, %d\n", gz->io, RBASIC(gz->io)->flags);
+ fflush(stdout);
if (!SPECIAL_CONST_P(gz->io) && OBJ_IS_FREED(gz->io)) {
aborted = 1;
}
////////////////////////////////////////////////////////
// 検証用スクリプト
require 'zlib'
class C
def write(str)
end
end
p o = C.new
100.times {
p o = Zlib::GzipWriter.new(o)
}
o.write "a"
$stdout.puts "--------------- 0"
$stdout.flush
o = nil
GC.start
$stdout.puts "--------------- 1"
$stdout.flush
////////////////////////////////////////////////////////
// 結果
#<C:0x2ab4be8>
#<Zlib::GzipWriter:0x2ab4b58>
#<Zlib::GzipWriter:0x2ab4b28>
#<Zlib::GzipWriter:0x2ab4af8>
#<Zlib::GzipWriter:0x2ab4ac8>
#<Zlib::GzipWriter:0x2ab4a98>
#<Zlib::GzipWriter:0x2ab4a68>
#<Zlib::GzipWriter:0x2ab4a38>
#<Zlib::GzipWriter:0x2ab4a08>
#<Zlib::GzipWriter:0x2ab49d8>
#<Zlib::GzipWriter:0x2ab49a8>
#<Zlib::GzipWriter:0x2ab4978>
#<Zlib::GzipWriter:0x2ab4948>
#<Zlib::GzipWriter:0x2ab4918>
#<Zlib::GzipWriter:0x2ab48e8>
#<Zlib::GzipWriter:0x2ab48b8>
#<Zlib::GzipWriter:0x2ab4888>
#<Zlib::GzipWriter:0x2ab4858>
#<Zlib::GzipWriter:0x2ab4828>
#<Zlib::GzipWriter:0x2ab47f8>
#<Zlib::GzipWriter:0x2ab47c8>
#<Zlib::GzipWriter:0x2ab4798>
#<Zlib::GzipWriter:0x2ab4768>
#<Zlib::GzipWriter:0x2ab4738>
#<Zlib::GzipWriter:0x2ab4708>
#<Zlib::GzipWriter:0x2ab46d8>
#<Zlib::GzipWriter:0x2ab46a8>
#<Zlib::GzipWriter:0x2ab4678>
#<Zlib::GzipWriter:0x2ab4648>
#<Zlib::GzipWriter:0x2ab4618>
#<Zlib::GzipWriter:0x2ab45e8>
#<Zlib::GzipWriter:0x2ab6dd8>
#<Zlib::GzipWriter:0x2ab6c70>
#<Zlib::GzipWriter:0x2ab6bc8>
#<Zlib::GzipWriter:0x2ab6b98>
#<Zlib::GzipWriter:0x2ab6b38>
#<Zlib::GzipWriter:0x2ab6a30>
#<Zlib::GzipWriter:0x2ab6940>
#<Zlib::GzipWriter:0x2ab6838>
#<Zlib::GzipWriter:0x2ab6760>
#<Zlib::GzipWriter:0x2ab66a0>
#<Zlib::GzipWriter:0x2ab6628>
#<Zlib::GzipWriter:0x2ab6568>
#<Zlib::GzipWriter:0x2ab6430>
#<Zlib::GzipWriter:0x2ab6400>
#<Zlib::GzipWriter:0x2ab63d0>
#<Zlib::GzipWriter:0x2ab63a0>
#<Zlib::GzipWriter:0x2ab6370>
#<Zlib::GzipWriter:0x2ab4b10>
#<Zlib::GzipWriter:0x2ab4ab0>
#<Zlib::GzipWriter:0x2ab4a50>
#<Zlib::GzipWriter:0x2ab49f0>
#<Zlib::GzipWriter:0x2ab4990>
#<Zlib::GzipWriter:0x2ab4930>
#<Zlib::GzipWriter:0x2ab48d0>
#<Zlib::GzipWriter:0x2ab4870>
#<Zlib::GzipWriter:0x2ab4810>
#<Zlib::GzipWriter:0x2ab47b0>
#<Zlib::GzipWriter:0x2ab4750>
#<Zlib::GzipWriter:0x2ab46f0>
#<Zlib::GzipWriter:0x2ab4690>
#<Zlib::GzipWriter:0x2ab6c88>
#<Zlib::GzipWriter:0x2ab6bb0>
#<Zlib::GzipWriter:0x2ab6aa8>
#<Zlib::GzipWriter:0x2ab6928>
#<Zlib::GzipWriter:0x2ab6748>
#<Zlib::GzipWriter:0x2ab65c8>
#<Zlib::GzipWriter:0x2ab6418>
#<Zlib::GzipWriter:0x2ab63b8>
#<Zlib::GzipWriter:0x2ab4b40>
#<Zlib::GzipWriter:0x2ab4a80>
#<Zlib::GzipWriter:0x2ab49c0>
#<Zlib::GzipWriter:0x2ab4900>
#<Zlib::GzipWriter:0x2ab4840>
#<Zlib::GzipWriter:0x2ab4780>
#<Zlib::GzipWriter:0x2ab4660>
#<Zlib::GzipWriter:0x2ab4600>
#<Zlib::GzipWriter:0x2ab45b8>
#<Zlib::GzipWriter:0x2ab4588>
#<Zlib::GzipWriter:0x2ab4558>
#<Zlib::GzipWriter:0x2ab4528>
#<Zlib::GzipWriter:0x2ab44f8>
#<Zlib::GzipWriter:0x2ab44c8>
#<Zlib::GzipWriter:0x2ab4498>
#<Zlib::GzipWriter:0x2ab4468>
#<Zlib::GzipWriter:0x2ab4438>
#<Zlib::GzipWriter:0x2ab4408>
#<Zlib::GzipWriter:0x2ab43d8>
#<Zlib::GzipWriter:0x2ab43a8>
#<Zlib::GzipWriter:0x2ab4378>
#<Zlib::GzipWriter:0x2ab4348>
#<Zlib::GzipWriter:0x2ab6bf8>
#<Zlib::GzipWriter:0x2ab6988>
#<Zlib::GzipWriter:0x2ab6640>
#<Zlib::GzipWriter:0x2ab63e8>
#<Zlib::GzipWriter:0x2ab4ae0>
#<Zlib::GzipWriter:0x2ab4960>
#<Zlib::GzipWriter:0x2ab47e0>
#<Zlib::GzipWriter:0x2ab46c0>
#<Zlib::GzipWriter:0x2ab45d0>
#<Zlib::GzipWriter:0x2ab4570>
--------------- 0
zlib -------------> 02AB4378, 18
freelist =============> 02AB4348
zlib -------------> 02AB43A8, 18
freelist =============> 02AB4378
zlib -------------> 02AB43D8, 18
freelist =============> 02AB43A8
zlib -------------> 02AB4408, 18
freelist =============> 02AB43D8
zlib -------------> 02AB4438, 18
freelist =============> 02AB4408
zlib -------------> 02AB4468, 18
freelist =============> 02AB4438
zlib -------------> 02AB4498, 18
freelist =============> 02AB4468
zlib -------------> 02AB44C8, 18
freelist =============> 02AB4498
zlib -------------> 02AB44F8, 18
freelist =============> 02AB44C8
zlib -------------> 02AB4528, 18
freelist =============> 02AB44F8
zlib -------------> 02AB4558, 18
freelist =============> 02AB4528
zlib -------------> 02AB4588, 18
freelist =============> 02AB4558
zlib -------------> 02AB45D0, 18
freelist =============> 02AB4570
zlib -------------> 02AB45B8, 18
freelist =============> 02AB4588
zlib -------------> 02AB4600, 18
freelist =============> 02AB45B8
zlib -------------> 02AB46C0, 18
freelist =============> 02AB45D0
zlib -------------> 02AB4618, 18
freelist =============> 02AB45E8
zlib -------------> 02AB4660, 18
freelist =============> 02AB4600
zlib -------------> 02AB4648, 18
freelist =============> 02AB4618
zlib -------------> 02AB4678, 18
freelist =============> 02AB4648
zlib -------------> 02AB4780, 18
freelist =============> 02AB4660
zlib -------------> 02AB46A8, 18
freelist =============> 02AB4678
zlib -------------> 02AB46F0, 18
freelist =============> 02AB4690
zlib -------------> 02AB46D8, 18
freelist =============> 02AB46A8
zlib -------------> 02AB47E0, 18
freelist =============> 02AB46C0
zlib -------------> 02AB4708, 18
freelist =============> 02AB46D8
zlib -------------> 02AB4750, 18
freelist =============> 02AB46F0
zlib -------------> 02AB4738, 18
freelist =============> 02AB4708
zlib -------------> 02AB4768, 18
freelist =============> 02AB4738
zlib -------------> 02AB47B0, 18
freelist =============> 02AB4750
zlib -------------> 02AB4798, 18
freelist =============> 02AB4768
zlib -------------> 02AB4840, 18
freelist =============> 02AB4780
zlib -------------> 02AB47C8, 18
freelist =============> 02AB4798
zlib -------------> 02AB4810, 18
freelist =============> 02AB47B0
zlib -------------> 02AB47F8, 18
freelist =============> 02AB47C8
zlib -------------> 02AB4960, 18
freelist =============> 02AB47E0
zlib -------------> 02AB4828, 18
freelist =============> 02AB47F8
zlib -------------> 02AB4870, 18
freelist =============> 02AB4810
zlib -------------> 02AB4858, 18
freelist =============> 02AB4828
zlib -------------> 02AB4900, 18
freelist =============> 02AB4840
zlib -------------> 02AB4888, 18
freelist =============> 02AB4858
zlib -------------> 02AB48D0, 18
freelist =============> 02AB4870
zlib -------------> 02AB48B8, 18
freelist =============> 02AB4888
zlib -------------> 02AB48E8, 18
freelist =============> 02AB48B8
zlib -------------> 02AB4930, 18
freelist =============> 02AB48D0
zlib -------------> 02AB4918, 18
freelist =============> 02AB48E8
zlib -------------> 02AB49C0, 18
freelist =============> 02AB4900
zlib -------------> 02AB4948, 18
freelist =============> 02AB4918
zlib -------------> 02AB4990, 18
freelist =============> 02AB4930
zlib -------------> 02AB4978, 18
freelist =============> 02AB4948
zlib -------------> 02AB4AE0, 18
freelist =============> 02AB4960
zlib -------------> 02AB49A8, 18
freelist =============> 02AB4978
zlib -------------> 02AB49F0, 18
freelist =============> 02AB4990
zlib -------------> 02AB49D8, 18
freelist =============> 02AB49A8
zlib -------------> 02AB4A80, 18
freelist =============> 02AB49C0
zlib -------------> 02AB4A08, 18
freelist =============> 02AB49D8
zlib -------------> 02AB4A50, 18
freelist =============> 02AB49F0
zlib -------------> 02AB4A38, 18
freelist =============> 02AB4A08
zlib -------------> 02AB4A68, 18
freelist =============> 02AB4A38
zlib -------------> 02AB4AB0, 18
freelist =============> 02AB4A50
zlib -------------> 02AB4A98, 18
freelist =============> 02AB4A68
zlib -------------> 02AB4B40, 18
freelist =============> 02AB4A80
zlib -------------> 02AB4AC8, 18
freelist =============> 02AB4A98
zlib -------------> 02AB4B10, 18
freelist =============> 02AB4AB0
zlib -------------> 02AB4AF8, 18
freelist =============> 02AB4AC8
zlib -------------> 02AB63E8, 18
freelist =============> 02AB4AE0
zlib -------------> 02AB4B28, 18
freelist =============> 02AB4AF8
zlib -------------> 02AB6370, 18
freelist =============> 02AB4B10
zlib -------------> 02AB4B58, 18
freelist =============> 02AB4B28
zlib -------------> 02AB63B8, 18
freelist =============> 02AB4B40
zlib -------------> 02AB4BE8, 34
freelist =============> 02AB4B58
zlib -------------> 02AB63A0, 18
freelist =============> 02AB6370
zlib -------------> 02AB63D0, 18
freelist =============> 02AB63A0
zlib -------------> 02AB6418, 18
freelist =============> 02AB63B8
zlib -------------> 02AB6400, 18
freelist =============> 02AB63D0
zlib -------------> 02AB6640, 18
freelist =============> 02AB63E8
zlib -------------> 02AB6430, 18
freelist =============> 02AB6400
zlib -------------> 02AB65C8, 18
freelist =============> 02AB6418
zlib -------------> 02AB6568, 18
freelist =============> 02AB6430
zlib -------------> 02AB6628, 18
freelist =============> 02AB6568
zlib -------------> 02AB6748, 18
freelist =============> 02AB65C8
zlib -------------> 02AB66A0, 18
freelist =============> 02AB6628
zlib -------------> 02AB6988, 18
freelist =============> 02AB6640
zlib -------------> 02AB6760, 18
freelist =============> 02AB66A0
zlib -------------> 02AB6928, 18
freelist =============> 02AB6748
zlib -------------> 02AB6838, 18
freelist =============> 02AB6760
zlib -------------> 02AB6940, 18
freelist =============> 02AB6838
zlib -------------> 02AB6AA8, 18
freelist =============> 02AB6928
zlib -------------> 02AB6A30, 18
freelist =============> 02AB6940
zlib -------------> 02AB6BF8, 18
freelist =============> 02AB6988
zlib -------------> 02AB6B38, 18
freelist =============> 02AB6A30
zlib -------------> 02AB6BB0, 18
freelist =============> 02AB6AA8
zlib -------------> 02AB6B98, 18
freelist =============> 02AB6B38
zlib -------------> 02AB6BC8, 18
freelist =============> 02AB6B98
zlib -------------> 02AB6C88, 18
freelist =============> 02AB6BB0
zlib -------------> 02AB6C70, 18
freelist =============> 02AB6BC8
zlib -------------> 02AB4348, 582687
この 02AB4348 は前に freelist ====> に出てくる値です。おそらく、
T_DATA の解放関数でオブジェクトが新規作成され、そのとき再利用
されるのでしょう。（それにしては flag の値がおかしい気もしますが）
>|でも、これって zlib.c のバグというより GC のバグのような・・・
>
>これをバグ認定するとすると、これから解放するオブジェクトに対
>して、再度マークスイープを実行して...、とかいうようなことを
>する必要が出てくるわけなんですが、それだけのGCの速度低下を許
>容できる人はいないように思います。私が思いつかないだけで高速
>な実装が可能なのかもしれませんが。
私も GC に詳しいわけではないのですが、Boehm GC はどうなってる
んでしょう。
># えーと、今思いついたのは、たとえばスイープの前にマークされ
># てないT_DATAでもmark関数を呼んでおく(つまり、T_DATAからマー
># クされていると解放が1GCサイクル遅くなる)という手はあります
># けど、あんまりやりたくないです。
>
>ということで、私としてはzlib.cに「finalizeの中で解放されてい
>るかもしれないオブジェクトには触らないほうがよい(メソッド呼
>び出すとかもってのほか)」とアドバイスしたいところであります。
解放関数では Ruby オブジェクトを操作できない、ということでしょうか。
それならそれで明快な気もします。(zlib はバッファに RString を
使っているので、C のヒープを使うべきなんでしょうか）