ruby-dev

MoonWolfです。

Yukihiro Matsumoto <matz@ruby-lang.org>:
> |# 別件になりますがユニコード対応してないから文字化けがたまに起きますね。
> |# レジストリをダンプするプログラムを組もうとして嵌りました^^;
> 
> ま、Unicode対応しているとうたってる言語でも文字化けは発生し
> てるみたいですしね。で、今回はどのような経緯で文字化けしまし
> たか。同じような問題が今後発生しないような仕組みを考えるヒン
> トにしたいので、ぜひ教えてください。

Windowsのレジストリ内部は全てユニコードで記録されています。
現状のwin32/registryで呼び出しているANSI版のAPIでは
ANSI(CP932)書き込み→内部(UCS2)→ANSI(CP932)読み込みという風になります。
CP932しか使わないのであれば全く問題はありません。

今回発生した文字化けはレジストリのバックアップ/リストア処理で起きました。
レジストリを読み出す段階でCP932に無い文字は'?'に置き換えられてしまいます。
そのため、読み込んだ値をそのまま書き戻すということができませんでした。

とりあえず明示的にユニコードで読み書きするメソッドを追加してみました。
readのUTF-8版としてread_utf8、writeにはwrite_utf8というのを追加しています。

Attachments (1)

reg.patch (5.35 KB, text/x-diff)

Index: registry.rb
===================================================================
--- registry.rb	(revision 12112)
+++ registry.rb	(working copy)
@@ -330,6 +330,48 @@
     include Constants
     include Enumerable
     
+    # UTF-16 to UTF-8
+    def self::u16tou8(u16str)
+      u8 = ''
+      sur = nil
+      u16str.unpack("S*").each {|x|
+        if sur
+          if (0xdc00..0xdfff).include?(x)
+            sur = (x - 0xdc00) + 0x10000
+            u8 << [x].pack("U")
+            sur = nil
+          else
+            raise "Invalid UTF-16"
+          end
+        else
+          if (0xd800..0xdbff).include?(x)
+            sur = (x - 0xd800) << 10
+          else
+            u8 << [x].pack("U")
+          end
+        end
+      }
+      u8.chop
+    end
+    
+    # UTF-8 to UTF-16
+    def self::u8tou16(u8str)
+      ary = u8str.unpack("U*")
+      u16 =""
+      ary.each {|ucs|
+        case ucs
+        when 0..0xffff
+          u16 << [ucs].pack("S")
+        when 0x10000..0x10ffff
+          ucs -= 0x10000
+          h = (ucs >> 10) & 0x3ff + 0xd800
+          l = ucs & 0x3ff + 0xdc00
+          u16 << [h,l].pack("SS")
+        end
+      }
+      u16
+    end
+    
     #
     # Error
     #
@@ -382,6 +424,8 @@
         %w/RegEnumKeyExA    LLPPLLLP     L/,
         %w/RegQueryValueExA LPLPPP       L/,
         %w/RegSetValueExA   LPLLPL       L/,
+        %w/RegQueryValueExW LPLPPP       L/,
+        %w/RegSetValueExW   LPLLPL       L/,
         %w/RegDeleteValue   LP           L/,
         %w/RegDeleteKey     LP           L/,
         %w/RegFlushKey      L            L/,
@@ -453,10 +497,26 @@
         [ unpackdw(type), data[0, unpackdw(size)] ]
       end
       
+      def QueryValueW(hkey, name)
+        type = packdw(0)
+        size = packdw(0)
+        name = Win32::Registry::u8tou16(name)
+        check RegQueryValueExW.call(hkey, name, 0, type, 0, size)
+        data = '\x00\x00' * unpackdw(size)
+        check RegQueryValueExW.call(hkey, name, 0, type, data, size)
+        [ unpackdw(type), Win32::Registry::u16tou8(data[0, unpackdw(size)]) ]
+      end
+      
       def SetValue(hkey, name, type, data, size)
         check RegSetValueExA.call(hkey, name, 0, type, data, size)
       end
       
+      def SetValueW(hkey, name, type, data, size)
+        name = Win32::Registry::u8tou16(name)
+        data = Win32::Registry::u8tou16(data)
+        check RegSetValueExW.call(hkey, name, 0, type, data, data.size)
+      end
+      
       def DeleteValue(hkey, name)
         check RegDeleteValue.call(hkey, name)
       end
@@ -511,11 +571,11 @@
     end
     
     def self.wtime2time(wtime)
-      Time.at((wtime - 116444736000000000) / 10000000)
+      Time.at((wtime - 116444736000000000).quo(10000000))
     end
     
     def self.time2wtime(time)
-      time.to_i * 10000000 + 116444736000000000
+      time.to_i * 10000000 + time.usec * 10 + 116444736000000000
     end
     
     #
@@ -687,6 +747,21 @@
       end
     end
     
+    def read_utf8(name, *rtype)
+      type, data = API.QueryValueW(@hkey, name)
+      unless rtype.empty? or rtype.include?(type)
+        raise TypeError, "Type mismatch (expect #{rtype.inspect} but #{type} present)"
+      end
+      case type
+      when REG_SZ, REG_EXPAND_SZ
+        [ type, data ]
+      when REG_MULTI_SZ
+        [ type, data.split(/\0/) ]
+      else
+        read(name, *rtype)
+      end
+    end
+    
     def [](name, *rtype)
       type, data = read(name, *rtype)
       case type
@@ -702,6 +777,14 @@
     def read_s(name)
       read(name, REG_SZ)[1]
     end
+
+    def read_s_utf8(name)
+      type, data = API.QueryValueW(@hkey, name)
+      if type != REG_SZ
+        raise TypeError, "Type mismatch (#{type} present)"
+      end
+      data
+    end
     
     def read_s_expand(name)
       type, data = read(name, REG_SZ, REG_EXPAND_SZ)
@@ -712,6 +795,15 @@
       end
     end
     
+    def read_s_expand_utf8(name)
+      type, data = API.QueryValueW(@hkey, name)
+      if type == REG_EXPAND_SZ
+        Registry.expand_environ(data)
+      else
+        data
+      end
+    end
+    
     def read_i(name)
       read(name, REG_DWORD, REG_DWORD_BIG_ENDIAN, REG_QWORD)[1]
     end
@@ -742,7 +834,30 @@
       end
       API.SetValue(@hkey, name, type, data, data.length)
     end
-    
+
+    def write_utf8(name, type, data)
+      case type
+      when REG_SZ, REG_EXPAND_SZ
+        API.SetValueW(@hkey, name, type, data, data.length)
+      when REG_MULTI_SZ
+        API.SetValueW(@hkey, name, type, data, data.length)
+      when REG_BINARY
+        data = data.to_s
+        API.SetValue(@hkey, name, type, data, data.length)
+      when REG_DWORD
+        data = API.packdw(data.to_i)
+        API.SetValue(@hkey, name, type, data, data.length)
+      when REG_DWORD_BIG_ENDIAN
+        data = [data.to_i].pack('N')
+        API.SetValue(@hkey, name, type, data, data.length)
+      when REG_QWORD
+        data = API.packqw(data.to_i)
+        API.SetValue(@hkey, name, type, data, data.length)
+      else
+        raise TypeError, "Unsupported type #{type}"
+      end
+    end
+
     def []=(name, rtype, value = nil)
       if value
         write name, rtype, value
@@ -765,6 +880,10 @@
       write name, REG_SZ, value.to_s
     end
     
+    def write_s_utf8(name, value)
+      write_utf8 name, REG_SZ, value.to_s
+    end
+    
     def write_i(name, value)
       write name, REG_DWORD, value.to_i
     end

Thread

Prev Next

In This Thread

Prev Next

[#30454] [ANN] CVSサービスの廃止 — Shugo Maeda <shugo@...>

[#30455] a[] += 1 で SEGV — "Hiroki Motoyoshi" <himotoyoshi.ml@...>

[#30465] Ruby 1.8.6 delayed for seven days — "Akinori MUSHA" <knu@...>

[#30476] Re: [ruby-cvs:19211] Ruby:r11975: * lib/fileutils.rb (touch): last #touch change causes error when :mtime option was not given. — Nobuyoshi Nakada <nobu@...>

[#30477] ThreadError on shell.rb — "Akira ODA" <oda.org@...>

[#30482] Ruby 1.8.6 preview3 has been released — "Akinori MUSHA" <knu@...>

[#30486] str.lines[1..-1] — Minero Aoki <aamine@...>

[#30487] Time#to_s format — OHARA Shigeki <os@...>

[#30500] MD5.md5 in RubyGems — Kazuhiro Yoshida <moriq@...>

[#30504] num_tとthread_initという名前について — "Yutaka Kanemoto" <kinpoco@...>

[#30505] [BUG?] SIGTERM の取り扱い — Hidetoshi NAGAI <nagai@...>

[#30507] monitor.rb for ruby-1.9 — keiju@... (Keiju ISHITSUKA)

[#30524] DL::Symbol#call dumps core — sheepman <sheepman@...>

[#30530] dl/import is not thread safe — sheepman <sheepman@...>

[#30531] DL::PtrData#to_a dumps core — sheepman <sheepman@...>

[#30532] DL::PtrData#inspect — sheepman <sheepman@...>

[#30536] Ruby 1.8.6 will be released in 24 hours — "Akinori MUSHA" <knu@...>

[#30537] Microsoft-IIS 上で cgi.rb がレスポンス行を２重に出力する — MIYASAKA Masaru <alkaid@...>

[#30547] dl memory leak — sheepman <sheepman@...>

[#30549] [ANN] Ruby 1.8.6 has been released — "Akinori MUSHA" <knu@...>

[#30551] rb_rescue2 with vm — wanabe <s.wanabe@...>

[#30553] help: lib/shell for ruby 1.9 — keiju@... (Keiju ISHITSUKA)

[#30554] rb_ary2cary in dl.c has a bug — sheepman <sheepman@...>

[#30556] ruby 1.8.6のdateとrails — akira yamada / やまだあきら <akira@...>

[#30563] ext/openssl のメンテナについて — "Akinori MUSHA" <knu@...>

[#30564] release code and revision number — Nobuyoshi Nakada <nobu@...>

[#30575] RSS::Parser.parseでSecurityError — Kazuhiro NISHIYAMA <zn@...>

[#30576] Changelog-1.8.6 — moonwolf@...

[#30581] [ruby-core:10580] Thread後のexecでENOTSUPPエラー対応 — "H.Holon" <holon@...>

[#30586] Etc.getgrgid が getgid の値に固定されています — 植田裕之 <ueda@...>

[#30593] [BUG] waitpid for ruby-trunk — keiju@... (Keiju ISHITSUKA)

[#30612] stdout flush in fork — keiju@... (Keiju ISHITSUKA)

[#30615] Etc モジュールのテスト — 植田 裕之 <ueda@...>

[#30618] possible bug in rb_detach_process() — "KUBO Takehiro" <kubo@...>

[#30620] DL.callback as free function causes errors — sheepman <sheepman@...>

[#30622] dl when $SAFE = 1 — sheepman <sheepman@...>

[#30624] test/rdoc occurs errors. — NAKANO Kouichi <curving@...>

[#30627] [ ruby-Bugs-9360 ] Matrix inverse problem — Yukihiro Matsumoto <matz@...>

[#30629] at_exit — keiju@... (Keiju ISHITSUKA)

[#30637] strlcpy and strlcat — "U.Nakamura" <usa@...>

[#30641] Re: [ruby-cvs:19321] Ruby:r12085: fixed the problem reported in — Urabe Shyouhei <shyouhei@...>

[#30648] YARVでのalias名の取得方法 — Yukihiro Matsumoto <matz@...>

[#30653] [BUG] queue 0x424010 freed with live thread(s) waiting — Masatoshi SEKI <m_seki@...>

[#30655] win32/registry wtime — moonwolf@...

[#30657] ロード済み拡張ライブラリの判別 — arton <artonx@...>

[#30667] $! assignment does not work properly — Yukihiro Matsumoto <matz@...>

[#30671] invalid CFP in rb_protect — wanabe <s.wanabe@...>

[#30672] fail to copy argv on Win32? — wanabe <s.wanabe@...>

[#30674] socket blocking on mingw32 — wanabe <s.wanabe@...>

[#30675] coredump on NetBSD with trunk — "U.Nakamura" <usa@...>

[#30676] excel is not closed in runtime using excel.Quit() — "Md. Ruhul Amin" <amin@...>

[ruby-dev:30670] Re: Unicode対応と文字化け(win32/registry)

Attachments (1)

Thread

In This Thread

[#30615] Etc モジュールのテスト — 植田裕之 <ueda@...>