[#32498] Re: [ruby-cvs:21399] Ruby:r14162 (trunk): * parse.y (expr): redefinable not (!) operator. — SASADA Koichi <ko1@...>

 ささだです.

9 messages 2007/12/09

[#32512] Re: [ruby-cvs:21409] Ruby:r14172 (trunk): * transcode.c: new file to provide encoding conversion features. — Nobuyoshi Nakada <nobu@...>

なかだです。

33 messages 2007/12/10
[#32520] Re: [ruby-cvs:21409] Ruby:r14172 (trunk): * transcode.c: new file to provide encoding conversion features. — Martin Duerst <duerst@...> 2007/12/10

中田さん、こんにちは。

[#32527] Re: [ruby-cvs:21409] Ruby:r14172 (trunk): * transcode.c: new file to provide encoding conversion features. — Nobuyoshi Nakada <nobu@...> 2007/12/10

なかだです。

[#32535] Re: [ruby-cvs:21409] Ruby:r14172 (trunk): * transcode.c: new file to provide encoding conversion features. — Yukihiro Matsumoto <matz@...> 2007/12/11

まつもと ゆきひろです

[#32537] Re: [ruby-cvs:21409] Ruby:r14172 (trunk): * transcode.c: new file to provide encoding conversion features. — Martin Duerst <duerst@...> 2007/12/11

At 15:33 07/12/11, Yukihiro Matsumoto wrote:

[#32538] Re: [ruby-cvs:21409] Ruby:r14172 (trunk): * transcode.c: new file to provide encoding conversion features. — Yukihiro Matsumoto <matz@...> 2007/12/11

まつもと ゆきひろです

[#32539] Re: [ruby-cvs:21409] Ruby:r14172 (trunk): * transcode.c: new file to provide encoding conversion features. — Nobuyoshi Nakada <nobu@...> 2007/12/11

なかだです。

[#32550] Binary String — Hidetoshi NAGAI <nagai@...>

永井@知能.九工大です.

204 messages 2007/12/12
[#32551] Re: Binary String — Yukihiro Matsumoto <matz@...> 2007/12/12

まつもと ゆきひろです

[#32552] Re: Binary String — Hidetoshi NAGAI <nagai@...> 2007/12/12

永井@知能.九工大です.

[#32553] Re: Binary String — Yukihiro Matsumoto <matz@...> 2007/12/12

まつもと ゆきひろです

[#32560] Re: Binary String — Hidetoshi NAGAI <nagai@...> 2007/12/12

永井@知能.九工大です.

[#32561] Re: Binary String — Nobuyoshi Nakada <nobu@...> 2007/12/12

なかだです。

[#33018] Re: Binary String — Hidetoshi NAGAI <nagai@...> 2008/01/11

永井@知能.九工大です.

[#33019] Re: Binary String — Tanaka Akira <akr@...> 2008/01/11

In article <20080111.171950.78716471.nagai@ai.kyutech.ac.jp>,

[#33024] Re: Binary String — Hidetoshi NAGAI <nagai@...> 2008/01/11

永井@知能.九工大です.

[#33027] Re: Binary String — Tanaka Akira <akr@...> 2008/01/11

In article <20080111.184442.74744388.nagai@ai.kyutech.ac.jp>,

[#33041] Re: Binary String — Hidetoshi NAGAI <nagai@...> 2008/01/11

永井@知能.九工大です.

[#33047] Re: Binary String — Tanaka Akira <akr@...> 2008/01/11

In article <20080112.004750.74741782.nagai@ai.kyutech.ac.jp>,

[#33055] Re: Binary String — Hidetoshi NAGAI <nagai@...> 2008/01/12

永井@知能.九工大です.

[#33080] Re: Binary String — Tanaka Akira <akr@...> 2008/01/13

In article <20080112.100830.112615025.nagai@ai.kyutech.ac.jp>,

[#33104] Re: Binary String — Hidetoshi NAGAI <nagai@...> 2008/01/14

永井@知能.九工大です.

[#33108] Re: Binary String — "NARUSE, Yui" <naruse@...> 2008/01/15

成瀬です。

[#33121] Re: Binary String — Hidetoshi NAGAI <nagai@...> 2008/01/15

永井@知能.九工大です.

[#33123] Re: Binary String — "NARUSE, Yui" <naruse@...> 2008/01/15

成瀬です。

[#33127] Re: Binary String — Hidetoshi NAGAI <nagai@...> 2008/01/16

永井@知能.九工大です.

[#33138] Re: Binary String — "NARUSE, Yui" <naruse@...> 2008/01/16

成瀬です。

[#33147] Re: Binary String — Hidetoshi NAGAI <nagai@...> 2008/01/17

永井@知能.九工大です.

[#33152] Re: Binary String — "NARUSE, Yui" <naruse@...> 2008/01/17

成瀬です。

[#33153] Re: Binary String — 遊楽庵 <yu_raku_an@...> 2008/01/17

遊楽庵です。

[#33154] Re: Binary String — "NARUSE, Yui" <naruse@...> 2008/01/17

成瀬です。

[#33157] Re: Binary String — Yukihiro Matsumoto <matz@...> 2008/01/17

まつもと ゆきひろです

[#33330] Re: Binary String — "NARUSE, Yui" <naruse@...> 2008/01/23

成瀬です。

[#33336] Re: Binary String — Tanaka Akira <akr@...> 2008/01/23

In article <47975933.8010907@airemix.com>,

[#33337] Re: Binary String — Yukihiro Matsumoto <matz@...> 2008/01/23

まつもと ゆきひろです

[#33346] Re: Binary String — "U.Nakamura" <usa@...> 2008/01/24

こんにちは、なかむら(う)です。

[#33348] Re: Binary String — Yukihiro Matsumoto <matz@...> 2008/01/24

まつもと ゆきひろです

[#33352] Re: Binary String — "U.Nakamura" <usa@...> 2008/01/24

こんにちは、なかむら(う)です。

[#33353] Re: Binary String — Yukihiro Matsumoto <matz@...> 2008/01/24

まつもと ゆきひろです

[#33122] Re: Binary String — Tanaka Akira <akr@...> 2008/01/15

In article <20080115.024201.41653719.nagai@ai.kyutech.ac.jp>,

[#33126] Re: Binary String — Hidetoshi NAGAI <nagai@...> 2008/01/16

永井@知能.九工大です.

[#33151] Re: Binary String — Tanaka Akira <akr@...> 2008/01/17

In article <20080116.102057.41656941.nagai@ai.kyutech.ac.jp>,

[#33160] Re: Binary String — Hidetoshi NAGAI <nagai@...> 2008/01/17

永井@知能.九工大です.

[#33165] Re: Binary String — Tanaka Akira <akr@...> 2008/01/18

In article <20080117.233832.74721189.nagai@ai.kyutech.ac.jp>,

[#33188] Re: Binary String — Hidetoshi NAGAI <nagai@...> 2008/01/19

永井@知能.九工大です.

[#33193] Re: Binary String — Yukihiro Matsumoto <matz@...> 2008/01/19

まつもと ゆきひろです

[#33202] Re: Binary String — Hidetoshi NAGAI <nagai@...> 2008/01/19

永井@知能.九工大です.

[#33230] Re: Binary String — Yukihiro Matsumoto <matz@...> 2008/01/20

まつもと ゆきひろです

[#33236] Re: Binary String — Hidetoshi NAGAI <nagai@...> 2008/01/21

永井@知能.九工大です.

[#33238] Re: Binary String — SASADA Koichi <ko1@...> 2008/01/21

 m17n には近づかないようにしているささだです。

[#33241] Re: Binary String — "NARUSE, Yui" <naruse@...> 2008/01/21

成瀬です。

[#33248] Re: Binary String — Yukihiro Matsumoto <matz@...> 2008/01/21

まつもと ゆきひろです

[#33281] Re: Binary String — Hidetoshi NAGAI <nagai@...> 2008/01/22

永井@知能.九工大です.

[#33285] Re: Binary String — "NARUSE, Yui" <naruse@...> 2008/01/22

成瀬です。

[#33318] Re: Binary String — Hidetoshi NAGAI <nagai@...> 2008/01/23

永井@知能.九工大です.

[#33186] Ruby1.9 String バイト列へのインデックス アクセス — "Hisanori Kiryu" <hkiryu@...> 2008/01/19

長文失礼します。

[#33192] Re: Ruby1.9 String バイト列へのインデックス アクセス — Yukihiro Matsumoto <matz@...> 2008/01/19

まつもと ゆきひろです

[#33195] Re: Ruby1.9 String バイト列へのインデックス アクセス — rubikitch@... 2008/01/19

From: Yukihiro Matsumoto <matz@ruby-lang.org>

[#33199] Re: Ruby1.9 String バイト列へのインデックス アクセス — "NARUSE, Yui" <naruse@...> 2008/01/19

成瀬です。

[#33020] Re: Binary String — "NARUSE, Yui" <naruse@...> 2008/01/11

成瀬です。

[#32610] 1.9.1 issues left (as of 12/15) — Yukihiro Matsumoto <matz@...>

まつもと ゆきひろです

14 messages 2007/12/15

[#32715] issues left as of 12/25 2:00am JST — Yukihiro Matsumoto <matz@...>

まつもと ゆきひろです

41 messages 2007/12/24
[#32738] issues left as of 12/25 noon JST — Yukihiro Matsumoto <matz@...> 2007/12/25

まつもと ゆきひろです

[#32739] Re: issues left as of 12/25 noon JST — Yukihiro Matsumoto <matz@...> 2007/12/25

まつもと ゆきひろです

[#32791] Re: [ruby-list:44387] [ANN] Ruby 1.9.0 is released — SASADA Koichi <ko1@...>

 ささだです。

21 messages 2007/12/25

[#32823] class TimeSpan — "NARUSE, Yui" <naruse@...>

成瀬です。

18 messages 2007/12/27

[#32843] Windowでのデフォルトエンコーディング — KIMURA Koichi <kimura.koichi@...>

木村です。

30 messages 2007/12/28
[#32845] Re: Windowでのデフォルトエンコーディング — "U.Nakamura" <usa@...> 2007/12/28

こんにちは、なかむら(う)です。

[#32851] Re: Window でのデフォルトエンコーディング — Martin Duerst <duerst@...> 2007/12/28

At 13:55 07/12/28, U.Nakamura wrote:

[#32853] Re: Windowでのデフォルトエンコーディング — "NARUSE, Yui" <naruse@...> 2007/12/28

U.Nakamura wrote:

[#32857] Re: Windowでのデフォルトエンコーディング — "U.Nakamura" <usa@...> 2007/12/28

こんにちは、なかむら(う)です。

[#32852] Resolv::DNS#getaddresses doesn't return IPv6 address — "NARUSE, Yui" <naruse@...>

成瀬です。

17 messages 2007/12/28
[#32923] Re: Resolv::DNS#getaddresses doesn't return IPv6 address — Takahiro Kambe <taca@...> 2008/01/05

こんにちは。

[#32924] Re: Resolv::DNS#getaddresses doesn't return IPv6 address — "NARUSE, Yui" <naruse@...> 2008/01/05

成瀬です。

[#32925] Re: Resolv::DNS#getaddresses doesn't return IPv6 address — Takahiro Kambe <taca@...> 2008/01/05

In message <477EF0C9.4060103@airemix.com>

[#32929] Re: Resolv::DNS#getaddresses doesn't return IPv6 address — "NARUSE, Yui" <naruse@...> 2008/01/05

成瀬です

[ruby-dev:32705] Re: ISO-2022-JP output for transcode

From: "NARUSE, Yui" <naruse@...>
Date: 2007-12-24 08:39:39 UTC
List: ruby-dev #32705
成瀬です。

NARUSE, Yui wrote:
> transcode_data_japanese.c は自動生成な気がしたのと、transcode.c の構造体
> に依存しているので、下手に分けずにとりあえず transcode.c に入れておきま
> した。
> transcoding と transcoder を transcode_data.h いれればいいのですかね。作
> 業しつつとりあえずこの状態で投稿しておきます。

分けました。
これでよさそうでしたらコミットします。

-- 
NARUSE, Yui  <naruse@airemix.com>
DBDB A476 FDBD 9450 02CD 0EFC BCE3 C388 472E C1EA

Attachments (1)

iso_2022_jp_patch.r3.txt (13.5 KB, text/x-diff)
Index: transcode_data_japanese.c
===================================================================
--- transcode_data_japanese.c	(revision 14583)
+++ transcode_data_japanese.c	(working copy)
@@ -23618,4 +23618,209 @@
     to_EUC_JP_infos
 };
 
-/* Footprint (bytes): gross: 212680, saved: 50764, net: 161916 */
+#define ISO_2022_ENCODING(escseq, byte) ((escseq<<8)|byte)
+enum ISO_2022_ESCSEQ {
+    ISO_2022_CZD   = '!',
+    ISO_2022_C1D   = '"',
+    ISO_2022_GZD4  = '(',
+    ISO_2022_G1D4  = ')',
+    ISO_2022_G2D4  = '*',
+    ISO_2022_G3D4  = '+',
+    ISO_2022_G1D6  = '-',
+    ISO_2022_G2D6  = '.',
+    ISO_2022_G3D6  = '/',
+    ISO_2022_GZDM4 = ISO_2022_ENCODING('$','('),
+    ISO_2022_G1DM4 = ISO_2022_ENCODING('$',')'),
+    ISO_2022_G2DM4 = ISO_2022_ENCODING('$','*'),
+    ISO_2022_G3DM4 = ISO_2022_ENCODING('$','+'),
+    ISO_2022_G1DM6 = ISO_2022_ENCODING('$','-'),
+    ISO_2022_G2DM6 = ISO_2022_ENCODING('$','.'),
+    ISO_2022_G3DM6 = ISO_2022_ENCODING('$','/'),
+    ISO_2022_DOCS  = ISO_2022_ENCODING('%','I'),
+    ISO_2022_IRR   = '&'
+};
+
+
+#define ISO_2022_GZ_ASCII                       ISO_2022_ENCODING(ISO_2022_GZD4, 'B')
+#define ISO_2022_GZ_JIS_X_0201_Katakana         ISO_2022_ENCODING(ISO_2022_GZD4, 'I')
+#define ISO_2022_GZ_JIS_X_0201_Roman            ISO_2022_ENCODING(ISO_2022_GZD4, 'J')
+#define ISO_2022_GZ_JIS_C_6226_1978             ISO_2022_ENCODING(ISO_2022_GZDM4,'@')
+#define ISO_2022_GZ_JIS_X_0208_1983             ISO_2022_ENCODING(ISO_2022_GZDM4,'B')
+#define ISO_2022_GZ_JIS_X_0212_1990             ISO_2022_ENCODING(ISO_2022_GZDM4,'D')
+#define ISO_2022_GZ_JIS_X_0213_2000_1           ISO_2022_ENCODING(ISO_2022_GZDM4,'O')
+#define ISO_2022_GZ_JIS_X_0213_2000_2           ISO_2022_ENCODING(ISO_2022_GZDM4,'P')
+#define ISO_2022_GZ_JIS_X_0213_2004_1           ISO_2022_ENCODING(ISO_2022_GZDM4,'Q')
+
+static int
+get_iso_2022_mode(char **in_pos)
+{
+    int new_mode;
+    char *in_p = *in_pos;
+    switch (*in_p++)
+    {
+    case '(':
+	switch (*in_p++)
+	{
+	case 'B': case 'I': case 'J':
+	    new_mode = ISO_2022_ENCODING(ISO_2022_GZD4, *(in_p-1));
+	    break;
+	default:
+	    rb_raise(rb_eRuntimeError /*change exception*/, "this mode is not supported (ESC ( %c)", *(in_p-1));
+	    break;
+	}
+	break;
+    case '$':
+	switch (*in_p++)
+	{
+	case '@': case 'A': case 'B':
+	    new_mode = ISO_2022_ENCODING(ISO_2022_GZDM4, *(in_p-1));
+	    break;
+	case '(':
+	    switch (*in_p++)
+	    {
+	    case 'D': case 'O': case 'P': case 'Q':
+		new_mode = ISO_2022_ENCODING(ISO_2022_GZDM4, *(in_p-1));
+		break;
+	    default:
+		rb_raise(rb_eRuntimeError /*change exception*/, "this mode is not supported (ESC $ ( %c)", *(in_p-1));
+		break;
+	    }
+	    break;
+	default:
+	    rb_raise(rb_eRuntimeError /*change exception*/, "this mode is not supported (ESC $ %c)", *(in_p-1));
+	    break;
+	}
+	break;
+    default:
+	    rb_raise(rb_eRuntimeError /*change exception*/, "this mode is not supported (ESC %c)", *(in_p-1));
+	break;
+    }
+    *in_pos = in_p;
+    return new_mode;
+}
+
+void
+from_iso_2022_jp_transcoder_preprocessor(char **in_pos, char **out_pos,
+				     char *in_stop, char *out_stop,
+				     transcoder *my_transcoder,
+				     transcoding *my_transcoding)
+{
+    char *in_p = *in_pos, *out_p = *out_pos;
+    int cur_mode = ISO_2022_GZ_ASCII;
+    unsigned char c1;
+    char *out_s = out_stop - my_transcoder->max_output + 1;
+    while (in_p < in_stop) {
+	if (out_p >= out_s) {
+	    int len = (out_p - *out_pos);
+	    int new_len = (len + my_transcoder->max_output) * 2;
+	    *out_pos = (*my_transcoding->flush_func)(my_transcoding, len, new_len);
+	    out_p = *out_pos + len;
+	    out_s = *out_pos + new_len - my_transcoder->max_output;
+	}
+	c1 = *in_p++;
+	if (c1 == 0x1B) {
+	    cur_mode = get_iso_2022_mode(&in_p);
+	} else if (c1 == 0x1E || c1 == 0x1F) {
+	    /* SHIFT */
+	    rb_raise(rb_eRuntimeError /*change exception*/, "shift is not supported");
+	} else if (c1 >= 0x80) {
+	    rb_raise(rb_eRuntimeError /*change exception*/, "illegal byte sequence");
+	} else {
+	    switch (cur_mode) {
+	    case ISO_2022_GZ_ASCII:
+	    case ISO_2022_GZ_JIS_X_0201_Roman:
+		*out_p++ = c1;
+		break;
+	    case ISO_2022_GZ_JIS_X_0201_Katakana:
+		*out_p++ = 0x8E;
+		*out_p++ = c1 | 0x80;
+		break;
+	    case ISO_2022_GZ_JIS_X_0212_1990:
+		*out_p++ = 0x8F;
+	    case ISO_2022_GZ_JIS_C_6226_1978:
+	    case ISO_2022_GZ_JIS_X_0208_1983:
+		*out_p++ = c1 | 0x80;
+		*out_p++ = *in_p++ | 0x80;
+		break;
+	    }
+	}
+    }
+    /* cleanup */
+    *in_pos  = in_p;
+    *out_pos = out_p;
+}
+
+static int
+select_iso_2022_mode(char **out_pos, int new_mode)
+{
+    char *out_p = *out_pos;
+    *out_p++ = '\e';
+    switch (new_mode>>8)
+    {
+    case ISO_2022_GZD4:
+	*out_p++ = new_mode >> 8;
+	*out_p++ = new_mode & 0x7F;
+	break;
+    case ISO_2022_GZDM4:
+	*out_p++ = new_mode >> 16;
+	if ((new_mode & 0x7F) != '@' &&
+	    (new_mode & 0x7F) != 'A' &&
+	    (new_mode & 0x7F) != 'B')
+	{
+	    *out_p++ = (new_mode>>8) & 0x7F;
+	}
+	*out_p++ = new_mode & 0x7F;
+	break;
+    default:
+	rb_raise(rb_eRuntimeError /*change exception*/, "this mode is not supported.");
+	break;
+    }
+    *out_pos = out_p;
+    return new_mode;
+}
+
+void
+to_iso_2022_jp_transcoder_postprocessor(char **in_pos, char **out_pos,
+				     char *in_stop, char *out_stop,
+				     transcoder *my_transcoder,
+				     transcoding *my_transcoding)
+{
+    char *in_p = *in_pos, *out_p = *out_pos;
+    int cur_mode = ISO_2022_GZ_ASCII, new_mode = 0;
+    unsigned char next_byte;
+    char *out_s = out_stop - my_transcoder->max_output + 1;
+    while (in_p < in_stop) {
+	if (out_p >= out_s) {
+	    int len = (out_p - *out_pos);
+	    int new_len = (len + my_transcoder->max_output) * 2;
+	    *out_pos = (*my_transcoding->flush_func)(my_transcoding, len, new_len);
+	    out_p = *out_pos + len;
+	    out_s = *out_pos + new_len - my_transcoder->max_output;
+	}
+	next_byte = *in_p++;
+	if (next_byte < 0x80) {
+	    new_mode = ISO_2022_GZ_ASCII;
+	} else if (next_byte == 0x8E) {
+	    new_mode = ISO_2022_GZ_JIS_X_0201_Katakana;
+	    next_byte = *in_p++;
+	} else if (next_byte == 0x8F) {
+	    new_mode = ISO_2022_GZ_JIS_X_0212_1990;
+	    next_byte = *in_p++;
+	} else {
+	    new_mode = ISO_2022_GZ_JIS_X_0208_1983;
+	}
+	if (cur_mode != new_mode)
+	    cur_mode = select_iso_2022_mode(&out_p, new_mode);
+	if (cur_mode < 0xFFFF) {
+	    *out_p++ = next_byte & 0x7F;
+	} else {
+	    *out_p++ = next_byte & 0x7F;
+	    *out_p++ = *in_p++ & 0x7F;
+	}
+    }
+    if (cur_mode != ISO_2022_GZ_ASCII)
+	cur_mode = select_iso_2022_mode(&out_p, ISO_2022_GZ_ASCII);
+    /* cleanup */
+    *in_pos  = in_p;
+    *out_pos = out_p;
+}
Index: transcode_data.h
===================================================================
--- transcode_data.h	(revision 14583)
+++ transcode_data.h	(working copy)
@@ -1,3 +1,20 @@
+/**********************************************************************
+
+  transcode_data.h -
+
+  $Author$
+  $Date$
+  created at: Mon 10 Dec 2007 14:01:47 JST 2007
+
+  Copyright (C) 2007 Martin Duerst
+
+**********************************************************************/
+
+#include "ruby/ruby.h"
+
+#ifndef RUBY_TRANSCODE_DATA_H
+#define RUBY_TRANSCODE_DATA_H 1
+
 typedef unsigned char base_element;
 
 typedef struct byte_lookup {
@@ -37,3 +54,25 @@
 #define TWOTRAIL       /* legal but undefined if two more trailing UTF-8 */
 #define THREETRAIL     /* legal but undefined if three more trailing UTF-8 */
 
+/* dynamic structure, one per conversion (similar to iconv_t) */
+/* may carry conversion state (e.g. for iso-2022-jp) */
+typedef struct transcoding {
+    VALUE ruby_string_dest; /* the String used as the conversion destination,
+			       or NULL if something else is being converted */
+    char *(*flush_func)(struct transcoding*, int, int);
+} transcoding;
+
+/* static structure, one per supported encoding pair */
+typedef struct transcoder_st{
+    const char *from_encoding;
+    const char *to_encoding;
+    const BYTE_LOOKUP *conv_tree_start;
+    int max_output;
+    int from_utf8;
+    void (*preprocessor)(char**, char**, char*, char*,
+				struct transcoder_st *transcoder, struct transcoding*);
+    void (*postprocessor)(char**, char**, char*, char*,
+				 struct transcoder_st *transcoder, struct transcoding*);
+} transcoder;
+
+#endif /* RUBY_TRANSCODE_DATA_H */
Index: transcode.c
===================================================================
--- transcode.c	(revision 14583)
+++ transcode.c	(working copy)
@@ -60,22 +60,17 @@
 extern const BYTE_LOOKUP rb_to_SHIFT_JIS;
 extern const BYTE_LOOKUP rb_to_EUC_JP;
 
+extern void from_iso_2022_jp_transcoder_preprocessor(char**, char**, char*, char*,
+				struct transcoder_st *transcoder, struct transcoding*);
+extern void to_iso_2022_jp_transcoder_postprocessor(char**, char**, char*, char*,
+				struct transcoder_st *transcoder, struct transcoding*);
 
 /* declarations probably need to go into separate header file, e.g. transcode.h */
 
-/* static structure, one per supported encoding pair */
-typedef struct {
-    const char *from_encoding;
-    const char *to_encoding;
-    const BYTE_LOOKUP *conv_tree_start;
-    int max_output;
-    int from_utf8;
-} transcoder;
-
 /* todo: dynamic structure, one per conversion (stream) */
 
 /* in the future, add some mechanism for dynamically adding stuff here */
-#define MAX_TRANSCODERS 33  /* todo: fix: this number has to be adjusted by hand */
+#define MAX_TRANSCODERS 35  /* todo: fix: this number has to be adjusted by hand */
 static transcoder transcoder_table[MAX_TRANSCODERS];
 
 /* not sure why it's not possible to do relocatable initializations */
@@ -100,6 +95,29 @@
 }
 
 static void
+register_functional_transcoder(const char *from_e, const char *to_e,
+    const BYTE_LOOKUP *tree_start, int max_output, int from_utf8,
+    void (*preprocessor)(char**, char**, char*, char*, transcoder*, transcoding*),
+    void (*postprocessor)(char**, char**, char*, char*, transcoder*, transcoding*))
+{
+    static int n = 0;
+    if (n >= MAX_TRANSCODERS) {
+	/* we are initializing, is it okay to use rb_raise here? */
+	rb_raise(rb_eRuntimeError /*change exception*/, "not enough transcoder slots");
+    }
+    transcoder_table[n].from_encoding = from_e;
+    transcoder_table[n].to_encoding = to_e;
+    transcoder_table[n].conv_tree_start = tree_start;
+    transcoder_table[n].max_output = max_output;
+    transcoder_table[n].from_utf8 = from_utf8;
+    transcoder_table[n].conv_tree_start = tree_start;
+    transcoder_table[n].preprocessor = preprocessor;
+    transcoder_table[n].postprocessor = postprocessor;
+
+    n++;
+}
+
+static void
 init_transcoder_table(void)
 {
     register_transcoder("ISO-8859-1",  "UTF-8", &rb_from_ISO_8859_1, 2, 0);
@@ -135,6 +153,10 @@
     register_transcoder("EUC-JP", "UTF-8",      &rb_from_EUC_JP, 3, 0);
     register_transcoder("UTF-8", "SHIFT_JIS",   &rb_to_SHIFT_JIS, 2, 1);
     register_transcoder("UTF-8", "EUC-JP",      &rb_to_EUC_JP, 2, 1);
+    register_functional_transcoder("ISO-2022-JP", "UTF-8", &rb_from_EUC_JP,
+				   8, 0, &from_iso_2022_jp_transcoder_preprocessor, NULL);
+    register_functional_transcoder("UTF-8", "ISO-2022-JP", &rb_to_EUC_JP,
+				   8, 1, NULL, &to_iso_2022_jp_transcoder_postprocessor);
 
     register_transcoder(NULL, NULL, NULL, 0, 0);
 }
@@ -165,15 +187,7 @@
     return NULL;
 }
 
-/* dynamic structure, one per conversion (similar to iconv_t) */
-/* may carry conversion state (e.g. for iso-2022-jp) */
-typedef struct transcoding {
-    VALUE ruby_string_dest; /* the String used as the conversion destination,
-			       or NULL if something else is being converted */
-    char *(*flush_func)(struct transcoding*, int, int);
-} transcoding;
 
-
 /*
  *  Transcoding engine logic
  */
@@ -331,6 +345,23 @@
 	    rb_raise(rb_eArgError, "transcoding not supported (from %s to %s)", from_e, to_e);
 	}
 
+	if (my_transcoder->preprocessor)
+	{
+	    fromp = sp = RSTRING_PTR(str);
+	    slen = RSTRING_LEN(str);
+	    blen = slen + 30; /* len + margin */
+	    dest = rb_str_tmp_new(blen);
+	    bp = RSTRING_PTR(dest);
+	    my_transcoding.ruby_string_dest = dest;
+	    (*my_transcoder->preprocessor)(&fromp, &bp, (sp+slen), (bp+blen), my_transcoder, &my_transcoding);
+	    if (fromp != sp+slen) {
+		rb_raise(rb_eArgError, "not fully converted, %d bytes left", sp+slen-fromp);
+	    }
+	    buf = RSTRING_PTR(dest);
+	    *bp = '\0';
+	    rb_str_set_len(dest, bp - buf);
+	    str = dest;
+	}
 	fromp = sp = RSTRING_PTR(str);
 	slen = RSTRING_LEN(str);
 	blen = slen + 30; /* len + margin */
@@ -346,6 +377,23 @@
 	buf = RSTRING_PTR(dest);
 	*bp = '\0';
 	rb_str_set_len(dest, bp - buf);
+	if (my_transcoder->postprocessor)
+	{
+	    str = dest;
+	    fromp = sp = RSTRING_PTR(str);
+	    slen = RSTRING_LEN(str);
+	    blen = slen + 30; /* len + margin */
+	    dest = rb_str_tmp_new(blen);
+	    bp = RSTRING_PTR(dest);
+	    my_transcoding.ruby_string_dest = dest;
+	    (*my_transcoder->postprocessor)(&fromp, &bp, (sp+slen), (bp+blen), my_transcoder, &my_transcoding);
+	    if (fromp != sp+slen) {
+		rb_raise(rb_eArgError, "not fully converted, %d bytes left", sp+slen-fromp);
+	    }
+	    buf = RSTRING_PTR(dest);
+	    *bp = '\0';
+	    rb_str_set_len(dest, bp - buf);
+	}
 
 	if (encoding_equal(my_transcoder->to_encoding, to_e)) {
 	    final_encoding = 1;

In This Thread