From: Michael Selig Date: 2008-10-28T07:20:38+09:00 Subject: [ruby-core:19562] Re: String literal encoding (Was: Default source encoding (Was: [Bug #680] csv.rb: CSV.parse is toolate when encoding is mismatch)) On Mon, 27 Oct 2008 23:07:16 +1100, Nobuyoshi Nakada wrote: > Michael Selig wrote in [ruby-core:19540]: >> But my original point is that 1.8 scripts exist which contain multibyte >> characters (eg UTF-8) which work fine under 1.8 without-K, but will fail >> under 1.9 unless a magic comment or -K is provided. > > It just seemed working by chance. Actually I think that any ruby program which uses ASCII everywhere except in string literals will work without -K in 1.8. I think that this might be a reasonably common scenario, at least in countries which use UTF-8 or an ISO 8859 variant (outside the ASCII domain). These scripts will then fail in 1.9 without magic comment or -K. >> > But it's very ambiguous and dangerous to imply encodings. We >> > can't trust locale for this purpose, at least. >> >> It's a trade-off between that and backward compatibility. I think the >> "danger" is not high and it gives backward compatibility, so my vote >> would >> be to use it. > > And it will suddenly crash or behave weirdly by moving other > locales. Indeed - that's the danger. But I think the cost of losing backward compatibility is worse. How bad is this "danger" in reality? Simple scripts are often written just to be used on the same computer or LAN. There are probably lots of these, and there is almost certainly no danger there. I think that when people send out a script to other countries it would be perfectly reasonable to say that it must be run with -KU (or whatever) under 1.9 if their locale is not UTF-8 (or whatever). If they don't, then there is a high chance it will fail with an obvious error (eg: illegal multibyte char) - not a big deal to diagnose. There is a very small chance that it may run and behave weirdly. But then I would think that under 1.8 similar weird behaviour would have happened anyhow. > Anyway, I think I understand the needs to specify source > encoding without magic comments. Is the option for that > purpose an acceptable solution? In my opinion it is an acceptable workaround to add -K to RUBYOPT environment variable, but it is far from ideal. My preference is still to default to the locale encoding like -e does. Another alternative may be to default to ASCII-8BIT (which would be even more 1.8 compatible :) ), but I haven't thought through if there are any other repercussions to this. Losing backward compatibility for "simple" ruby scripts is a major negative IMHO. Martin wrote "It may be somewhat suboptimal in the transition phase, but ...." If I may say, I think that you guys might be underestimating the grief caused when people upgrade to 1.9 or 2.0 and find that even simple scripts no longer work. Cheers Mike