From: Yukihiro Matsumoto <matz@...>
Date: 2006-06-26T15:08:02+09:00
Subject: Re: Unicode roadmap?

Hi,

In message "Re: Unicode roadmap?"
    on Mon, 26 Jun 2006 13:51:33 +0900, Izidor Jerebic <ij.rubylist@gmail.com> writes:

|>
|> I am not sure what you expect about separation, but I doubt separation
|> would make above code to "be more logical and break far less".
|
|Above code assumes all file operations return byte arrays. What is  
|the code when we want to obtain String of characters?

  line = File.open(filename, "r", "utf8") {|f| f.gets } 

|What if there is some $KCODE (or equivalent) setting somewhere in the  
|program before these lines? What would be the effect of that?

I think IO#read shall always return "binary" string, since its
specified length should always be in bytes.  Anyway, when in doubt,
you can explicitly specify "binary" encoding,

|The problem is the auto-magic encoding handling which is required to  
|have text processing be as simple as it is now. You can have either  
|text processing (which adds encoding handling for us, combines bytes  
|in characters etc.) or byte processing (which does not). How do we  
|distinguish between the two modes of operation?

By explicitly setting their encoding to "binary", e.g.

  text = obtain_string_data()
  text.encoding = "binary"
  ...

|The obvious way is by adding a ByteArray. But maybe there is better  
|way...

Show me the pseudo code using ByteArray, I will show you its
counterpart using String with encoding tag.

							matz.