[ruby-core:44852] Re: [ruby-trunk - Feature #6361] Bitwise string operations

On Thursday, May 3, 2012 at 9:16 AM, "Martin J. D端rst" wrote:
> On 2012/04/30 1:50, Joshua Ballanco wrote:
>  
> > I know it seems like this class is just wrapping String and always defaulting to byte-wise operations, but it's more fundamental than that. Because there is no encoding on the bytes, there will never be an encoding error when working with them. This could be extremely useful for applications that combine bytes from multiple sources (e.g. Socket data + a file on disk + immediate strings in code) that could potentially have different encodings. By operating on bytes, you can defer the encoding checks until later, if at all.
>  
> I'm not saying I'm totally against this, but "extremely useful" could  
> also mean "too useful". There are clearly cases where one needs to put  
> things together at the byte level. But there are also quite some cases  
> that seem to "just work" when using byte-wise operations, at least as  
> long as nothing else but US-ASCII gets used. Things then blow up  
> terribly once some other characters get into the mix.
>  
>  


So, as an addendum to the spec, what about adding a flag when doing a string conversion:

    d.string_with_encoding('UTF-8', reject_if_invalid: true)

So that we could ensure that the return value is always either nil or a string with valid encoding.
  
> Actually, the binary/ASCII-8bit encoding is very close to a Blob. It was
> mostly Akira Tanaka who didn't want to distinguish between "true" binary  
> and ASCII-8bit, because that would have made the use of regular  
> expressions with binary impossible or convoluted.
>  
>  


My problem with String and ASCII-8BIT/BINARY encoding currently is that you *can't* just set a string's encoding to binary and forget about encodings. You will still run into issues working with binary data using Ruby 1.9 strings. I demonstrated the issue here: http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/40269 (where I, consequently, also made a plea for a Data/Blob type).
  
> Despite the title of this issue, I didn't see any *bit*wise operations
> (e.g. bitwise and/or/xor/not) proposed. Were you just taking them for  
> granted? What about adding these to String, maybe limiting them to  
> binary/ASCII-8bit?
>  
>  


I was taking bit-wise operations for granted. Ideally, a Data/Blob type would just represent N groupings of 8 1s and/or 0s, with byte-wise access and bit-wise manipulation. i.e. Less structured than an Array, less restrictive than a String. Just data.