From: Sung Pae <sungpae@...>
Date: 2012-08-01T17:04:25+09:00
Subject: Re: working with large binary strings?

bob hope <lists@ruby-forum.com> writes:

> The goal is, with 4 random strings, to make a 4th random string that
> xors to 0 for each row with the other strings.

I have no idea what you're trying to accomplish, but here are some tips:

> I don't know how to concatenate with a number so that is why I make it
> a string here.

Strings are the proper format for transferring and storing binary data.
You just have to work on them in chunks.

> require "securerandom"
> require "openssl"
>
> a = SecureRandom.random_bytes(1_000_000).unpack("B*")[0]
> b = SecureRandom.random_bytes(1_000_000).unpack("B*")[0]
> c = SecureRandom.random_bytes(1_000_000).unpack("B*")[0]
> d = SecureRandom.random_bytes(1_000_000).unpack("B*")[0]

SecureRandom.random_bytes returns a string with each byte containing
a random value from 0x00 to 0xff. You are converting each byte into
an 8-byte string representation of this number, which is incredibly
wasteful.

    SecureRandom.random_bytes(len).unpack('C*')

will return the string as an array of `len` unsigned integers, which you
can then XOR without any more string conversions.

>  vsb = ""
>  100_000.times do |x|

Why 100,000 when you have created strings of 8,000,000 characters in
length?

>  puts x
>  column = a[x].to_i ^ b[x].to_i ^ c[x].to_i ^ d[x].to_i ^ 0

This code actually XORs a single _bit_ at a time.

If you have arrays of integers, you can XOR byte(s) at a time without
string conversions.

Also, n XOR 0 always returns n, so that part does nothing. If this is an
important part of your algorithm, I think you need to think this through
a bit longer.

>  case column
>  when 0
>    vsb << 0.to_s
>    else
>    vsb << 1.to_s
>  end

This is very circuitous. You already have the value 0 or 1, so just push
it onto `vsb` directly!

> end
>
> puts vsb.to_i

Why does vsb need to be a number? Numbers larger than your CPU's native
bit size are very inefficient to work with. Binary data should be passed
around as a string.

If I am understanding you, this is what you want: [1]

    len = 1_000_000
    a, b, c, d = 4.times.map { SecureRandom.random_bytes(len).unpack 'L*' }
    (len/4).times.map { |i| a[i] ^ b[i] ^ c[i] ^ d[i] }.pack 'L*'

However, this is totally useless, since the input is random, and 4
random inputs XORed together are equivalent to a single random input.

If you're planning on supplying your own data, this smells an awful lot
like home-rolled encryption, which is either admirable or horrifying
depending on your goal.

HTH
guns

[1]: Note that I am chunking the string into 32-bit unsigned longs for
     performance