[ruby-core:40267] Re: Counseling

On Oct 21, 2011, at 6:30 PM, Bill Kelly wrote:

> Gon=E7alo Silva wrote:
>>=20
>> While I believe that this is a necessary evil for the real encoding
>> support that ruby currently provides, it would be great to have a
>> compile time flag to revert back to the old behavior (or something
>> similar). I don't think this is implemented, but I could be wrong.
>=20
> It appears that defining NO_LOCALE_CHARMAP when building ruby, may
> have this effect:
>=20
> $ git branch
> * ruby_1_9_3
>=20
> $ cflags=3D'-DNO_LOCALE_CHARMAP' ./configure --prefix=3D/opt/ruby193 =
--program-suffix=3D19 --enable-shared --disable-install-doc
> $ make
> $ sudo make install
>=20
> $ /opt/ruby193/bin/ruby19 -v -e 'puts "".encoding'
> ruby 1.9.3dev (2011-10-11 revision 33457) [x86_64-linux]
> ASCII-8BIT
>=20
> $ /opt/ruby193/bin/ruby19 -e 'puts `ls`.encoding'
> ASCII-8BIT
>=20
>=20
> Hope this helps,

Thanks Bill.  I will try that.  That looks very promising.  Just as good =
of an alternative would be to change my default to UTF-8 instead of =
US-ASCII.

To the others of interest: I am running a Mac Lion (10.7) (development) =
as well as AIX 5.3 (production).  Rails 2.3.x.  Ruby 1.9.2 p290.  I've =
had encoding problems running Mac 10.6, Rails 2.3.5 up, and Ruby 1.9.1 =
and up.  Essentially, when I switched to Ruby 1.9, I started having =
encoding problems.  I'm using PostgreSQL as my database backend with =
"pg" as the Ruby interface to ActiveRecord.  All my gems are fairly up =
to date.  (Oh... I also have ActiveLdap for interfacing with an LDAP =
server too.)

The Rails web site is used world wide and a lot of my encoding problems =
come with users in Korea but I can hit problems on my own Mac with my =
own requests.  I try (very hard) to keep all i/o as UTF-8.

My first attempt to solve this was to put a UTF-8 coding into all my =
ruby files.  This appeared to help but upon reflection, I don't think it =
really did.  Adding the -KU in scripts like thin's startup script helps =
more.  In fact, I think it solves 99.99% of my problems.  But when I =
update thin (for example) I forget to add the -KU to the script and hit =
errors until I add the -KU back.

One recent saga involved memcache-client (which I've mentioned).  =
memcache-client tries to concatenate a command, key, and Marshall'ed =
data.  If the Marshalled data really is ASCII-8BIT, then the =
concatenation dies.  I cobbled around that by forcing the marshalled =
data back to UTF-8 but... I wonder if that is really the right solution. =
 Someone mentioned Dalli.  I plan to move to that as soon as possible. =
(this weekend I hope)

Rails tends to have erb files and often the response built up so far is =
not compatible with a new piece about to be added and there is no easy =
way to fix this that I've found yet.  Today's encoding problem was =
because the -KU had been left out of one of my startup files.  I need to =
add a step in my update process to put the -KU flags in.  Someone =
mentioned that Bundler has a way to do that.

I guess, deep down, my belief is that all the strings really really =
really are UTF-8 but they are being marked otherwise for some reason (as =
in the case of Marshalled data).

I hope that helps others understand my woes a bit better.

Thanks for your help,
pedz