From: Jon Leighton <j@...>
Date: 2011-11-07T19:29:21+09:00
Subject: [ruby-core:40811] [Backport93 - Backport #5564] 1.9.3 regression with encoding conversion


Issue #5564 has been updated by Jon Leighton.


My installation is Fedora 15. We have also reproduced on the Rails CI server (Ubuntu Lucid), and I just asked some people to test on a Mac, and they also can reproduce. So I do not think it's platform specific.

You should be able to reproduce on your own computer by setting up the Rails test suite. (This is a bit of a pain I know, I'm sorry I haven't found a simpler test case.)

1. Clone git://github.com/rails/rails.git
2. bundle install
3. cd actionpack/
4. Open lib/action_view/template.rb and comment the conditional block starting with if defined?(RUBY_ENGINE) (this disables the workaround for the bug) [only necessary if you are running exactly 1.9.3p0, but this bug can be repro'd on trunk also]
5. ruby -Itest test/template/template_test.rb -n test_default_external_works
----------------------------------------
Backport #5564: 1.9.3 regression with encoding conversion
http://redmine.ruby-lang.org/issues/5564

Author: Jon Leighton
Status: Assigned
Priority: Normal
Assignee: Yuki Sonoda
Category: 
Target version: 


Hello,

We are experiencing a bug in the Rails test suite against ruby 1.9.3.

The bug looks like this:

  1) Error:
test_default_external_works(TestERBTemplate):
ActionView::Template::Error: code converter not found (ISO-8859-1 to UTF-8)
    /home/turnip/Code/rails/actionpack/lib/action_view/template/handlers/erb.rb:83:in `encode!'
    /home/turnip/Code/rails/actionpack/lib/action_view/template/handlers/erb.rb:83:in `call'
    /home/turnip/Code/rails/actionpack/lib/action_view/template.rb:252:in `compile'
    /home/turnip/Code/rails/actionpack/lib/action_view/template.rb:189:in `compile!'
    /home/turnip/Code/rails/actionpack/lib/action_view/template.rb:142:in `block in render'
    /home/turnip/Code/rails/activesupport/lib/active_support/notifications.rb:55:in `instrument'
    /home/turnip/Code/rails/actionpack/lib/action_view/template.rb:141:in `render'
    test/template/template_test.rb:53:in `render'
    test/template/template_test.rb:137:in `block in test_default_external_works'
    test/template/template_test.rb:176:in `with_external_encoding'
    test/template/template_test.rb:135:in `test_default_external_works'
    /home/turnip/.rvm/gems/ruby-1.9.3-p0/gems/mocha-0.10.0/lib/mocha/integration/mini_test/version_230_to_251.rb:28:in `run'

A simplified view of what that code is doing is the following:

 s = "hello \u{fc}mlat"
 s.force_encoding "ISO-8859-1"
 s.encode! 'UTF-8'

However, I have been unable to create any simplified test case for this bug.

What I have been able to find out is the following:

1. Via git-bisect, I tracked down the changed which introduced the bug (it was not present in 1.9.3-preview1). That change was https://github.com/ruby/ruby/commit/05c2cc90895750255f64254f2491ff0ad77cc13a (r32791)
2. If I do some sort of conversion at *load time* in the file action_view/template/handlers/erb.rb, the bug does not occur. For example, if I insert "Encoding::Converter.new('ISO-8859-1', 'UTF-8')" (or the 3 lines above) at the top of the file, I do not experience the bug.

Let me know if I can provide any more information. Also any information about what exactly is triggering this would be very helpful so that we can put a workaround in Rails (if possible).

Thanks


-- 
http://redmine.ruby-lang.org