From: "Alexey I. Froloff" <sir_raorn@...>
Date: 2007-09-12T03:05:55+09:00
Subject: Strange ripper bug


--DIOMP1UsTsWJauNi
Content-Type: multipart/mixed; boundary="LpQ9ahxlCli8rRTG"
Content-Disposition: inline


--LpQ9ahxlCli8rRTG
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Sometimes, ripper can't parse valid code (trunk from yesterday).
Examples attached taken from rubygems/format.rb (version 0.9.0).

$ ./sexp format.rb format1.rb format2.rb format3.rb
# format.rb
nil

Ripper.sexp returns nil.

# format1.rb
[:program,
 [[:iter_block,
   [:call,
    [:var_ref, [:@ident, "pkg", [1, 0]]],
    :".",
    [:@ident, "each", [1, 4]]],
   [:do_block, [], [:@kw, "end", [4, 0]]]]]]

Changed "," to "=3D>" in hash.  Block contents is missing.

# format2.rb
[:program,
 [[:binary,
   [:call,
    [:var_ref, [:@ident, "format", [2, 2]]],
    :".",
    [:@ident, "file_entries", [2, 9]]],
   :<<,
   [:array,
    [[:hash,
      [:assoclist_from_args,
       [[[:string_literal,
          [:string_content, [:@tstring_content, "size", [2, 28]]]],
         [:call,
          [:var_ref, [:@ident, "entry", [2, 37]]],
          :".",
          [:@ident, "size", [2, 43]]]],
        [[:string_literal,
          [:string_content, [:@tstring_content, "mode", [2, 50]]]],
         [:call,
          [:var_ref, [:@ident, "entry", [2, 59]]],
          :".",
          [:@ident, "mode", [2, 65]]]],
        [[:string_literal,
          [:string_content, [:@tstring_content, "path", [3, 7]]]],
         [:call,
          [:var_ref, [:@ident, "entry", [3, 16]]],
          :".",
          [:@ident, "full_name", [3, 22]]]]]]],
     [:call,
      [:var_ref, [:@ident, "entry", [3, 34]]],
      :".",
      [:@ident, "read", [3, 40]]]]]]]]

Leaving only block body.  This one looks good.

# format3.rb
[:program,
 [:call,
  [:var_ref, [:@ident, "entry", [3, 14]]],
  :".",
  [:@ident, "full_name", [3, 20]]]]

Block body from first example, reverted "," -> "=3D>" changes.

--=20
Regards,
Sir Raorn.

--LpQ9ahxlCli8rRTG
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename=sexp

#!/usr/bin/ruby -w

require 'pp'
require 'ripper'
require 'ripper/core.rb'
require 'ripper/sexp.rb'

ARGV.each do |f|
  puts "# #{f}"
  pp Ripper.sexp(File.new(f,"r").read, f)
end

# vim: set ft=ruby ts=2 sts=2 sw=2 et:

--LpQ9ahxlCli8rRTG
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="format.rb"

pkg.each do |entry|
  format.file_entries << [{"size", entry.size, "mode", entry.mode,
      "path", entry.full_name}, entry.read]
end

--LpQ9ahxlCli8rRTG
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="format1.rb"

pkg.each do |entry|
  format.file_entries << [{"size" => entry.size, "mode" => entry.mode,
      "path" => entry.full_name}, entry.read]
end

--LpQ9ahxlCli8rRTG
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="format2.rb"

#pkg.each do |entry|
  format.file_entries << [{"size" => entry.size, "mode" => entry.mode,
      "path" => entry.full_name}, entry.read]
#end

--LpQ9ahxlCli8rRTG
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="format3.rb"

#pkg.each do |entry|
  format.file_entries << [{"size", entry.size, "mode", entry.mode,
      "path", entry.full_name}, entry.read]
#end

--LpQ9ahxlCli8rRTG--

--DIOMP1UsTsWJauNi
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)

iD8DBQFG5tj7VqT7+fkT8woRAk8KAKCSJpcD6zWmhIfZVRh+ZkzOXRW4UACfSK2M
K46wv/1EHeaFfc7kceD1XrA=
=UvSV
-----END PGP SIGNATURE-----

--DIOMP1UsTsWJauNi--