From: Austin Ziegler <halostatue@...>
Date: 2004-08-25T13:29:57+09:00
Subject: Paging _why, YAML problems

_why:

I am having a problem with Ruby 1.8.2 preview2; the problem seems to
be YAML with a large enough file, but the file with which I've noticed
this I can't share, and I am still investigating to ensure that it
isn't in my own code, but I'm 99% sure at this point.

Basically, Ruwiki supports a YAML backend that simply uses YAML.dump
and YAML.load for a nested hash:

  { 'page' => { 'content' => actual-content-data } }

I wrote a custom backend for Ruwiki because of problems with Ruby
1.8.1. The default pages are in this format and as I was going through
and updating some code to reflect necessary changes to support a new
deployment mechanism (for RubyGems and RPA, as well as central server
installations), I tested the conversion routines to be able to switch
between the Flatfiles, Marshal, and Yaml backends. The roundtrip
Flatfiles => Marshal => Flatfiles appears to work regardless of the
filesize. The Flatfiles => Yaml => Flatfiles works fine for small
files (~2k or so), but fails on my ~14k file. The majority of these
files is the content (13,636 bytes for the large file) is the content
saved with \n (not \r\n, explicitly so).

The problem is that the file should be:

   section!item:<Tab>content...\n
   <Tab>content...\n
   <Tab>content...\n
   section!item:<Tab>content...\n

With this test program:

  p1    = Ruwiki::Backend::Flatfiles.load(data) # 1
  yaml  = Ruwiki::Backend::Yaml.dump(p1) # 2
  p2    = Ruwiki::Backend::Yaml.load(yaml) # 3
  ff    = Ruwiki::Backend::Flatfiles.dump(p2) # 4

In step 3 (which is simply "::YAML.load(buffer)" with a rescue to
catch ArgumentError and raise it as a custom error), with a large
enough file, backslashes are doubled (e.g., \\WikiWord becomes
\\\\WikiWord and \n becomes \\n), which means that the result of #4
will be completely wrong.

If you contact me offlist, I can give you the files that exhibit this
behaviour. I understand you introduced some changes in 1.8.2 post
preview2, but as of the moment, that's all I have access to on
Windows.

-austin
-- 
Austin Ziegler * halostatue@gmail.com
               * Alternate: austin@halostatue.ca