From: Austin Ziegler <halostatue@...>
Date: 2004-11-26T01:22:01+09:00
Subject: Re: YAML Problems Clearinghouse (Re: Marshal vs. YAML...)

On Thu, 25 Nov 2004 16:36:30 +0900, why the lucky stiff
<ruby-talk@whytheluckystiff.net> wrote:
> Austin Ziegler wrote:
>> Ruwiki has specific needs that may or may not be present in a
>> generic application that needs something externally editable. In
>> particular, Ruwiki files can be large because of the content --
>> which is a large \n-delimited string. For a Ruwiki file at work
>> and for the Ruwiki::WikiMarkup pages, syck failed (as in
>> coredump) in Ruby 1.8.1 on Windows on writing strings longer than
>> about 7k and reading strings longer than about 4k. I don't think
>> it was a Windows-only problem. In Ruby 1.8.2p2, syck was confused
>> by \n and sometimes escaped them when it shouldn't, resulting in
>> unusable code.
> I'm sure syck can handle large quantities of text, as well as
> continuous strings, since the (Poignant) Guide and my blog have
> both been using syck since February of this year. But I definitely
> don't use the emitter (the outputting component of syck) as much
> as your wiki must.

Well, I know that with Ruby 1.8.1 I couldn't read a 15k YAML file
without coredumping. When I split the 12k content section into
multiple parts with an array, I found that on Windows the coredump
happened with a big block of text between 7k and 8k; on Linux it was
closer to 11k. These tests were several months ago, and they went
away with Ruby 1.8.2 preview 2. Unfortunately, there was the \n
problem you fixed after preview 2, and I haven't had time to rerun
my tests with preview 3: this will be run in the next day or two as
I prepare to release Ruwiki 0.9.0 (I still have to verify the YAML
and Marshal storage formats since I support them even though I use
and prefer the Ruwiki::Exportable format).

> I feel that syck's parser is quite strong, but the emitter is very
> poor in comparison. Would you offer up your time in helping me
> test and kill further bugs in the emitter? (This is an open
> invitation to any of you out there who have an interest in
> improving YAML support.)

I can definitely give you some test data and a program (it's part of
Ruwiki, actually) to test it so you can look at the results, but I
don't have a lot of time to commit to testing at this point -- as
soon as I release Ruwiki 0.9.0, I have to start work on both Ruwiki
0.10.0 and Diff::LCS 1.2.0.

>> I prefer YAML to XML for most editable configuration files, but
>> YAML has its own issues.
> You've mentioned coredumps on outputting YAML and oddities with
> line endings. Did I miss anything else? Could you outline exactly
> what other issues you've encountered?
> 
> And do they revolve centrally around the Syck implementation or
> are they related to the nature of YAML and its specification?

XML is overly verbose and YAML is problematic because of space/tab
issues. I like YAML because it's cleaner than XML, but I don't like
the indentation model.

-austin
-- 
Austin Ziegler * halostatue@gmail.com
               * Alternate: austin@halostatue.ca