From: Bil Kleb <Bil.Kleb@...>
Date: 2007-05-03T22:50:06+09:00
Subject: Beyond YAML? (scaling)

Hi,

I've been using YAML files to store hashes of numbers, e.g.,

  { ["O_Kc_01"] => [ 0.01232, 0.01212, 0.03222, ... ], ... }

This has worked wonderfully for portability and visibility
into the system as I've been creating it.

Recently, however, I've increased my problem size by orders
of magnitude in both the number of variables and the number
of associated values. The resulting YAML files are prohibitive:
10s of MBs big and requiring 10s of minutes to dump/load.

Where should I go from here?

Thanks,
--
Bil Kleb
http://fun3d.larc.nasa.gov