From: Robert Klemme <shortcutter@...>
Date: 2011-02-17T19:01:27+09:00
Subject: Re: Marshal data too short

On Wed, Feb 16, 2011 at 6:06 PM, Mike Flint <mfcoder-ap@yahoo.co.uk> wrote:
> Hello,
>
> On Ruby 1.8.7.
>
> Having an odd occasional occurrence of the above error - only some times
> so I feel it's down to timing caused by file buffer issues (maybe!)
>
> I have a Web Server running under Sinatra/Rack. Part of the process
> involves a Cache class which is a singleton. This class checks for the
> existence of an index file. If the file does exist, then its contents
> are loaded, and then Marshal loaded as a hash. If the file does not
> exist, then a Hash is created, and Marshal dumped (ro a string), then
> written to the index file.

Are you using binary IO?  I.e.

cache = File.open(cache_file, "rb") {|io| Marshal.load(io)}

File.open(cache_file, "wb") {|io| Marshal.dump(cache, io)}

> About 1 in 20 times the Marshal.load gives the 'too short' message.
>
> Once I get this message it remains. That is, an hour later, long after
> the index file has been created the error still happens. I can hexdump
> the file but the Web Server still gets the error. Looking at the index
> file which is 'broken' it seems no different to a good one. They are
> both 82 bytes long.

You should probably also remove the file if it cannot be loaded.  This
will give you a more robust code base.

> There is only a single user accessing the system when the error occurs.
> The website is not busy.

Well, since the object is a singleton and most web pages consist of
multiple HTTP requests it may not matter how many different users
actually access it.  You get concurrency anyway.  For that you may use
file system level locking to avoid writing the file concurrently.

> I am using blocks on the read/write of the file so my file IO channel is
> being closed correctly between the writing of it, and subsequent reads.
> And even if it's not closed correctly, I figure that an hour and several
> messages later it will have been closed.
>
> My gut says it's a buffer issue, or a problem with how I'm writing the
> dump to the file ... using puts in a subsequent write of the string
> representation of the marshalled object rather than directly on the
> .dump ... but why does it work most of the time?

Chance.  We do not know what you put in the Hash so...  But using puts
to write marshaled content is plain wrong.  You should be marshalling
directly from and to the IO instance (see above).

> I've added some debugging which should shed light, but anyone have
> thoughts on the matter?

See above.  If it does not work, show the code.

Cheers

robert

-- 
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/