From: Eric Wong <normalperson@...>
Date: 2010-07-22T09:01:04+09:00
Subject: Re: Writing to file delay

Matt Triviski <matt.triviski@gmail.com> wrote:
> Hello All,
> 
> I am having a very annoying issue.  This is the situation:
> 
> I am reading a file on one machine that is continuosly being written to
> and writing out to a socket.  On another machine I am reading from the
> socket and writing it to a file.  The problem is that when I tail the
> original file and the file I am creating, the new file has a delay of
> around 1 minute until I see the new data.  It seems that the OS (Linux
> Redhat 5) is waiting until it gets a block of data before it actually
> writes to the file.  I want it to write immediately.
> 
> Here is a code snippet of how I am reading from the socket and writing
> to the file.
> 
> file = File.new(fname, "w")
> file.sync = true

Hi Matt,

"file.sync = true" looks good already.

Any chance you're writing to a file on NFS or some other remote
filesystem?

Since you're on Linux, try stracing both the Ruby process and your tail
process to see what it's doing.

> buff = ""
> 
> loop{
>   begin
>    ios = IO.select([server.socket], [file],[server.socket, file],
> 0.0001)

There's no point in ever calling IO.select on a regular file,
they're always ready for reading/writing.

Since you're down to one socket, I wouldn't bother with select at all:

     file.write(server.socket.readpartial(8192))
> 
>   rescue  Errno::ECONNRESET, SocketError
>    puts "ERROR:  Lost connection to server"
>    exit 1
>   rescue EOFError
>   end
> 
>   sleep 0.01
>   }
> }
> 
> I first used just a regular readline from the socket and then did a
> file.puts line, but I was having the same issue sot that is why I tried
> to go to select.  I am wondering if this is a Linux issue, since I
> turned the file sync to true.  However, I am not sure, so I am hoping
> that other code heads out there could help me out.

Whatever it is, there's a high likelyhood that strace-ing one of the
processes involved will tell you what the problem is :>

-- 
Eric Wong