From: Markus Schirp <mbj@...>
Date: 2011-05-13T00:51:05+09:00
Subject: Re: Optimize write of large file

IMHO the primary speed bottleneck is the disk drive itself and the "possible"
File-System fragmentation.

RAM just let the operating system do the writes "as optimal as
possible". The effect of drastically more ram wont be more than 1-5%.

When you use a ramdisk this will differ much ;)

But when you are worried about file persistence you should not do this
*g*

I do not knew any details about your use case, there are other
possiblities:
* writing direkt to the block device, bypassing file systems
* mirror ramdisk writes to other machines for persistence
* ?

On Fri, May 13, 2011 at 12:07:30AM +0900, Yoann M. wrote:
> You're right, doing the buffer myself does not make it faster. For
> writing 10 millions lines, with an array of strings, one string, and no
> homemade-buffer (code is attached) :
> Buffer array : 11.141s
> Buffer string : 9.748s
> No buffer : 10.344s
> 
> Don't you think using more RAM before writing on disk could make the
> process faster ? I thought so, then I'd like to say to File how much RAM
> it can uses to speed things up, because I can use a lot of RAM.
> 
> Regards
> 
> Attachments:
> http://www.ruby-forum.com/attachment/6191/test_write.rb
> 
> 
> -- 
> Posted via http://www.ruby-forum.com/.
> 

-- 
Markus Schirp
Phone:  049 201 / 647 59 63
Mobile: 049 178 / 529 91 42
Web:    www.seonic.net
Email:  info@seonic.net
Seonic IT-Systems GbR
Anton Shatalov & Markus Schirp
Walterhohmannstra�e 1
D-45141 Essen