From: Lionel Bouton <lionel-subscription@...>
Date: 2008-04-29T03:50:07+09:00
Subject: Re: Transpose a large file(>2GB)

Xavier Noria wrote:
> On Apr 28, 2008, at 19:20 , Lionel Bouton wrote:
> 
>> With this method you'll always read/write sequential data on disk  
>> when it's possible with as large data chunks as possible without  
>> hitting the swap.
> 
> In what sense is it sequential?

It doesn't read or write one byte at a time :-) Which would be the 
result of a na誰ve transpose algorithm using direct disk access.

> These approeaches require prepending/ 
> appending *rectangular* chunks of text in the ouput file?
> 
> 

There's no way around that if you can't fit all the matrix in memory, 
the only thing you can do is optimize the rectangles to minimize the 
number of chunks to read...

Lionel