From: tamouse mailing lists Date: 2013-04-30T09:57:53+09:00 Subject: Re: writing lines to a file --089e014942269eb3c004db897dd2 Content-Type: text/plain; charset=UTF-8 On Apr 29, 2013 6:38 PM, "peteV" wrote: > > On 04/29/2013 07:34 PM, tamouse mailing lists wrote: >> >> >> On Apr 29, 2013 4:37 PM, "Joel Pearson" wrote: >> > >> > Or you could do the whole operation with a single read, string >> > processing, and a single write >> >> This can be a problem if the source file is large, but certainly, yes. > > the source file is 17000 + lines :) well, ok, that would have meant opening, appending a line and closing it 17,000 times in the original. it is sometimes hard to conceive of computers doing actual work, but that is a lot of unnecessary opens and closes. otoh, 17,000 lines may not costbthat much ram if read in all at one time, filtered to a buffer, and appended en masse. so joel does have a point. a billion lines is more into the problem space of too much to buffer. (or in the case i am familiar with, 4GB database file) --089e014942269eb3c004db897dd2 Content-Type: text/html; charset=UTF-8


On Apr 29, 2013 6:38 PM, "peteV" <pete0verse@gmail.com> wrote:
>
> On 04/29/2013 07:34 PM, tamouse mailing lists wrote:
>>
>>
>> On Apr 29, 2013 4:37 PM, "Joel Pearson" <lists@ruby-forum.com> wrote:
>> >
>> > Or you could do the whole operation with a single read, string
>> > processing, and a single write
>>
>> This can be a problem if the source file is large, but certainly, yes.
>
> the source file is 17000 + lines :)

well, ok, that would have meant opening, appending a line and closing it 17,000 times in the original. it is sometimes hard to conceive of computers doing actual work, but that is a lot of unnecessary opens and closes.

otoh, 17,000 lines may not costbthat much ram if read in all at one time, filtered to a buffer, and appended en masse. so joel does have a point.

a billion lines is more into the problem space of too much to buffer. (or in the case i am familiar with, 4GB database file)

--089e014942269eb3c004db897dd2--