From: Robert Klemme <shortcutter@...>
Date: 2009-03-11T23:24:31+09:00
Subject: Re: sorting question

--00221532c7186afc5e0464d8a762
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

2009/3/11 Miroslaw Niegowski <trednu@gmail.com>:
> Hi,
> I have following log in simple format:
>
> -- START - DATE (YYYY-MM-DD HH:MM:SS.sss)
> <xml>
> ....
> </xml>
> -- END - DATE (YYYY-MM-DD HH:MM:SS.sss)
>
> -- START - DATE (YYYY-MM-DD HH:MM:SS.sss)
> <xml>
> ....
> </xml>
> -- END - DATE (YYYY-MM-DD HH:MM:SS.sss)
>
> -- START - DATE (YYYY-MM-DD HH:MM:SS.sss)
> <xml>
> ....
> </xml>
> -- END - DATE (YYYY-MM-DD HH:MM:SS.sss)
>
> -- START - DATE (YYYY-MM-DD HH:MM:SS.sss)
> <xml>
> ....
> </xml>
> -- END - DATE (YYYY-MM-DD HH:MM:SS.sss)
>
> What is the most efficient way to sort it in ruby by START DATE & TIME?
> Log size is generally smaller than 1GB...
> I was thinking of :
> 1) loading the file into memory
> 2) splitting into hash (date | xml)
> 3) sorting key values
> 4) output
>
> Is there a better way to do this ? I'm mostly concerned with memory
> consumption...

You can at least optimize it like this

1. while reading the file build up a structure (I'd prefer Array) in memory

2. sort it (using sort_by or sort!)

3. output it

Sample attached.  You can as well add parsing of timestamps which
_might_ save a bit of memory.

Kind regards

robert

-- 
remember.guy do |as, often| as.you_can - without end

--00221532c7186afc5e0464d8a762
Content-Type: application/octet-stream; name="lsort.rb"
Content-Disposition: attachment; filename="lsort.rb"
Content-Transfer-Encoding: base64
X-Attachment-Id: f_fs63sxwa0

IyEvYmluL2VudiBydWJ5MTkKCkxvZ1JlY29yZCA9IFN0cnVjdC5uZXcgOnN0YXJ0LCA6ZW5kLCA6
eG1sCgpkYXQgPSBbXQpjdXJyZW50ID0gbmlsCgpEQVRBLmVhY2ggZG8gfGxpbmV8CiAgY2FzZSBs
aW5lCiAgd2hlbiAlcnteLS0gU1RBUlQgLSBEQVRFIFwoKFteKV0qKVwpfQogICAgY3VycmVudCA9
IExvZ1JlY29yZC5uZXcgJDEsIG5pbCwgIiIKICB3aGVuICVye14tLSBFTkQgLSBEQVRFIFwoKFte
KV0qKVwpfQogICAgY3VycmVudC5lbmQgPSAkMQogICAgZGF0IDw8IGN1cnJlbnQKICAgIGN1cnJl
bnQgPSBuaWwKICBlbHNlCiAgICBjdXJyZW50LnhtbCA8PCBsaW5lIGlmIGN1cnJlbnQKICBlbmQK
ZW5kCgpkYXQuc29ydF9ieSB7fHJ8IFtyLnN0YXJ0LCByLmVuZF19LmVhY2ggZG8gfHJ8CiAgcHJp
bnQgIi0tIFNUQVJUIC0gREFURSAoIiwKICAgIHIuc3RhcnQsCiAgICAiKVxuIiwKICAgIHIueG1s
LAogICAgIlxuLS0gRU5EIC0gREFURSAoIiwKICAgIHIuZW5kLAogICAgIilcblxuIgplbmQKCiMg
YWx0ZXJuYXRpdmUsIHNvcnQgaW4gcGxhY2UKZGF0LnNvcnQhIHt8YSxifCBjID0gYS5zdGFydCA8
PT4gYi5zdGFydDsgYyA9PSAwID8gYS5lbmQgPD0+IGIuZW5kIDogY30KZGF0LmVhY2ggZG8gfHJ8
CiAgcHJpbnQgIi0tIFNUQVJUIC0gREFURSAoIiwKICAgIHIuc3RhcnQsCiAgICAiKVxuIiwKICAg
IHIueG1sLAogICAgIlxuLS0gRU5EIC0gREFURSAoIiwKICAgIHIuZW5kLAogICAgIilcblxuIgpl
bmQKCgpfX0VORF9fCi0tIFNUQVJUIC0gREFURSAoWVlZWS1NTS1ERCBISDpNTTpTUy5zc3MpCjx4
bWw+Ci4uLi4KPC94bWw+Ci0tIEVORCAtIERBVEUgKFlZWVktTU0tREQgSEg6TU06U1Muc3NzKQoK
LS0gU1RBUlQgLSBEQVRFIChZWVlZLU1NLUREIEhIOk1NOlNTLnNzcykKPHhtbD4KLi4uLgo8L3ht
bD4KLS0gRU5EIC0gREFURSAoWVlZWS1NTS1ERCBISDpNTTpTUy5zc3MpCgotLSBTVEFSVCAtIERB
VEUgKFlZWVktTU0tREQgSEg6TU06U1Muc3NzKQo8eG1sPgouLi4uCjwveG1sPgotLSBFTkQgLSBE
QVRFIChZWVlZLU1NLUREIEhIOk1NOlNTLnNzcykKCi0tIFNUQVJUIC0gREFURSAoWVlZWS1NTS1E
RCBISDpNTTpTUy5zc3MpCjx4bWw+Ci4uLi4KPC94bWw+Ci0tIEVORCAtIERBVEUgKFlZWVktTU0t
REQgSEg6TU06U1Muc3NzKQo=
--00221532c7186afc5e0464d8a762--