From: Mark Hahn Date: 2002-01-16T03:25:47+09:00 Subject: RE: memory usage question > you should consider using a database Doing 200 million lookups in a db would take way too long for my script. If there is no other work-around than using a DB, then I'll have to go back to Java, which can handle this kind of memory usage. I'm dissapointed that in one of the first quick scripts I went to Ruby for, Ruby couldn't do the job. > Don't try to compete with me on oldtimer stories. I spent two summers programming an IBM 650 which had NO core memory. It was a 1-bit processor with only drum memory. Even the registers were on the drum. It used about 100 vacumn tubes to implement the cpu. > Even today when RAM approaches a gigabyte in daily life The server I'm running this on has 4 gigabytes of ram. -----Original Message----- From: Hal E. Fulton [mailto:hal9000@hypermetrics.com] Sent: Tuesday, January 15, 2002 10:10 AM To: ruby-talk ML Subject: Re: memory usage question ----- Original Message ----- From: "Mark Hahn" To: "ruby-talk ML" Sent: Tuesday, January 15, 2002 11:37 AM Subject: memory usage question > I need to write a script that will use a hash with 4 million strings of 16 > chars each. As a test I wrote this quick script: > > h = {}; s = 'aaaaaaaaaaaaaaaa' > for i in 1..4_000_000 > h[i] = (s = s.succ) > end > > This ran relatively fast, but it consumed 265 megs of windows memory and > failed with out of memory error at around 3 million elements. > > Why did Ruby fail at 265 megs and is there another way for me to do this? Four million strings, eh? :) Buncha young kids wanna allocate data structures with four million items... in my day, we had 64K of RAM and part of that was the OS and the runtime. The compiler wouldn't all fit in memory at once. It was swapped out to this clunky external Shugart floppy disk drive that would walk across the table if you didn't tape it down. It made a noise like a boa constrictor gagging on a weasel. And the compiler couldn't handle arrays bigger than a thousand elements. And we LIKED IT... But seriously... :) Even today when RAM approaches a gigabyte in daily life (and your PC may have a terabyte drive soon), memory ain't always the way to go. Eventually you should consider using a database or something instead. If you want something that will have a low impact on your code, try the dbm interface. Very easy, not as much overhead as a "real" database. There's also gdbm and maybe one or two other flavors. Not sure what the differences are. Last time I tried, gdbm worked better for me on Windows. I think it's part of the standard distribution. Hope this helps. Hal