From: James Gray Date: 2010-11-04T05:31:56+09:00 Subject: [ruby-core:33047] [Ruby 1.9-Feature#4017] [PATCH] CSV parsing speedup Issue #4017 has been updated by James Gray. Ah, sorry. I misunderstood. I think I get it now. Basically, read_limit isn't a limit so much as a read-at-a-time control dial that can be used to tune parsing speed. It's set to a reasonable default and users can tune it the their specific needs. Am I understanding better now? I was thinking it was bubbling the limit used by methods like IO#gets() up to the user. I've long wanted to do that, to account for data like: "...gigs of data containing anything but a quote Beyond that, the refactoring sounds neat to me. Under what circumstances does it turn out to be slower than the old parser? It seems like there would have to be some. I don't say that because the old parser was well tuned either. It's just that we've now shifted the focus of the parser (from newlines to quotes, if I'm understanding correctly). For example, if each line is 20 simple fields, all unnecessarily but legally quoted, do the increased IO operations slow things down? I guess if there are such cases, we just need to decide if the ones we are optimizing for are common. My thoughts there are probably that the test sweet isn't an ideal speed measuring tool. We might want to grab several realistic CSV documents to use for that purpose. Any thoughts on my babbling here Tim? Am I making sense. ---------------------------------------- http://redmine.ruby-lang.org/issues/show/4017 ---------------------------------------- http://redmine.ruby-lang.org