From: Eric Mahurin Date: 2008-02-04T09:30:24+09:00 Subject: Re: [QUIZ] Parsing JSON (#155) ------=_Part_3836_18216911.1202085030823 Content-Type: multipart/alternative; boundary="----=_Part_3837_26110848.1202085030824" ------=_Part_3837_26110848.1202085030824 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline On Feb 1, 2008 7:55 AM, Ruby Quiz wrote: > In honor of that, this week's Ruby Quiz is to write a parser for JSON. My first solution uses an parser generator API similar to my rubyforge "grammar" package. The parser generator is only 89 lines (excluding comment/blank lines). The API is fairly complete to build a variety of parsers. The same API can be use to build a parser, lexer, preprocessor, etc (and you could multi-thread them). The reason this is so simple is that I'm using Ruby as a DSL to specify the language grammar. The file specifying the JSONParser class is only 58 lines. Here is the simple Grammar0 parsing DSL class and the JSONParser class using it: http://pastie.caboo.se/147074 http://pastie.caboo.se/147075 ------=_Part_3837_26110848.1202085030824 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline On Feb 1, 2008 7:55 AM, Ruby Quiz <james@grayproductions.net> wrote:
In honor of that, this week's Ruby Quiz is to write a parser for JSON.

My first solution uses an parser generator API similar to my rubyforge "grammar" package.  The parser generator is only 89 lines (excluding comment/blank lines).  The API is fairly complete to build a variety of parsers.  The same API can be use to build a parser, lexer, preprocessor, etc (and you could multi-thread them).  The reason this is so simple is that I'm using Ruby as a DSL to specify the language grammar.  The file specifying the JSONParser class is only 58 lines.

Here is the simple Grammar0 parsing DSL class and the JSONParser class using it:

http://pastie.caboo.se/147074

http://pastie.caboo.se/147075


------=_Part_3837_26110848.1202085030824-- ------=_Part_3836_18216911.1202085030823 Content-Type: application/octet-stream; name=grammar0.rb Content-Transfer-Encoding: base64 X-Attachment-Id: f_fc8adfdp0 Content-Disposition: attachment; filename=grammar0.rb IyBtaW5pbWFsbHkgZmVhdHVyZWQgR3JhbW1hciBjbGFzcyAoY29tcGxldGVseSBvbi10aGUtZmx5 LCBubyBjb2RlIGdlbikKCmNsYXNzIEdyYW1tYXIwCgogICAgIyBwcm94eSBmb3IgYW5vdGhlciBn cmFtbWFyIG9yIGEgYmxvY2sgYWN0aW5nIGFzIGEgZ3JhbW1hcgogICAgZGVmIGluaXRpYWxpemUo Z3JhbW1hcj1uaWwsJmJsb2NrKSAjIDp5aWVsZDogaW5wdXQsIG91dHB1dCwgbG9va2FoZWFkLCBm YWlsCiAgICAgICAgQGdyYW1tYXIgPSBncmFtbWFyfHxibG9jawogICAgZW5kCiAgICAjIHBhcnNl IGlucHV0IHRvIG91dHB1dAogICAgZGVmIHNjYW4oaW5wdXQsb3V0cHV0LGxvb2thaGVhZD1baW5w dXQuZ2V0Y10sZmFpbD10cnVlKQogICAgICAgIEBncmFtbWFyW2lucHV0LG91dHB1dCxsb29rYWhl YWQsZmFpbF0KICAgIGVuZAogICAgYWxpYXNfbWV0aG9kKDpbXSwgOnNjYW4pCiAgICAjIGFsdGVy bmF0aW9uIG9mICtzZWxmKyBvciArb3RoZXIrIAogICAgZGVmIHwob3RoZXIpCiAgICAgICAgR3Jh bW1hcjAgeyB8aW5wdXQsb3V0cHV0LGxvb2thaGVhZCxmYWlsfAogICAgICAgICAgICBzZWxmW2lu cHV0LG91dHB1dCxsb29rYWhlYWQsZmFsc2VdIHx8CiAgICAgICAgICAgIG90aGVyW2lucHV0LG91 dHB1dCxsb29rYWhlYWQsZmFpbF0KICAgICAgICB9CiAgICBlbmQKICAgICMgc2VxdWVuY2Ugb2Yg K3NlbGYrIGZvbGxvd2VkIGJ5ICtvdGhlcisKICAgIGRlZiArKG90aGVyKQogICAgICAgIEdyYW1t YXIwIHsgfGlucHV0LG91dHB1dCxsb29rYWhlYWQsZmFpbHwKICAgICAgICAgICAgc2VsZltpbnB1 dCxvdXRwdXQsbG9va2FoZWFkLGZhaWxdICYmCiAgICAgICAgICAgIG90aGVyW2lucHV0LG91dHB1 dCxsb29rYWhlYWQsdHJ1ZV0KICAgICAgICB9CiAgICBlbmQKICAgICMgcHJvY2VzcyBvdXRwdXQg ZnJvbSArc2VsZisgYW5kIGludG8gdGhlIG91dHB1dC4KICAgICMgQmxvY2sgc2hvdWxkIHByb2Nl c3Mgb3V0cHV0IGZyb20gZ3JhbW1hciAoK2J1ZispIHRvIHRoZSArb3V0cHV0Ky4KICAgIGRlZiBw cm9jZXNzKGJ1ZjA9Vk9JRCwmY29kZSkgIyA6eWllbGQ6IG91dHB1dCwgYnVmLCByZXN1bHQsIGZh aWwKICAgICAgICBHcmFtbWFyMCB7IHxpbnB1dCxvdXRwdXQsbG9va2FoZWFkLGZhaWx8CiAgICAg ICAgICAgIHNlbGZbaW5wdXQsYnVmPWJ1ZjAuY2xvbmUsbG9va2FoZWFkLGZhaWxdICYmIChjb2Rl W291dHB1dCxidWZdO3RydWUpCiAgICAgICAgfQogICAgZW5kCiAgICAjIGZpbHRlciBvdXRwdXQg ZnJvbSArc2VsZisuICBCbG9jayByZXR1cm5zIGEgc2VxdWVuY2UgZm9yIG91dHB1dCBjb25jYXQu CiAgICBkZWYgZmlsdGVyKGJ1ZjA9Vk9JRCwmY29kZSkgIyA6eWllbGQ6IGJ1ZgogICAgICAgIHBy b2Nlc3MoYnVmMCkgeyB8b3V0LGJ1ZnwKICAgICAgICAgICAgY29kZSA/IG91dC5jb25jYXQoY29k ZVtidWZdKSA6IHRydWUKICAgICAgICB9CiAgICBlbmQKICAgICMgZ3JvdXAgb3V0cHV0IGZyb20g K3NlbGYrLiAgQmxvY2sgcmV0dXJucyBhbiBvYmplY3QgZm9yIG91dHB1dCBhcHBlbmQuCiAgICBk ZWYgZ3JvdXAoYnVmMD1WT0lELCZjb2RlKSAjIDp5aWVsZDogYnVmCiAgICAgICAgcHJvY2Vzcyhi dWYwKSB7IHxvdXQsYnVmfAogICAgICAgICAgICBvdXQgPDwgKGNvZGUgPyBjb2RlW2J1Zl0gOiBi dWYpCiAgICAgICAgfQogICAgZW5kCiAgICAjIHJlcGVhdCArc2VsZisgMCsgdGltZXMgb3B0aW9u YWxseSB0ZXJtaW5hdGVkIGJ5ICt0ZXJtKwogICAgZGVmIHJlcGVhdDAodGVybT1uaWwpCiAgICAg ICAgdGVybSA/IFJpZ2h0IHsgfGd8IHRlcm0gfCBzZWxmICsgZyB9IDogUmlnaHQgeyB8Z3wgc2Vs ZiArIGcgfCBOVUxMIH0KICAgIGVuZAogICAgIyByZXBlYXQgK3NlbGYrIDErIHRpbWVzIG9wdGlv bmFsbHkgdGVybWluYXRlZCBieSArdGVybSsKICAgIGRlZiByZXBlYXQxKHRlcm09bmlsKQogICAg ICAgIHRlcm0gPyBSaWdodCB7IHxnfCBzZWxmICsgKHRlcm0gfCBnKSB9IDogc2VsZiArIHJlcGVh dDAKICAgIGVuZAogICAgIyBNYXRjaCBhIHNpbmdsZSBlbGVtZW50IGZyb20gdGhlIGlucHV0CiAg ICBjbGFzcyBFIDwgR3JhbW1hcjAKICAgICAgICBkZWYgaW5pdGlhbGl6ZShwYXR0ZXJuKQogICAg ICAgICAgICBzdXBlcigpIHsgfGlucHV0LG91dHB1dCxsb29rYWhlYWQsZmFpbHwKICAgICAgICAg ICAgICAgIGlmIHBhdHRlcm49PT1sb29rYWhlYWRbMF0KICAgICAgICAgICAgICAgICAgICBvdXRw dXQgPDwgbG9va2FoZWFkWzBdCiAgICAgICAgICAgICAgICAgICAgbG9va2FoZWFkWzBdID0gaW5w dXQuZ2V0YwogICAgICAgICAgICAgICAgICAgIHRydWUKICAgICAgICAgICAgICAgIGVsc2lmIGZh aWwKICAgICAgICAgICAgICAgICAgICBlcnJvcihwYXR0ZXJuLmluc3BlY3QsIGxvb2thaGVhZCkK ICAgICAgICAgICAgICAgIGVuZAogICAgICAgICAgICB9CiAgICAgICAgZW5kCiAgICBlbmQKICAg ICMgSGFuZGxlIHJlY3Vyc2lvbiB3aXRoIGEgYmxvY2sgdGhhdCBnaXZlcyB0aGUgb3V0ZXIgZ3Jh bW1hciBmcm9tIHRoZSBpbm5lcgogICAgY2xhc3MgUmVjdXJzZSA8IEdyYW1tYXIwCiAgICAgICAg ZGVmIGluaXRpYWxpemUoJmJsb2NrKSAjIDp5aWVsZDogc2VsZgogICAgICAgICAgICBzdXBlcihi bG9ja1tzZWxmXSkKICAgICAgICBlbmQKICAgIGVuZAogICAgIyBIYW5kbGUgcmlnaHQgcmVjdXJz aW9uIChzaG91bGQgcmVhbGx5IGRvIHNvbWUgdGFpbC1jYWxsIG9wdGltaXphdGlvbikKICAgIFJp Z2h0ID0gUmVjdXJzZQogICAgCiAgICAjIEhhY2sgdG8gZW11bGF0ZSBjYWxsYWJsZSBjbGFzc2Vz IChsaWtlIHB5dGhvbikKICAgIChjb25zdGFudHMgPDwgdG9fcykuZWFjaCB7IHxrbGFzc3wKICAg ICAgICBldmFsKCIKICAgICAgICAgICAgZGVmICN7a2xhc3N9KCphcmdzLCZibG9jayk7I3trbGFz c30ubmV3KCphcmdzLCZibG9jayk7ZW5kCiAgICAgICAgICAgIGRlZiBzZWxmLiN7a2xhc3N9KCph cmdzLCZibG9jayk7I3trbGFzc30ubmV3KCphcmdzLCZibG9jayk7ZW5kCiAgICAgICAgIikKICAg IH0KICAgIAogICAgZGVmIHNlbGYuZXJyb3IoZXhwZWN0ZWQsIGxvb2thaGVhZCwgZ290PWxvb2th aGVhZFswXSkKICAgICAgICByYWlzZSgiZXhwZWN0ZWQgI3tleHBlY3RlZH0sIGdvdCAje2dvdH0i KQogICAgZW5kCiAgICBkZWYgZXJyb3IoKmFyZ3MpOyBzZWxmLmNsYXNzLmVycm9yKCphcmdzKTsg ZW5kCiAgICAKICAgICMgTWF0Y2ggbm90aGluZwogICAgTlVMTCA9IEdyYW1tYXIwIHsgdHJ1ZSB9 CiAgICAjIE1hdGNoIHRoZSBlbmQgb2Ygb3VyIGlucHV0IChlbmQtb2YtZmlsZSkKICAgIEVPRiA9 IEdyYW1tYXIwIHsgfF8sXyxsb29rYWhlYWQsZmFpbHwKICAgICAgICAhbG9va2FoZWFkWzBdIHx8 IGZhaWwgJiYgZXJyb3IoIkVPRiIsIGxvb2thaGVhZCkKICAgIH0KICAgICMgTWF0Y2ggYW55IHNp bmdsZSBlbGVtZW50CiAgICBBTlkgPSBHcmFtbWFyMCB7IHxpbnB1dCxvdXRwdXQsbG9va2FoZWFk LGZhaWx8CiAgICAgICAgaWYgbG9va2FoZWFkWzBdCiAgICAgICAgICAgIG91dHB1dCA8PCBsb29r YWhlYWRbMF0KICAgICAgICAgICAgbG9va2FoZWFkWzBdID0gaW5wdXQuZ2V0YwogICAgICAgICAg ICB0cnVlCiAgICAgICAgZWxzaWYgZmFpbAogICAgICAgICAgICBlcnJvcigiQU5ZIiwgbmlsLCAi RU9GIikKICAgICAgICBlbmQKICAgIH0KCiAgICAjIEFuIGVtcHR5IGNvbnRhaW5lciB0aGF0IGRp c2NhcmRzIGFueXRoaW5nIHlvdSBwdXQgaW4gaXQuICBOb3QgYSBncmFtbWFyLgogICAgVk9JRCA9 IE9iamVjdC5uZXcKICAgIGNsYXNzIDw8IFZPSUQKICAgICAgICBkZWYgY29uY2F0KHNlcSk7IHNl bGY7IGVuZAogICAgICAgIGRlZiA8PChzZXEpOyBzZWxmOyBlbmQKICAgIGVuZAoKZW5kCgoK ------=_Part_3836_18216911.1202085030823 Content-Type: application/octet-stream; name=jsonparser.grammar0.rb Content-Transfer-Encoding: base64 X-Attachment-Id: f_fc8adxra1 Content-Disposition: attachment; filename=jsonparser.grammar0.rb cmVxdWlyZSAnZ3JhbW1hcjAnCnJlcXVpcmUgJ3N0cmluZ2lvJwoKY2xhc3MgSlNPTlBhcnNlciA8 IEdyYW1tYXIwCgogICAgZGVmIGluaXRpYWxpemUKCiAgICAgICAgd3MgPSAoRSg/XHMpfEUoP1x0 KXxFKD9cbil8RSg/XHIpKS5yZXBlYXQwLmZpbHRlcgogICAgCiAgICAgICAgZGlnaXQgPSBFKD8w Li4/OSkKICAgICAgICBkaWdpdHMgPSBkaWdpdC5yZXBlYXQxCiAgICAgICAgZSA9IChFKD9lKXxF KD9FKSkgKyAoRSg/Kyl8RSg/LSl8TlVMTCkKICAgICAgICBleHAgPSBlICsgZGlnaXRzCiAgICAg ICAgZnJhYyA9IEUoPy4pICsgZGlnaXRzCiAgICAgICAgaW50ID0gRSg/LSkgKyAoRSg/MCkgfCBk aWdpdHMpIHwgRSg/MCkgfCBkaWdpdHMKICAgICAgICBudW1iZXIgPSAoaW50Lmdyb3VwKCIiKSAr IChmcmFjICsgKGV4cHxOVUxMKSB8IGV4cCB8IE5VTEwpLmdyb3VwKCIiKSkuCiAgICAgICAgICAg ICAgICAgZ3JvdXAoW10pIHsgfG58IG5bMV0uZW1wdHk/ID8gblswXS50b19pIDogbi50b19zLnRv X2YgfQoKICAgICAgICBoZXggPSBkaWdpdCB8IEUoP2EuLj9mKSB8IEUoP0EuLj9GKQogICAgICAg IGNoYXIgPSBFKD9cXCkuZmlsdGVyICsgKAogICAgICAgICAgICAgICAgRSg/XCIpLmdyb3VwIHsg P1wiIH0gfAogICAgICAgICAgICAgICAgRSg/XFwpLmdyb3VwIHsgP1xcIH0gfAogICAgICAgICAg ICAgICAgRSg/XC8pLmdyb3VwIHsgP1wvIH0gfAogICAgICAgICAgICAgICAgRSg/YikuZ3JvdXAg eyA/XGIgfSB8CiAgICAgICAgICAgICAgICBFKD9mKS5ncm91cCB7ID9cZiB9IHwKICAgICAgICAg ICAgICAgIEUoP24pLmdyb3VwIHsgP1xuIH0gfAogICAgICAgICAgICAgICAgRSg/cikuZ3JvdXAg eyA/XHIgfSB8CiAgICAgICAgICAgICAgICBFKD90KS5ncm91cCB7ID9cdCB9IHwKICAgICAgICAg ICAgICAgIEUoP3UpLmZpbHRlciArIChoZXgraGV4K2hleCtoZXgpLmdyb3VwKCIiKSB7IHxzfCBz LnRvX2koMTYpIH0KICAgICAgICAgICAgKSB8CiAgICAgICAgICAgIEFOWQogICAgICAgIHN0cmlu ZyA9IEUoP1wiKS5maWx0ZXIgKyBjaGFyLnJlcGVhdDAoRSg/XCIpLmZpbHRlcikuZ3JvdXAoIiIp CiAgICAgICAgCiAgICAgICAgYWxwaGEgPSBFKD9fKSB8IEUoP2EuLj96KSB8IEUoP0EuLj9aKQog ICAgICAgIGJvb2xlYW4gPSBhbHBoYS5yZXBlYXQxLmdyb3VwKCIiKSB7IHxzfAogICAgICAgICAg ICBjYXNlIHMKICAgICAgICAgICAgd2hlbiAidHJ1ZSIgOiB0cnVlCiAgICAgICAgICAgIHdoZW4g ImZhbHNlIiA6IGZhbHNlCiAgICAgICAgICAgIHdoZW4gIm51bGwiIDogbmlsCiAgICAgICAgICAg IGVsc2UgZXJyb3IoInRydWV8ZmFsc2V8bnVsbCIsIG5pbCwgcykKICAgICAgICAgICAgZW5kCiAg ICAgICAgfQogICAgICAgIAogICAgICAgIHZhbHVlID0gUmVjdXJzZSB7IHx2YWx1ZXwKICAgICAg ICAgICAgZWxlbWVudHMgPSBSaWdodCB7IHxlbGVtZW50c3wKICAgICAgICAgICAgICAgIHZhbHVl ICsgd3MgKyAoRSg/LCkuZmlsdGVyICsgd3MgKyBlbGVtZW50cyB8IE5VTEwpCiAgICAgICAgICAg IH0KICAgICAgICAgICAgYXJyYXkgPSBFKD9bKS5maWx0ZXIgKyB3cyArIChlbGVtZW50cyArIHdz fE5VTEwpLmdyb3VwKFtdKSArIEUoP10pLmZpbHRlcgogICAgICAgICAgICAKICAgICAgICAgICAg cGFpciA9IChzdHJpbmcgKyB3cyArIEUoPzopLmZpbHRlciArIHdzICsgdmFsdWUpLnByb2Nlc3Mo W10pIHsgfGgsa3Z8CiAgICAgICAgICAgICAgICBoW2t2WzBdXSA9IGt2WzFdCiAgICAgICAgICAg IH0KICAgICAgICAgICAgbWVtYmVycyA9IFJpZ2h0IHsgfG1lbWJlcnN8CiAgICAgICAgICAgICAg ICBwYWlyICsgd3MgKyAoRSg/LCkuZmlsdGVyICsgd3MgKyBtZW1iZXJzIHwgTlVMTCkKICAgICAg ICAgICAgfQogICAgICAgICAgICBvYmplY3QgPSBFKD97KS5maWx0ZXIgKyB3cyArIChtZW1iZXJz fE5VTEwpLmdyb3VwKHt9KSArIEUoP30pLmZpbHRlciAKCiAgICAgICAgICAgIHN0cmluZyB8IG51 bWJlciB8IG9iamVjdCB8IGFycmF5IHwgYm9vbGVhbgogICAgICAgIH0KCiAgICAgICAgc3VwZXIo d3MgKyB2YWx1ZSArIHdzICsgRU9GKQoKICAgIGVuZAoKICAgIGRlZiBwYXJzZShzdHIpCiAgICAg ICAgc2VsZi5zY2FuKFN0cmluZ0lPLm5ldyhzdHIpLCBvdXRwdXQ9W10pCiAgICAgICAgcmFpc2Ug aWYgb3V0cHV0Lmxlbmd0aCE9MQogICAgICAgIG91dHB1dFswXQogICAgZW5kCgplbmQKCgo= ------=_Part_3836_18216911.1202085030823--