[ruby-core:106813] [Ruby master Misc#16944] questions about Net::IMAP patches

Issue #16944 has been updated by nevans (Nicholas Evans).


FYI: this issue should be closed as well.  It's all been addressed on the `net/imap` issue tracker.  Thanks!

----------------------------------------
Misc #16944: questions about Net::IMAP patches
https://bugs.ruby-lang.org/issues/16944#change-95624

* Author: nevans (Nicholas Evans)
* Status: Open
* Priority: Normal
----------------------------------------
I work on a product that contains many changes to Net::IMAP, both to the client and especially to the response parser. One part of it is an `EventMachine` daemon that makes extensive use of `Net::IMAP::ResponseParser`, but only uses a little bit of the other code from the `Net::IMAP` client (the client is used in other places though).

I've submitted a few smaller patches already (better CAPABILITY parsing #16626, [ID][RFC2971] #16610, [NAMESPACE][RFC2342] #16627), and I have many more that I haven't gotten around to cleaning up for a patch yet (e.g. [remembering server-sent CAPABILITIES][RFC3501] and even using them in a few places, [COMPRESS=DEFLATE][RFC4978], [UIDPLUS][RFC4315], [SASL-IR][RFC4959], [SPECIAL USE][RFC6154], [AUTH=XOAUTH2][XOAUTH2] and [AUTH=OAUTHBEARER][RFC7628], [OBJECTID][RFC8474], [UTF8=ACCEPT][RFC6855], [ENABLE][RFC5161], and many many [RFC3501][RFC3501] parser bugs).

But before I prepare some of the bigger patches, I should ask:

* I would find it easier if `net/imap.rb` were split into several files. Would a patch like that be welcome and accepted?
  * especially: `net/imap/response_parser.rb` for `Net::IMAP::ResponseParser`.
  * But also: `net/imap/response_data.rb` for all of the response data `Struct`s (e.g. `TaggedResponse`, `FetchData`, `MailboxList`, etc),
  * and also: `net/imap/request_data.rb` (e.g. all of the `#send_data`/`#validate` classes, like `Atom`, `Literal`, `QuotedString`, etc).

* I would really like to split `Net::IMAP::ResponseParser` into three parts: the parser, the lexer, and a "builder" (so the data structures built by the parser can be more easily customized).  Would a patch that does just this (and nothing else) this be accepted?

* I have made many adhoc changes to the response parser over the years as I've worked around various server bugs or bugs in the parser. But the biggest issue seems to be that the parser is still largely based on an obsolete IMAP dialect. It's clear from looking at the comments and code around astrings (just as one example) that much of it pre-dates [RFC3501] (March 2003, over 17 years ago!) and was written to the older [RFC2060] (December 1996) standards. Although I can continue creating extracting and cleaning up smaller patches to update smaller pieces, would a much larger cleanup project to more closely match [RFC3501] be accepted?

* And lastly, what do you think about rewriting the parser/lexer using racc and/or ragel?  Unlike the above, I haven't even started on this project yet. And I wouldn't want to even start on this project until after the above has been taken care of, and the test suite has been added to. However, CPU is the limiting resource for my IMAP daemons, and I've wanted to rewrite or replace our parser for years now.  Even using EventMachine on some very powerful servers, I usually can't hold more than ~500 mostly idle IMAP connections per OS process before CPU is fully maxed (*far* fewer if they are very active at once).  For our newest project, my team has been using golang instead of ruby, even though the go IMAP library has fewer features and is more difficult to work with. That makes me sad. :(

[RFC2060]: https://tools.ietf.org/html/rfc2060
[RFC2342]: https://tools.ietf.org/html/rfc2342
[RFC2971]: https://tools.ietf.org/html/rfc2971
[RFC3501]: https://tools.ietf.org/html/rfc3501
[RFC4315]: https://tools.ietf.org/html/rfc4315
[RFC4959]: https://tools.ietf.org/html/rfc4959
[RFC4978]: https://tools.ietf.org/html/rfc4978
[RFC5161]: https://tools.ietf.org/html/rfc5161
[RFC6154]: https://tools.ietf.org/html/rfc6154
[RFC6855]: https://tools.ietf.org/html/rfc6855
[RFC7628]: https://tools.ietf.org/html/rfc7628
[RFC8474]: https://tools.ietf.org/html/rfc8474
[XOAUTH2]: https://developers.google.com/gmail/imap/imap-extensions



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>