[ruby-core:33939] Re: [Ruby 1.9-Feature#4211][Open] Converting the Ruby and C API documentation to YARD syntax

On Dec 26, 2010, at 13:00, Loren Segal wrote:

> Feature #4211: Converting the Ruby and C API documentation to YARD syntax
> http://redmine.ruby-lang.org/issues/show/4211
> 
> Author: Loren Segal
> Status: Open, Priority: Normal
> Category: DOC
> 
> The Ruby (high level core/stdlib) documentation and its C API (low level) counterparts currently use two different formats (and tools) to write and generate the final docs. This creates a problem for committers and users alike, where:
> 
> * Documentation is hard to write, because there is no single documentation style to follow (it depends on the API), and the two existing syntaxes are very different.
> * Documentation is harder to read because the style and formatting differ due to the lack of consistent enforcement of a single style.
> * Documentation for the C API (specifically) is harder to find
> 
> Currently, Doxygen @tag style syntax is slowly being introduced to improve the documentation of Ruby's C API, but this does not solve the three issues noted above. I propose to unify the documentation style used in the codebase to a single format (originally on ruby-core:33883[1]) by using YARD[2] syntax, which is very much like the Doxygen @tag style syntax being introduced anyway. Switching to YARD introduces a number of benefits, namely:
> 
> * There would be a single syntax to learn for committers wishing to document code, making it easier to write,
> * The documentation would be formatted and styled consistently across both APIs for users to read,
> * Documentation would be generated by a single tool for both APIs, meaning a simpler workflow for documenters and users wishing to generate the docs themselves.
> 
> I pointed out in the original mailing list that much of the documentation problems come from a lack of unified styling, causing parts of documentation to be (or become) inaccurate due to a variety of "human-error" type issues, and because there are no tools to check the correctness. I believe switching to a unified style and making sure it is used consistently will solve many of those issues even without tooling, because it is easier to manually check for errors with a consistent formatting. Furthermore, using a consistent style allows us to take advantage of our tooling to check basic correctness (or "lint") the docs for simple errors. YARD already has tools to do this kind of thing (and they are easily improved), but they depend on that consistent syntax.
> 
> As far as the C API goes, there is little difference in the existing doxygen syntax (except that I'd suggest the '@tag' instead of Doxygen's alternative '\tag' prefixes, for compatibility). As I wrote in the above ruby-core thread, YARD can already handle most of the written doxygen documentation. Granted, a lot of the support for actually *generating* documentation for a straight "C" style API is missing in YARD, but as I mentioned, I would be willing to improve this support if there is a willingness by the ruby-core developers to create a unified documentation style.

I highly support having a unified documentation tool and style for Ruby, but I personally dislike an enforced template like doxygen requires.

I find the doxygen-style template leads to stilted wording and low-quality documentation as the author merely has to check all the boxes to consider their work "complete".  I've read documentation generated by such tools many times and have always found them difficult to navigate as there is not enough context to understand the interaction between all the components of an API.

If ruby adopts a documentation minimum standard I don't think an enforced template should be our only requirement.

> h3. Steps forward:
> 
> We should first discuss whether the Ruby core developers are in favor of such a change. In the event that they are, we would have to look at a few things:
> 
> * Maintaining compatibility with RDoc (or adding YARD's @tag style support to RDoc) for the high level Ruby API docs while converting the syntax. I have a few ideas on how this can be done.
> * Improving YARD's ability to generate HTML for straight "C" codebases (which I can implement, if we get this far)
> * Any other issues / reservations raised by the Ruby core team
> 
> This is certainly a large proposal, and has some compatibility snags (with RDoc, for instance). Regardless, I think these issues can be worked around or dealt with for the most part, and the benefits of much improved documentation, which Ruby really needs, are certainly worth the effort.

I question merging YARD into Ruby and replacing RDoc.

Why aren't extensions or enhancements to RDoc being made instead?

Based on download numbers from rubygems.org it appears that a minority (about 25%) of rubyists prefer YARD to rdoc.  There have been 8300 downloads of RDoc's latest release (1100/day over the last 90 days) compared to 1200 for YARD (285/day).  Since March 2010 when the ri data format changed there have been 12000 downloads of rdoc-data which is 20% of YARD's total downloads since March 2007 of 60000.

If every YARD downloader prefers doxygen-style syntax, for me, the numbers imply that extensions should be made to RDoc to support different types of syntax over replacing RDoc.

RDoc has several APIs for adding such extensions, including plugins allowing alternate generators and additional directives:

http://rdoc.rubyforge.org/RDoc/Generator.html
http://rdoc.rubyforge.org/RDoc/RDoc.html
http://rdoc.rubyforge.org/RDoc/Markup/PreProcess.html#method-c-register

(I do not know if the APIs are sufficient as I've never received any communication from anyone attempting to use them for this task.  I would love some feedback on this matter.)

Finally, I'm unsure why you've never emailed me or otherwise contacted me to propose adding extensions to RDoc to support YARD-style comments.  I am not opposed to an extension that would support it.

If ruby-core preferred such an extension to RDoc over doxygen I would be happy to have it as part of RDoc.