From: Zev Blut <rubyzbibd@...>
Date: 2005-05-19T20:55:50+09:00
Subject: Re: [ ruby-Bugs-1930 ] CGI::escapeHTML escapes the ampersand in an existing escape command

Konbanwa,

On Thu, 19 May 2005 20:08:57 +0900, nobuyoshi nakada  
<nobuyoshi.nakada@ge.com> wrote:
>
> At Thu, 19 May 2005 17:18:54 +0900,
> Zev Blut wrote in [ruby-core:05044]:
>> > It is exactly expected behavior.
>>
>> OK.  It would be nice if that was in the documentation somewhere, so
>> that other people do not make the same mistake.
>
> $ ri CGI.escapeHTML | cat
> -------------------------------------------------------- CGI::escapeHTML
>      CGI::escapeHTML(string)
> ------------------------------------------------------------------------
>      Escape special characters in HTML, namely &\"<>
>
>        CGI::escapeHTML('Usage: foo "bar" <baz>')
>           # => "Usage: foo &quot;bar&quot; &lt;baz&gt;"
>

Thanks, I should have double checked the documents before making the
above comment.

>> > The way to let escapeHTML not to escape a particular ampersand is
>> > another story.
>>
>> I can see the need for the current behavior and of course I have the
>> need for the changed behavior.  Two techniques for providing a
>> preserving escape come to my mind.  One would be to add an optional
>> boolean argument that when true will keep escape commands, the
>> default value is false and thus current usage will not change. Another
>> is to add another method that implements the patch provided in the
>> report.
>
> Leaving ampersands breaks round-trip by escapeHTML and unescapeHTML.
> I think it would be a problem.

I don't think this breaks round-trip usage.  See below:
----------------------------------------------------------------------
require 'cgi'

def patch_escape(string)
     # using tmp to prevent line wrapping in this mail.
     tmp = string.gsub(/&(?!(\S+);)/n, '&amp;').gsub(/\"/n, '&quot;')
     tmp.gsub(/>/n, '&gt;').gsub(/</n, '&lt;')
end

# sample text is emoji
txt = 'Hello & 500 &yen; please." &#63651; moji "'

puts txt
#=> Hello & 500 &yen; please." &#63651; moji "

puts CGI.escapeHTML(txt)
#=> Hello &amp; 500 &amp;yen; please.&quot; &amp;#63651; moji &quot;

puts patch_escape(txt)
#=> Hello &amp; 500 &yen; please.&quot; &#63651; moji &quot;

puts CGI.unescapeHTML(patch_escape(txt))
#=> Hello & 500 &yen; please." &#63651; moji "

puts CGI.unescapeHTML(CGI.escapeHTML(txt))
#=> Hello & 500 &yen; please." &#63651; moji "
----------------------------------------------------------------------

I hope the above makes it clear.  Sorry for the use of emoji, but that
was what alerted me to my usage problem in the first place.

Best,
Zev