From: Stefan Mahlitz <stefan@...>
Date: 2007-08-23T03:46:49+09:00
Subject: Re: Test if file is binary ?

Simon Krahnke wrote:
> * Stefan Mahlitz <stefan@mahlitz-net.de> (09:25) schrieb:
> 
>> Maybe this was a bad example. You are right, the xml-file would be best
>> treated by clearcase as application/xml or text/xml. This did not work
>> (and I was bitten by this recently - so this strange behaviour was fresh
>> when I read your email).
> 
> Note that Subversion would just treat the file as binary and process it
> with its binary diff.

I didn't know this. Thanks for the info.

>> But I cannot see the problem with text-files containing long lines. If I
>> write a single paragraph with more than 1000 or 8000 characters - why
>> shouldn't this be text?
> 
> If that's really a paragraph there is no problem, except maybe of style.
> (When wrapped in lines of 80 characters, it's 100 lines!)

Agreed. But it is still text - which was the point I tried to make.

>> Why do you think it is not readable?
> 
> I think that an XML file that has huge lines is unreadable since a
> human wouldn't recognize any structure, when all the elements are on a
> single line.

My question was directed to the 8000 char-paragraph. I even find small
xml-files unreadable - so I completely agree with you that 8000 chars of
xml-data in a single line is far from being readable by a human. Anyway
- xml is meant to be processed by machines.

But even this case I would classify as text (I'm changing my earlier
definition slightly) if it does not contain binary data. The xml in a
file is semantics. And I assume the question text or binary refers to
syntax.

>>> One small change in that line will produce a patch of more than 8000
>>> characters. And if that change is at the end of the line the diff tool
>>> will have to use 4 pages of memory for the compare.
>> Sorry, I fail to see your point.
> 
> That's another point. Or actually two. The hard one being the limitation
> of the software used: It may have a maximum line length.

If I understand the original poster correctly he wants to
programmatically detect whether a file is "binary or text". My point was
that he shouldn't restrict his program artifically - but this depends on
context.

>> Are we really judging whether a file is text by how much memory pages
>> a diff will take or how many characters a patch has?
> 
> No, this has nothing to do with being text, just with being well suited
> as input to a diff algorithm.
> 
> Text usually is suited as well as everything else that is line oriented
> and typical changes affect only one or a few neighboring lines.

These are things I'm normally not concerned about, that's why I couldn't
follow that subject change.

Do I summarize correctly that depending on the purpose of the check one
could use a maximum line length - or any other of the posted approaches?

Aka 'use the right tool for the job' + 'There is no single answer to
this question'?

Stefan