From: Todd Benson <caduceass@...>
Date: 2008-04-02T02:13:51+09:00
Subject: Re: should I use a database or a flat file?

On Tue, Apr 1, 2008 at 11:55 AM, Lionel Bouton
<lionel-subscription@bouton.name> wrote:
> Todd Benson wrote:
>
>  > I'm going to slightly disagree with Lionel -- and also Robert -- on
>  > this one.  First of all, a database is not necessarily just for
>  > concurrency.  It's for data integrity
>
>  Yes I agree (as explained below concurrency is what I consider the main
>  problem to solve to enforce data integrity). That said if you write your
>  data in one pass as the OP, you don't need data integrity in the storage
>  layer... rename is atomic : you either renamed the temp file to its
>  final position before a crash or not.
>
>  The problem are partial updates where you need to maintain consistancy.
>  And on the top of my head the only problems with partial updates are :
>  - concurrent accesses (most common, counting both concurrent read and
>  write accesses),
>  - crashes (fortunately less common and can even be adressed by backups
>  in many cases).
>
>  These are why I disagree with people wanting to push all the consistency
>  logic into the applicaltion layer on database-backed applications with
>  concurrent access (like often advocated for Rails). It's simply not
>  doable without recoding the whole concurrent access manager  and
>  log-based/MVCC/... crash resistance of the database in the application
>  layer (good luck with that).
>
>  Lionel.

Maybe we are talking about different things.  By data integrity, I
mean you can be certain not just that the data was entered correctly,
but also that it coincides with the relationships present.  In a
modified version of the OP's model, for example...

Cat1
-name1
-name2
-name3
-etc...

Cat2
-name4
-name5
-name6
-etc...

Cat3
-name1
-name2
-name3
etc...

Note the same category names, but in different categories.

Now, surely, you can say, "Well, the application logic will take care
of that ambiguity."  But I say we should continue to separate
application logic from data logic.

I'm no CS guy, so I don't know the correct terms for this, but I do
see the potential pratfalls.

There certainly is a time and place for this, but I've found it's
usefulness generally not that beneficial.

Todd