Re: Where To Put Validation Logic?

On Thursday, December 30, 2010 09:08:23 pm Tom Wardrop wrote:
> I'm developing a Mongo object mapper for Ruby, inspired by Candy's
> write-on-set approach. I'm currently still in the planning phase, trying
> to work out the scope of the project and the most appropriate interface.

I may be trying to shoehorn this into what I already know, but I think this 
could be done well as a DataMapper adapter, rather than as an entirely new 
ORM. (Better yet, does the existing DataMapper adapter do what you want?) If 
that's the case, the obvious answer is:

> One question I now find myself asking, is whether to include validation
> helpers, and if so, where should they reside? In the model, or in the
> controller?

DataMapper already includes them, and puts them in the model.

> Will the mapper act as a foundation for all models (like most mappers),
> or will it be more like a tool that models will simply use, if they have
> to - something I'm still undecided on, but which I'm hoping this thread
> may help me with.

I would suggest that if there isn't already a good Ruby API for this kind of 
stuff, start with that. It might take a form that essentially looks like a 
mapper, but it really doesn't need to do any validation.

For example, look at dm-appengine. (I might be a teensy-bit biased here...) 
There's already a somewhat clumsy Java API, which appengine-jruby enhances 
_slightly_. It includes some of what you'd expect from an ORM, in that you do 
get "entity" objects which represent the state of an entity in the datastore. 
But fundamentally, the entire Java API, while it _could_ be used directly (and 
would be much nicer than writing raw GQL), is just a backend for dm-appengine.

The biggest change App Engine makes is removing some features and adding some 
types. This is actually pretty reasonable to do in DataMapper now.

In the case of Mongo, it looks like the biggest thing DataMapper doesn't do 
already is the ability to atomically update a single field -- for instance, it 
might be nice to declare a field as 'immediate' in some sense, so you could 
translate:

joe.n += 1

into something like the Mongo command:

db.people.update( { name:"Joe" }, { $inc: { n : 1 } } );

It's not perfect, because that entails actually retrieving the object first, 
but the problems here are more or less the same as the problems of writing an 
efficient 'update' command for SQL.

> If you look at validations as a means to ensure data integrity,
> then naturally, such logic should reside in the model; the data layer of
> the application. On the other hand, looking at validation as a means to
> validate user input and interactions with the UI, you would think the
> controller would be the better fit.

Not really.

If it's validation, what we're talking about is answering the question, "What 
states are valid for this object, and what are the valid transitions between 
those states?" It makes sense to me that this should be in the model, and that 
it is _always_ a data integrity question.

Think of it this way: If you were to create another controller, or another 
view -- basically, if there were another UI, or even an entirely different 
program accessing your data -- would you want it to obey the same constraints? 
If so, how are you going to share them? The model is the most obvious place.

It also makes sense to me that it belongs in the model, and no lower -- while 
some databases may support a subset of the validations you want, you're going 
to want something Turing-complete. And if you end up writing lots of Turing-
complete validations in your database layer, that's pushing too much of your 
application logic into your database. If you want multiple applications to be 
able to access the same database, either share the model code or expose an API 
on top of your application, something like REST.

I actually tend to favor the REST approach, which would tend to make this a 
moot question -- if you actually use MVC to expose that API, everything is 
going to hit your controllers anyway. But conceptually, all that's really 
doing is using an MVC framework to implement the model as a network service, 
rather than as part of a monolithic application.

> A good example that
> demonstrates this, is assume you had an application that can accept a
> home phone number, a work phone number, and a mobile/cell phone number.
> No field is mandatory, but at least one phone number must be given. You
> could call this multi-field validation.

I would.

> In theory, the data layer of
> your application shouldn't give a damn about whether or not at least one
> phone number is set.

Why not?

The way I see it, either you require at least one phone number, or you don't. 
If you do, presumably your application has a good reason to care, a good 
reason it's inconveniencing the user by forcing them to give you a phone 
number.

If your application doesn't give a damn, why should your users?

> All the data layer should care about is whether it
> has a valid phone or not.

But that's just it -- I see a valid phone as a component of a valid... 
whatever record you're applying that to. (User? Employee? Contact?)

Having a valid record absolutely _is_ the responsibility of the model, at 
least.

> How I see it, such "multi-field" validation
> can be classed as general application logic which should be the
> responsibility of the view/controller, not the data layer, who's only
> responsible should be persisting the data.

Maybe not the data layer, but I don't think it belongs in the controller, and 
certainly not the view!

> Remember, one the main reasons for bringing up this topic, is because
> I'm writing a Mongo object mapper that writes to Mongo when a field is
> set, hence such multi-field validation can't easily be achieved.

Why not?

Take it from the standpoint of locking in a traditional database. Either you 
need some sort of locks to preserve data integrity, in which case you can be 
sure the change you've made is valid, or you don't, and you can't. Take your 
phone number example -- if you were to implement it on top of the API you've 
described, you've got a race condition which could result in a record having 
no phone numbers, and thus being invalid, even though your UI forced the user 
to enter a phone number.

But just because you've decided to dismiss the possibility of that race 
condition (or prevent it some other way) doesn't mean you can't do _any_ 
validation on the entire record. It's no reason you can't have code which 
pulls in the other two phone number fields and checks that at least one of the 
three has a valid phone number before writing all three back out.

And I think that's how DM validations would effectively work. I think that's 
how they _do_ effectively work with SQL. DataMapper is just lazy enough that 
it's certainly capable of reading as much of a record as it needs (and of 
lazily ignoring fields it doesn't) and then writing only the fields it thinks 
it's updating. In fact, if you don't need to retreive an object, you can also 
do things like:

Person.all(:name => 'Joe').update(:n => 5)

I'm pretty sure that translates directly into the SQL you'd expect -- a single 
mass UPDATE statement. I'm pretty sure it'd work more or less the same way in 
a Mongo adapter.