Re: KirbyBase

Jamey Cribbs wrote:
> [e-mail] wrote:
> 
>> As I said, I'm an Object Guy and Jamey is a Database Guy. What does
>> that mean?
>>
> Sometimes it means we are like the "Odd Couple".  Hey, which one of us 
> is Felix?

I'm Felix. :) No question about it.

>>
> Hal, I considered this, but, again, putting on my "Database Guy" hat, I 
> thought, "What if there is more than one table that wants to link into 
> this table?  And what if this other table wants to use a different "key 
> field" to perform that link?"

Good point. Let's let other people weigh in here.

My gut reaction is: Fine, allow other tables to link via other fields, but
let me use the default when I need/want to.

>>
> Hal, I don't know if you have had a chance to take a look at the beta 
> yet, but I basically tried to implement a uniform way to specify 
> one-to-one links, one-to-many links, and calculated fields in the 
> create_table method.

That's what I thought... I'll have to play with it some. Hopefully
very soon.

>> And my reaction is, well, OK, fine. But first of all, I don't
>> personally
>> see a need for it. (Of course, I might discover a need next week.)
>>
>> Second of all, I am not sure what analogy that would have in object
>> terms. I suppose it would in effect be embedding an array where all the
>>
>> objects are objects derived from the rows of the child class. This
>> bothers me a *little* because all the elements of the array would be
>> of the same type. If you ever assign something else to that array,
>> you'll
>> get an error when you try to insert.
>>  
>>
> As a "Database Guy", I have a hard time understanding why this is a big 
> issue.  First of all, we aren't really "embedding" an array into the 
> linked field.  

No, I still think you are. Given, of course, the fact that all attributes
contain references instead of actually "containing" the object. But that
is beside the point.

> This field simply holds a reference to a KBResultSet 
> instance, which is just a sub-classed Array with some extra attributes.  
> In fact, the result of every #select in KirbyBase is an instance of 
> KBResultSet.  So, if you are link the Order.detail_items field to the 
> Order_Items table, the Order.detail_items field for a particular Order 
> record is going to hold a KBResultSet (i.e. Array) of all records from 
> the Order_Items table that belong to that Order table.

If KBResultSet is an array, then we are in fact embedding an array.

But maybe it is the word "embed" that you are objecting to. "Holds a
reference to" is technically more correct, of course. But even an Array
is only a collection of references -- when I say that an array contains
the string "Hello", that is technically wrong. It contains a reference
to the string "Hello" -- but that is mere nitpicking to me.

I had forgot that #select always returned KBResultSet, probably because
I always treat the result as an array (which it is, but through
inheritance).

Suppose I have an object with a "widgets" field. If I created an object
(not originating in the db) and then tried to store it, would the widgets
attribute have to be a KBResultSet in advance? Or no? If it did, I would
find that unpleasant.

>> Well, there's already some clash between Ruby and the database concept,
>>
>> because db fields have types, and attributes (as variables) don't have
>> types. But this seemed like going a step farther to me -- heightening
>> the clash by creating an array that has to be homogeneous, like a
>> Pascal array.
>>
> But that array is holding table records, just like the result of any 
> #select.  If you object to this array being a KBResultSet instance, it 
> would seem like you would object to the fact that the select method also 
> returns a KBResultSet.

Actually, I didn't object because I forgot. :)

It's different getting stuff *from* the database. I only deal with the
objects in the collection, and they do indeed come back as Foobar objects.

But I would never design an object by saying, "I'll give it this field,
which will normally be a string, and this one, which will normally be
a KBResultSet."

>> Third of all, if you start to have "one-to-one" and "one-to-many" links
>>
>> and such, you start having to distinguish between them. This makes
>> table
>> instantiation, even in the default case, just a tiny bit more
>> difficult.
>> It's creeping complexity. It's very small, but it adds up (and
>> sometimes
>> multiplies if you're not careful, the way probabilities multiply).
>>  
>>
> True, but the flip side is, if you don't allow for specifying different 
> types of links, you paint yourself into a corner down the road *if* you 
> decide to add additional link types.

I guess what I would like is a simple notation for the common, simple
operations; and a complex notation (if necessary) for more complex, rare
operations.

I don't want to complicate 100% of the notation because 1% of the time
somebody else is going to need that feature.

I'm still trying to figure out a way to say: "If there's no link type
specified, it's one-to-one. And if there's no field specified, use the
key field on the child table."

But your point is very well taken that there should be the *capability*
of other types of links.

>> See, there is a sort of "conservation of complexity" in any system. If
>> I knew more information theory, I could express it better.
>>
>> If I write a C program in 600 lines, I can probably write it in Ruby
>> in 100 lines. Where did the complexity go? It left my program and went
>> into the interpreter. That is where it belongs -- under the hood.
>> Information hiding is how humans manage complexity. The concept of the
>> black box is a greater human invention than the discovery of fire or
>> the wheel.
>>
> I agree completely.  My first couple of attempts at adding more 
> complexity to KirbyBase did not honor this concept.  I had the user 
> having to type too much code to tell KirbyBase how to define links, 
> calculated fields, etc.  Hal and I went back and forth, but I finally 
> "got" what he was saying about this.

There's always more than one way to do it.

> 
>>  3. "What type is that key field?" (Given the name, you can find it in
>> the
>>     child table's information.)
>>
> True, but what if they have not created the Person table yet?  How does 
> KirbyBase find the type?  If I have KirbyBase wait till runtime to find 
> the type from the child table, now I have to have KirbyBase open the 
> child table everytime it wants to get the type for the boss field, like, 
> for example, when it needs to check that the user is entering the proper 
> field type during a #insert.

I guess that would be a big problem, one I hadn't thought of at all.

> 
>>  4. "What if Person itself is another complex object?" (Relax, just
>> apply
>>     the same algorithm recursively. Worst that can happen is there is
>>     cyclic data, you'll go into an infinite loop, and I'll have to
>> kill you.)
>>  
>>
> Ok, so how did I solve this problem.  Well, I tried to find a solution 
> that was as simple for the user as possible, but left enough room for 
> further expansion and for the unforseen (see my comments on "key" fields 
> above).  So, here is how you would specify the link between the :boss 
> field and the :person table in the beta:
> 
> create_table(:mytablename,
> :alpha, :Integer,
> :beta, :String,
> :boss, { :DataType => :String, :Link => [:person, :person_id] })
> 
> This simply says, make a link between the value found in the boss field 
> to the person_id field in the person table.  It specifies the field type 
> for boss (:String) and it tells which field to link to (:person_id) 
> within the :person table, therefore we don't have to specify a key field 
> in the :person table.  It's a little more work, but, imo, not much, and 
> you only have to do this once, when you create the table.  After that, 
> KirbyBase handles everything for you automatically.

I'm not saying this is "bad." We're far ahead of things like DBM and Marshal
and such.

I'm just saying that (especially for the common case) THIS:

   :boss, { :DataType => :String, :Link => [:person, :person_id] }

is a little too far from

   :boss, :Person

I'm talking subtlety here. I'm talking the difference between having to
look up syntax "sometimes" or "never."

This is just me. Everybody's mind works differently. We all have different
sets of things that we always forget and have to look up.

When I shut my eyes, I can't even remember what the long version looks like.
I'd have to sit and scribble and figure it out.

If I were coding, I would run through a thought process sort of like this:

"OK, field name followed by type. Type, let's see, that's specified as a
hash. And one element tells the data type. Something like
"datatype" => :String. Wait, is that key a string or a symbol? I think
it's a symbol. Is it :datatype or :Datatype? No, wait, I think it's
:DataType. OK, and the other part of the hash tells about the kind of thing
I'm storing. That key is a symbol too. Or are they both strings? No, I'm
pretty sure they're both symbols. This one is some kind of database term,
what is it, :connect or :join or something? No, wait, I think it's
:one_to_one. No, it's :link_one_to_one. No, no, that's wrong, it's just
:link. But wait, it's capitalized, isn't it. OK, so that is associated
with some kind of array. Two elements, I think? No, it must be three. It's
the table name and the field name and the field type. No, that can't be
right. It's just the table name and the field name. Or does the field name
come first? No, I'm sure the table name comes first. OK, here we go."

If I were doing it the other way, I would think: "OK, we have a symbol
for the field name, followed by a symbol for its type."

Granted there might be times I would need something complex enough to
justify the extra work. But (for me) 99% of the time it would be waste.

>> Now, Jamey's first attempt at this had me writing code in the
>> MYtablename#kb_create method, calling a method named one_to_one_link or
>> some
>> such. It felt very manual to me, like I was hotwiring my car.
>>
> True.  What seemed like a good idea at the time, because I felt like it 
> gave total control of the "guts" to the user, turned out to be too 
> cumbersome for the user.  Hal finally got me to see this.

Very much a matter of opinion where to draw the line. Again, I do favor
"total control" (as in "total flexibility") -- but I think that most
of the time, you shouldn't have to exercise that total control.

>>
> Again, Hal may not have yet seen the examples in the beta of the new 
> version yet.  If you take a look at the example directory in the beta 
> distribution, I have tried to give a good example of one-to-one links 
> (link_test directory), one-to-many links (link_many_test directory), 
> calculated fields (calculated_test directory), etc.

I should study them before mouthing off any more.

> Of course, Hal may look at those examples and still not like them.  I'm 
> interested in hearing his feedback, and everyone else's feedback on the 
> beta.  I've tried to strike a good balance between ease of use for the 
> user and allowing for future functionality, but I'm sure when people 
> look at the beta examples, they are going to have lots of great ideas 
> for improvement.

Ultimately it's your software. It's not a democracy, it's a monarchy. :)

I appreciate the chance to give feedback to you. We all give feedback to
Matz, too, but he properly ignores most of it. Imagine how Ruby would
look if even HALF the RCRs were accepted.

>> Also, a word or two about the kb_create method (called automatically
>> when a row
>> is retrieved, to turn it into an object). In a perfect world, we should
>>
>> "normally" not have to define one at all. I'm thinking of ways to make
>> it
>> usually unnecessary. (In the case of "calculated fields," this might be
>> the
>> very best place to put them, however.)
>>  
> Yep.  My implementation in the beta means you don't have to use the 
> kb_create method to specify link fields, calculated fields, or any of 
> the new functionality.  You can specify it all as arguments to the 
> create_table method.
> 
>> Hope this helps clarify things a little.
>>
>> And I'm glad to see this discussion happening in public, just in case I
>> give
>> Jamey some really stupid advice. :) I'd hate to persuade him into a bad
>> design.
>>
> I'm glad this discussion went public also.  Prior to this, Hal was my 
> only feedback person for the new features and I think I was badgering 
> him to death.  :-)

No, not at all, quite the reverse. Most of the time if I email an author
and said, "This feature would be nice," he'll say, "Yeah, write up a
proposal and send me an email. I'll open it six months from now and
ignore it."

But you -- when I say, "This feature would be nice," I wait 24-48 hours
and BOOM, it shows up in my inbox.

My only fears were 1) that I was nagging you to death, being the Feature
Creep, and 2) that I might be nudging you into wrong design decisions.

> Well, if anyone is still reading this far, thanks!

And thanks from me too.

This is a useful tool already. It's gone from being a manual screwdriver
to being a powered one. Now we are only haggling over the shape of the
rubberized nubs on the handle. :)


Cheers,
Hal