Re: KirbyBase

[e-mail] wrote:

>As I said, I'm an Object Guy and Jamey is a Database Guy. What does
>that mean?
>
>  
>
Sometimes it means we are like the "Odd Couple".  Hey, which one of us 
is Felix?

>I was working on an app and using KirbyBase. Because of the work Jamey
>had done recently, creating objects from the database records was much
>easier.
>
>But I noticed something. My fields were all simple types. (After all,
>that's how databases work.) But I was taking my smaller objects and
>using them to build bigger objects. In most cases, I was storing a key
>into another table. So I would look up a record, and four of its fields
>
>would actually be keys into other tables. I'd then look up those four.
>I'd then embed my four little objects inside an object corresponding to
>
>the original row.
>
>I had a feeling that I was doing too much work, that some of this could
>
>be done for me.
>
>My thoughts were like this: OK, we have primitive types such as integer
>
>and string. From OOP, I have a habit of thinking of classes as abstract
>
>types (see Bertrand Meyer -- I think).
>
>So if I want an object to have a member which is (e.g.) a Person, why
>can't I treat Person just like String and Integer?
>
>In Ruby, we don't think of variables or attributes as being typed. I
>make an exception in the case of retrieving objects from databases,
>because I am very used to the idea of fields having types. (Although
>someday I will comment on that also.)
>
>After all, even in Ruby, we often think to ourselves, "This member
>will be a string" -- although if we are not careful, we might assign
>any other thing to it. But I won't go off on too much of a tangent
>here.
>
>So here I am. I'm thinking: Treat simple types (simple fields) the way
>we already do. We store a type for each field. Let's store the complex
>types the same way... but in the actual table, just store a key into
>another table.
>
>That requires the table name and key field name to be specified
>somehow,
>of course. My take: Let's get slightly railsy and assume the class name
>
>lowercased is the table name (e.g. Person => person.tbl). That means we
>
>  
>
Only problem with assuming this is that if the person table does not 
have a custom record class defined (default is a Struct), this won't 
work.  I'll show you an example from my beta below to show you how I 
ended up working it.

>only have to specify the field name explicitly (and I've been thinking
>of a way to eliminate that also, by optionally establishing a known
>"key field" for a table).
>  
>
Hal, I considered this, but, again, putting on my "Database Guy" hat, I 
thought, "What if there is more than one table that wants to link into 
this table?  And what if this other table wants to use a different "key 
field" to perform that link?"

>I was thinking "composite objects." I was thinking "embedding simpler
>objects inside more complex ones." I was thinking "this is how I can
>easily and transparently marshal my objects."
>
>When I mentioned this to Jamey, he said in effect, "Oh, you want one-
>to-one links between database tables." That's Mr. Database talking to
>Mr. Object. He was quite right, of course, but it confused me to think
>that way.
>
>It was as if my mechanic said, "Oh, you're trying to use this
>solenoid/starter assembly to initiate sequential explosions inside your
>
>cylinders with the aid of these spark plugs." To which I would say,
>"Uh, I just want to start my car." (As it happens, I really don't know
>much about cars or engines. That's a hardware problem.)
>
>So Mr. Database says, "Let's implement one-to-many also." (And he may
>have done others also, I don't have a KB install in front of me.)
>  
>
Hal, I don't know if you have had a chance to take a look at the beta 
yet, but I basically tried to implement a uniform way to specify 
one-to-one links, one-to-many links, and calculated fields in the 
create_table method.

>And my reaction is, well, OK, fine. But first of all, I don't
>personally
>see a need for it. (Of course, I might discover a need next week.)
>
>Second of all, I am not sure what analogy that would have in object
>terms. I suppose it would in effect be embedding an array where all the
>
>objects are objects derived from the rows of the child class. This
>bothers me a *little* because all the elements of the array would be
>of the same type. If you ever assign something else to that array,
>you'll
>get an error when you try to insert.
>  
>
As a "Database Guy", I have a hard time understanding why this is a big 
issue.  First of all, we aren't really "embedding" an array into the 
linked field.  This field simply holds a reference to a KBResultSet 
instance, which is just a sub-classed Array with some extra attributes.  
In fact, the result of every #select in KirbyBase is an instance of 
KBResultSet.  So, if you are link the Order.detail_items field to the 
Order_Items table, the Order.detail_items field for a particular Order 
record is going to hold a KBResultSet (i.e. Array) of all records from 
the Order_Items table that belong to that Order table.

>Well, there's already some clash between Ruby and the database concept,
>
>because db fields have types, and attributes (as variables) don't have
>types. But this seemed like going a step farther to me -- heightening
>the clash by creating an array that has to be homogeneous, like a
>Pascal array.
>
>  
>
But that array is holding table records, just like the result of any 
#select.  If you object to this array being a KBResultSet instance, it 
would seem like you would object to the fact that the select method also 
returns a KBResultSet.

>Third of all, if you start to have "one-to-one" and "one-to-many" links
>
>and such, you start having to distinguish between them. This makes
>table
>instantiation, even in the default case, just a tiny bit more
>difficult.
>It's creeping complexity. It's very small, but it adds up (and
>sometimes
>multiplies if you're not careful, the way probabilities multiply).
>  
>
True, but the flip side is, if you don't allow for specifying different 
types of links, you paint yourself into a corner down the road *if* you 
decide to add additional link types.

>See, there is a sort of "conservation of complexity" in any system. If
>I knew more information theory, I could express it better.
>
>If I write a C program in 600 lines, I can probably write it in Ruby
>in 100 lines. Where did the complexity go? It left my program and went
>into the interpreter. That is where it belongs -- under the hood.
>Information hiding is how humans manage complexity. The concept of the
>black box is a greater human invention than the discovery of fire or
>the wheel.
>
>  
>
I agree completely.  My first couple of attempts at adding more 
complexity to KirbyBase did not honor this concept.  I had the user 
having to type too much code to tell KirbyBase how to define links, 
calculated fields, etc.  Hal and I went back and forth, but I finally 
"got" what he was saying about this.

>Now, suppose I specify a table with KirbyBase that has two simple
>fields,
>an integer and a string. We do it something like this (I probably am
>forgetting the method name):
>
>  create_table(:mytablename,
>               :alpha, :Integer,
>               :beta, :String)
>
>Fine and dandy, nice and simple.
>
>Now suppose we add another field -- I'll abandon the Greek alphabet and
>
>call it "boss" which will be a Person object.
>
>In an absolutely perfect world, this would be "just another type." The
>software would read my mind and do exactly the right thing, and all I
>would type is:
>
>  create_table(:mytablename,
>               :alpha, :Integer,
>               :beta, :String,
>               :boss, :Person)
>
>But this leaves some unanswered questions. Here are the questions and
>my
>answers -- pardon me for personifying the database software:
>
>  1. "What table will I get this 'person' stuff from?" (Just derive the
>
>     table name from the class name.)
>  
>
What if they did not designate a class for the table, but let it default 
to Struct?

>  2. "What field in the child table will I use as a key?" (Hmm, maybe
>I'll
>     have to tell you this one. It might be cool to be able to
>designate a
>     key field, though.)
>  
>
See my comments on key fields above.

>  3. "What type is that key field?" (Given the name, you can find it in
>the
>     child table's information.)
>  
>
True, but what if they have not created the Person table yet?  How does 
KirbyBase find the type?  If I have KirbyBase wait till runtime to find 
the type from the child table, now I have to have KirbyBase open the 
child table everytime it wants to get the type for the boss field, like, 
for example, when it needs to check that the user is entering the proper 
field type during a #insert.

>  4. "What if Person itself is another complex object?" (Relax, just
>apply
>     the same algorithm recursively. Worst that can happen is there is
>     cyclic data, you'll go into an infinite loop, and I'll have to
>kill you.)
>  
>
Ok, so how did I solve this problem.  Well, I tried to find a solution 
that was as simple for the user as possible, but left enough room for 
further expansion and for the unforseen (see my comments on "key" fields 
above).  So, here is how you would specify the link between the :boss 
field and the :person table in the beta:

create_table(:mytablename,
 :alpha, :Integer,
:beta, :String,
:boss, { :DataType => :String, :Link => [:person, :person_id] })

This simply says, make a link between the value found in the boss field 
to the person_id field in the person table.  It specifies the field type 
for boss (:String) and it tells which field to link to (:person_id) 
within the :person table, therefore we don't have to specify a key field 
in the :person table.  It's a little more work, but, imo, not much, and 
you only have to do this once, when you create the table.  After that, 
KirbyBase handles everything for you automatically.

>Now, Jamey's first attempt at this had me writing code in the
>MYtablename#kb_create method, calling a method named one_to_one_link or
>some
>such. It felt very manual to me, like I was hotwiring my car.
>
>  
>
True.  What seemed like a good idea at the time, because I felt like it 
gave total control of the "guts" to the user, turned out to be too 
cumbersome for the user.  Hal finally got me to see this.

>"What's the big deal?" says the mechanic. "All you do is, you grab
>these two
>wires, not these two, and touch them together for a little while, not
>too long,
>and..." And I reply: "I don't want to reach for the wires and grab
>them, I
>don't want to memorize the colors, I don't want to estimate the time
>interval,
>and I don't want to see wires dangling. Keep that under the hood."
>
>In one iteration, perhaps not the present one, a one-to-many
>relationship was
>stored inside its parent object as a KBResultSet. Ugly to me. If we
>must have
>one-to-many, let it be just an array. I don't want my choice of
>database to
>intrude into my objects any more than necessary. Given the right glue,
>*any*
>database should be usable for my objects. KirbyBase is implementing
>this glue
>pretty well so far, INSIDE the db software where it belongs.
>
>In the one-to-one case (present iteration), the type information is
>specified
>as an array. This was my suggestion, and in general I like it. But it
>is too
>complex. We may not reach the ideal shown above, but let's strive
>toward it.
>  
>
Again, Hal may not have yet seen the examples in the beta of the new 
version yet.  If you take a look at the example directory in the beta 
distribution, I have tried to give a good example of one-to-one links 
(link_test directory), one-to-many links (link_many_test directory), 
calculated fields (calculated_test directory), etc.

Of course, Hal may look at those examples and still not like them.  I'm 
interested in hearing his feedback, and everyone else's feedback on the 
beta.  I've tried to strike a good balance between ease of use for the 
user and allowing for future functionality, but I'm sure when people 
look at the beta examples, they are going to have lots of great ideas 
for improvement.

>When I have Jamey's email in front of me, I'll tell you exactly what I
>mean.
>
>I was also disturbed a little to see that the class had to inherit from
>a KB
>class (I forget which one). It's hard to articulate why this bothers
>me. It's
>one more thing to remember, one more thing to do, and it's not a
>totally clean
>separation.
>  
>
I think I did away with this in the beta.  You are right, that wasn't a 
good idea.

>Also, a word or two about the kb_create method (called automatically
>when a row
>is retrieved, to turn it into an object). In a perfect world, we should
>
>"normally" not have to define one at all. I'm thinking of ways to make
>it
>usually unnecessary. (In the case of "calculated fields," this might be
>the
>very best place to put them, however.)
>  
>
Yep.  My implementation in the beta means you don't have to use the 
kb_create method to specify link fields, calculated fields, or any of 
the new functionality.  You can specify it all as arguments to the 
create_table method.

>Hope this helps clarify things a little.
>
>And I'm glad to see this discussion happening in public, just in case I
>give
>Jamey some really stupid advice. :) I'd hate to persuade him into a bad
>design.
>
>  
>
I'm glad this discussion went public also.  Prior to this, Hal was my 
only feedback person for the new features and I think I was badgering 
him to death.  :-)

Well, if anyone is still reading this far, thanks!

Jamey