From: David Masover Date: 2008-10-01T15:00:53+09:00 Subject: Re: How to make Ruby _THE_ scripting language of choice, fold inSQLite On Sunday 28 September 2008 17:30:02 John Carter wrote: > Big Money brooks no dead ends. SQL is a huge, very useful, successful > money spinner. Hence it will be and _is_ being extended both on the > client _and_ server side to do what scripting languages like Ruby does. > > No sane language designer would choose such a path, the SQL extended > languages I have seen are as ugly as sin... but as I say, nothing > stops the money. In other words: Much like COBOL. COBOL is, in fact, very successful. There's a huge amount of old COBOL code lying around, and there's quite a lot of high-paying jobs for maintaining it. Because of the sheer size of these codebases, it's been extended to do all kinds of things. But it's still COBOL, and still ugly as sin. > So tell me, assuming you, like millions of developers worldwide, have > to use some form of SQL database. To continue the analogy, suppose I had to code in COBOL. What are the reasons I would have to use COBOL? Most likely, it's because I'm supporting a legacy app. When was the last time you had to maintain a legacy Ruby app? Generally, if I'm being paid to write Ruby, it's because I'm developing a relatively new app, and with a corporation that has the presence of mind to choose the best tool for the job -- not the most popular tool for the job. So, if I'm being paid to write a brand-new Ruby app, don't you think there'd be discussion of whether or not to use a SQL database? Of whether a document-based database might be a better choice? In my discussion with Clifford Heath on this thread, I learned that there are applications which are suitable to a traditional transactional database (probably SQL), and not to a document-based database. I would guess that such applications are still the vast minority. > Is the integration between Ruby and SQL as slick as you could imagine? Define "slick". > The current adapters are damn Good, but not as slick as I could imagine. > > Neither in terms of syntax, Object / Relational integration, exception > handling nor performance. Extraordinary claims require extraordinary proof. > So let's talk a bit about what SQL gives, that I'd like to see cleanly > and comfortably and slickly integrated in ruby... > > * Relational Data model. Yup. I'm aware of the perfectly valid > criticisms that SQL doesn't go anywhere near close enough to Codd's > Laws... but it's one of the closest practical things that does. And > yes, I'd love it if Ruby had a true relational algebra embedded within it. Google isn't giving me a lot of help here. What's this about, exactly? What would it look like in Ruby? > So ORM / Db > adapters try mash these notions together, but the semantics aren't > quite the same. Which differences in semantics are bothering you? > I bet we could use Ruby Mixins to designate and extend certain > classes as tables. Or maybe inheritance. I know! We can make all classes that want to be tables inherit from a common base class! I think I'll call it ActiveRecord::Base. > I bet we could find a really Good Ruby way of defining the Data > Model that would be better than "create table". Indeed some of the > ORM libraries mention are excellent examples of this. In other words, an ORM library could do this -- nothing needs to be changed in Ruby itself. > * Data Persistence. Yip, marshal and restore gives us that. Marshal and restore aren't generally used for persistence. Generally, the ORM in question provides its own semantics for that -- certain properties of it correspond directly to columns, and telling the object to persist updates the row with those values. But, suppose we're talking about something like CouchDB, or RDDB: > - Indexes / updates and deletes? Done, done, and done. Calling "save" on any model will update the record with the values which have changed, or insert a new record if one doesn't exist. Calling "destroy" will delete it. > - Concurrent access? This is a more fundamental problem, and it's not unique to Ruby. But there is native (and easy) support for optimistic and pessimistic locking. > - Even if an application can persist it's object model it's > incredibly useful to query, and access that same data model from a > query tool, or (perhaps indirectly via view or a projection) by > another (loosely related) application. Or within the same. But yes, that's kind of inherent in the fact that a SQL database is used here: > * Data Persistence. Yip, marshal and restore gives us that. But is > that enough? Suppose we split it into one record per file. I'm not advocating this -- I would much rather > - Indexes / updates and deletes? Indexes: Yes, maintained separately. Updates: Unmarshal the record you need, change it, then re-marshal and save. Deletes: Delete > - Concurrent access? Can be done on a filesystem. Maildir gives a very simple example of how this might be done. For that matter, concurrent access in most databases cannot be done more finely than by table or by row. A system like flock should be more than adequate for this purpose. > - Even if an application can persist it's object model it's > incredibly useful to query, and access that same data model from a > query tool, or (perhaps indirectly via view or a projection) by > another (loosely related) application. Define "loosely related" -- why not simply load up the model in question into this other application? > * Data Model / Database Management. Whoops. Adding / deleting / > renaming columns to a data table created via marshal / restore is > painful. Not really. Maybe if you're marshalling/restoring the objects directly... But consider: Adding a "column" is trivial; you need only provide a default value, and adjust the model class to know about this. You don't actually have to change any data at all, until people start setting values for that "column". Existing document-based databases work exactly this way. Deleting a "column" could be done the same way -- let the model class know it should be deleted. Renaming would be the most difficult. Here, you'd have to version your data, and give the model class a procedure to run when loading records older than a certain version. > * ACID - Well, actually that would be kind of handy in a number of > other areas not traditionally "Database". Not all apps need or want ACID. In fact, it's much easier to scale most apps if you don't make them ACID-compliant. > * Query Language - selects, joins, group by's, sorts. Rich and > expressive stuff indeed. And confusing, archaic, and fundamentally different than how we program everything else. For that matter, we do have this, somewhat -- we have Enumerable#select and Enumerable#sort. > * Client / Server / Concurrency - Actually Ruby is very Good at that > already, except there is no support for concurrency at the marshal > and restore level. No, it's done by serializing with something else -- generally, an ORM like ActiveRecord. But as I've shown above, you can, in fact, have concurrency with marshal and restore. > Does this involve a multiyear rewrite of the core of Ruby? > > Nah. > > It involves folding in preexisting public domain code. Sqlite. If it is so trivial, fork Ruby and do it yourself. Show us that it can be done, and that it is useful, and that it's worth polluting Ruby's core namespace with SQL junk. And don't be surprised if, when you're done, nobody's interested, or nobody's willing to take the risk of breaking old code. After all, Ruby 1.9 barely changes syntax at all, and look how long it's taken -- it's more than just a technical problem. It's a design problem. > but folding in the appropriate parts of (select,update,delete,insert) of Sqlite's parse.y Ruby is already a powerful and expressive language for creating new domain specific languages. The Sequel library already provides a Ruby DSL for SQL queries. There are many domains which might benefit from customizing Ruby's parser. I imagine that Rake wouldn't mind being able to do this: foo.o: foo.c system 'gcc foo.c -o foo.o' Not the best Make example, but you get the idea. Instead, they've chosen to leave it as Ruby, which means it looks like this: file 'foo.c' => 'foo.o' do system 'gcc foo.c -o foo.o' end Much cleaner, and actually valid Ruby. And you still haven't addressed perhaps the most disturbing part of your proposal: Even if we agree with you that SQL is so important that it needs to be part of the language, you're essentially dictating one true SQL engine and flavor of SQL. Why should Ruby embed SQLite, when almost no Ruby programs use SQLite in production? What does this mean for users of other databases? And what does this mean for those who don't want to use SQL at all? When I call Array#select, will it do what I expect? Oh, and for what it's worth: There's a language called Judo, which includes SQL. I'm curious what you think of it: http://judoscript.org/