From: matthewdspencer@... (Matthew D. Spencer) Date: 2001-05-02T16:11:24+09:00 Subject: [ruby-talk:14537] Rajah: Ruby-Java integration Hi. Our names are Dan Munk and Matthew Spencer. We have noticed the recent threads of discussion regarding Ruby/Java integration, and decided that it was time for us to announce a project we have been quietly researching and developing for a while now. We would like to introduce Rajah, a bridge between the Ruby and Java languages. Below you will find an outline of the development we have done to date. Please excuse us if the doc below is a little rough. We want to introduce Rajah to developers in a detailed way, but bear in mind that this document is also a submission to the Ruby community outlining a work in progress. Before we get into the discussion of Rajah itself, we should mention that we would be very interested in any ideas or feedback regarding this document and Rajah itself. We are very open minded about the future of this project, and we would like all of the input we can get. We are both relatively new to Ruby, and we are most concerned about making Rajah a *complete* Ruby/Java solution, not simply an extension that is functionally sufficient. We look forward to hearing from you! Dan Munk danmunk@hotmail.com Matthew Spencer matthewdspencer@yahoo.com ================================================================================ 1. Introduction 2. Roadmap 3. Design 4. Known Issues 1. Introduction -------------------------------------------------------------------------------- Rajah is an open source project that provides a bridge between the Ruby and Java languages. Within the Rajah project there are three planned Milestones: + From Ruby to Java. The majority of this work is complete, but as we are fairly new to Ruby we are worried about correctness issues. Thus this post. :) A later iteration of this milestone would be Ruby extensions written in Java. This piece has not started, but we would like to mimic the existing C extension architecture as much as possible. + The second milestone is from the JVM to Ruby. The general approach and architecture has been considered, but no real implementation work has been done. + The third milestone would be a pure Java implementation of the Ruby interpreter. The rest of this document is focused on design and implementation details of the first milestone, from Ruby to Java. Any ideas/feedback related to any portion of this doc would be greatly appreciated. 2. Roadmap -------------------------------------------------------------------------------- Once we implement the changes from the feedback that we are hoping to receive from this post we are planning to release an alpha build on May 14 to the Ruby application archive. It is possible that this date will slip a week or two depending on the number and scope of changes. At this time CVS on Sourceforge will be used for versioning and snapshots will be continuously available. We are members of the Extreme Programming camp (thank you for the RubyUnit effort) and internally we have created several iterations for each milestone. The iterations for the first milestone will be updated and published on Sourceforge once they are adjusted to include tasks from your input. Following milestones will not be started until the previous milestone(s) have been met and are of production quality. We can test on Solaris(Sparc 7, 8), some Linux flavors, and Windows 98/NT/2000. Help testing with additional platforms would be greatly appreciated. The Rajah code is designed to run under any Java environment 1.1+. However, we will only be testing with 1.2 and later, and for the first iterations (at least) will not claim to support any Java environments prior to 1.2. 3. Design -------------------------------------------------------------------------------- One of the major factors influencing all design decisions is that we wanted to keep the Java side of the interface as transparent as possible. Code to open a JFrame would be: require 'Rajah' frmCls = Java.forName('javax.swing.JFrame') frame = frmCls.new("Example") frm.pack frm.show The remainder of this section outlines the design of Rajah's Ruby to Java interface. The Rajah library introduces a module named "Java". This module encapsulates the Java functionality, including methods for loading Java classes. Once a Java class is loaded into Ruby using Rajah, the class can be treated as a Ruby class, including features such as method invocation, reflection, sub-classing, and just about everything else you can do with a Ruby class. Loading Java classes The most fundamental capability of Rajah is loading Java classes into memory, and allowing them to be accessed as Ruby classes from within the Ruby language. This process involves the following primary steps: + Find the Java class. + Load the super class of the Java object. + Register the class with Ruby. + Introspect the Java class for all of it's public constructors. + Introspect the Java class for all of it's public methods (both static and instance methods). + Introspect the Java class for all of it's public fields (both static and instance fields). Let us look at these steps in detail. + Find the Java class In order to load a class into Ruby, the embedded virtual machine must be able to find the Java class. In order to correctly find the class, the fully qualified class name must be provided (e.g., "java.lang.String"). The classpath is then searched for the class. If the class is not found, an exception will be thrown. + Load the super class of the Java object In order to properly load a Java class into Ruby, it's super class must first be loaded. An attempt will be made to load the super class of the Java object (this is a recursive process, stopping when the top of the class hierarchy is reached). + Register the class with Ruby Registering a Java class with Ruby involves creating a new Ruby class that directly references the Java class. If the Java object has no super class (and thus is the "java.lang.Object" object), the new Ruby class is created with a super class of object Java::JavaObject. Otherwise, the new Ruby class is created with a super class of the Ruby representation of the Java super class (which must have already been registered with Ruby since super classes must be loaded before registration of a Java class can take place). Note that when the class is initially registered with Ruby, none of the Java constructors, methods, or fields are added to Ruby immediately. Introspection of the Java class does not occur until after the class is registered with Ruby (but before the class is finished being loaded by Rajah). + Introspect the Java class for all of it's public constructors The Java class is then introspected for all of the publicly accessible constructors. These constructors are added to the in-memory representation of the Java class. When the first constructor is added to the class, an "initialize" method with a variable number of parameters is added to the class. Every constructor for every class loaded via Rajah shares the same C function callback for handling the "initialize" method. See the section on object creation for more information. + Introspect the Java class for all of it's public methods The Java class is then introspected for all of the publicly accessible methods. These methods are added to the in-memory representation of the Java class. Every method of every class loaded via Rajah shares the same C function callback for handling the method. See the section on methods for more information. + Introspect the Java class for all of it's public fields The Java class is then introspected for all of the publicly accessible fields. Object creation Objects are created in Ruby using a class' "new" singleton method. When Java objects are registered with Ruby, a handler for the singleton method "new" is registered. When the new method is invoked, a new Java object of the Java class is instantiated. However, a constructor is not immediately invoked. Using JNI allows Rajah to instantiate a Java object without any constructor being invoked, thus we postpone construction of the object till later. After the new Java object is instantiated, it is wrapped in a Ruby object. Then, the initialize method is invoked with the same parameters the new method was invoked with. See the next section for more information on method invocation. The initialize method will invoke the appropriate constructor on the Java object created in the new method. If no matching constructor is found using the given Ruby arguments, an exception will be generated. Methods Before delving into the details of how Java methods are handled in Ruby, let us examine a few of the requirements for Java methods: + Support for static and instance methods of Java classes. + Support for overloaded Java methods. + Support for a lightweight invocation interface (i.e., Java method invocation should impose as little overhead as possible). All Java methods registered with Ruby use a variable length argument list to process the method. This serves a few purposes: 1. Allows every instance method of every class to use the same C function to handle the method. This minimizes code complexity, and keeps the C API simple. Likewise, every static method of every class can use another C function to handle the method. 2. Allows support for overloaded Java methods. 3. Since parameters need to wrapped into an array internally eventually anyway, the performance of this is negligible. Method invocation process When a method is invoked, the following steps occur: + The name of the method that was invoked is retrieved. + The Java class that the method was invoked on is searched for a method that matches the name, the argument count, static or non-static criteria, and the argument types. If no method is found that matches all criteria, an exception is raised. + The Ruby arguments are marshaled (see below) into corresponding Java parameters. + The Java method is invoked via JNI. If any Java exceptions occur, a corresponding exception will be generated and raised in Ruby. + The return value of the method invocation is then unmarshaled into a Ruby object and returned from the method invocation. Rules for mapping Ruby types to Java types When a Java method or constructor is invoked using Ruby arguments, the arguments must be converted into appropriate Java primitives or objects. This process we refer to as marshaling. Likewise, the return value of a method (if the method is not declared to return void) must be converted into the appropriate Ruby object. This process we refer to as unmarshaling. Java provides two fundamentally different entities that need to be examined for the marshaling and unmarshaling processes: primitives and objects. There are eight primitive types in the Java language: + boolean (values values are the Java keywords "true" and "false") + byte (signed 8-bit integer) + char (16 bit Unicode character) + short (signed 16-bit integer) + int (signed 32-bit integer) + long (signed 64-bit integer) + float (signed 32-bit floating point number) + double (signed 64-bit floating point number) These primitives are handled by Rajah to correctly map Ruby values to Java, as well as map Java values back to Ruby. In Java, every object (either directly or indirectly) extends the Java class "java.lang.Object". The single exception to this rule is the "java.lang.Object" class itself, which has no super class. Handling of primitives and objects should be fairly transparent from within Ruby. However, when overloaded methods and constructors surface, some familiarity with Rajah's mapping marshaling and unmarshaling rules is necessary. Marshaling Marshaling of each individual parameter occurs sequentially, starting with the first parameter. (Note that since the method is matched before any real marshaling occurs, errors during the marshaling process should be infrequent.) The Java type of the parameter is examined, and the Ruby value is marshaled into an appropriate Java type. Here is a list of the marshaling rules when the given Java type is encountered: + boolean - Ruby "true" will map to a Java "true". Ruby "false" will map to a Java "false". + byte - Fixnums within the range -128 to 127. + char - ??? + short - Fixnums within the range -32768 to 32767. + int - Fixnums or Bignums in the range -2^31 to (2^31 - 1). + long - Fixnums of Bignums in the range -2^63 to (2^63 - 1). + float - Floating poit values in the appropriate range. + double - Floating poit values in the appropriate range. + java.lang.String object - Ruby string objects will be marshaled to java.lang.String objets. Of course, java.lang.String objects are also acceptable. + arrays - Ruby arrays will be marshaled into Java arrays. However, this places restrictions on what the values in the Ruby array can be so that marshaling can properly occur for every element in the array. Unmarshaling Unmarshaling values returned from Java (either via return values from invoked methods or from field access) will map a Java object to a Ruby object. If the return type is a Java class that has not been loaded into Ruby yet, Rajah will go through the class loading process for the return value as outlined above. This section will be explained in more detail in the Rajah documentation. 4. Known Issues (in no particular order) -------------------------------------------------------------------------------- - Support for arrays is not complete. - There is currently a conflict between Java methods with the name "initialize" and the predefined "initialize" defined for each Java class when it is loaded into Ruby. Currently, no special checks are done when a Java method with the name "initialize" is loaded. The results are undefined if an attempt is made to load a Java class with a Java defined initialize method. - Support for exceptions from Java is not complete. - In Java, throwable types (java.lang.Throwable objects) extend java.lang.Object. In Ruby, the convention is to create exceptions that extend the class Exception. It would probably be nice if we made all of the Java Throwable objects somehow extend the Ruby Exception class. However, technically the Throwable object should extend the java.lang.Object class. Is it "bad" Ruby style to throw objects that do not extend Exception? - Support for Ruby objects implementing Java interfaces is being examined, but not currently implemented (no pun intended). JDK 1.3 might alleviate parts of this problem, as JDK 1.3 introduces runtime interfaces (Proxies) for objects. However, even this approach would probably require helper classes written in Java. - What about abstract classes? Abstract methods? Behavior needs to be defined when abstract classes are loaded. - What about Java interfaces? - When no Java constructor is supplied in the class, does introspecting the class still give us a no-parameter public constructor? If not, should we supply one ourselves? - How are methods that begin with capital letters or _'s handled in Ruby? - How are classes that begin with lowercase letters handled from Ruby? - If a method is invoked that is not found in the Java class, should we attempt to invoke the method directly on the super class, or is that even necessary? - Currently only public methods/fields are loaded. Do we want to load protected methods/fields too, in case a ruby class is sub-classing one of the java classes and wants access to protected fields? This might not even be possible with the reflection API, but it is something we should look in to. - Subclassing Java classes loaded into Ruby is not supported right now. - We should make the Java method "String toString()" map to "to_s" in Ruby.