/home/projects/ubik/rmi

Architecture

Overview

The Ubik RMI application programming interface can be separated into 3 conceptual layers:

  • The client API, which is the one end-developers are the most concerned with. It involves: server creation/export/binding, implementation of client-side and server-side interceptors, configuration of the different parameters of Ubik RMI's runtime environment.
  • The core: this layer is irrelevant to end-developers, but is interesting to people who wish to know more about Ubik RMI's internals, either for curiosity, to bring improvements, or to extend the core for their own needs. This layer encompasses the distributed garbage collection mechanism, the creation of remote references, the command protocol used to between clients and servers - which among other things, is the way through which remote method invocations are transmitted, etc.
  • The transport layer: this has been consciously decoupled from the core in order to allow remote method invocation over different "transport types", such as: raw socket (the default), HTTP, UDP, JMS and, who knows, Blue Tooth and the likes. The only transport type implemented as of now is raw socket - which is also the default one, as mentioned. This is designed in such a way as to isolate client and server endpoints from the core, so that the latter needs not concerning itself about the nitty-gritty details of the transport implementation it supercedes.

The Ubik RMI API

This section explains the layers enumerated in the previous section, in order to provide a clear picture of Ubik RMI's mechanics as a whole.

The Client API

The client API can be defined as "what the application developer deals with". This layer is pretty similar to its Java RMI counterpart - at least in appearance, especially with regards to the programming model.

The Hub

The single-entry point into the API is the Hub class, that allows to export objects as servers. This class is statically initialized upon its first invocation, where it creates all dependent objects that compose the Ubik RMI runtime. The Hub is a singleton, meaning that there is only a single instance of it per JVM (or, rather, per classloader).

Server vs Client

For the developer, implementing a distributed application involves creating clients and servers. Very often, these roles are intertwined: a server can be the client of another server. Within the Hub, this duality is modeled by two classes: the ServerRuntime and ClientRuntime classes (both in the org.sapia.ubik.rmi.server package). An instance of both is kept in the Hub singleton.

At a given Hub, the ServerRuntime receives incoming remote method calls and dispatches them to the appropriate remote object. In turn, if the Hub is also acting as a client, it is in charge of notifying the appropriate Hubs (which are its servers, in this case) when remote references are locally garbage-collected; the ClientRuntime within the Hub in fact deals with this aspect.

So, in short, the following is true:

  • A ClientRuntime at one Hub is indirectly the client of a ServerRuntime at another;
  • a Hub can be both server and client.

When an object is exported as a server through the Hub, a physical server is created, of which there is also a single instance per VM, per transport type(see the Transport section for more details). This server object implements the Server interface. The instance is kept as part of the ServerRuntime and is in charge of receiving the remote method calls and relaying them internally so that they can be dispatched to the appropriate remote objects. Once an object has been exported, all subsequent remote objects will receive their method calls through the same server.

The Core

The end-developer needs not concerning him/herself with the Ubik RMI core; yet, a superficial understanding of it can be useful.

Remote References

As was explained in the previous section, once the application exports an object, a server is internally started and thereafter processes all incoming requests. When an object that is an instance of java.rmi.Remote is returned as part of a remote method invocation, a stub is dynamically generated for it, which is sent (in place of the object itself) to the client. The remote object is kept internally in an object table; the stub encapsulates the object identifier under which the object is kept in that table - which is necessary to target remote method calls to the appropriate remote object.

For each client that performs a remote method invocation and receives a stub in return, the Ubik RMI runtime keeps a reference count, which is incremented each time a stub for a given remote object is returned. The reference count is tracked on a per-client basis, which is necessary to properly implement distributed gargbage collection (DGC).

Distributed Garbage Collection

When remote references are created on the server-side, they are locally registered with the server-side garbage collector (implemented as the ServerGC) that keeps the reference count for each remote object. As was mentioned, in place of the latter, a stub is sent to the client; upon the stub's arrival at the client, it registers itself with the client-side GC - modeled by the ClientGC.

Together, the ClientGC and ServerGC indirectly interact to implement the overall DGC algorithm. The latter works as follows:

  • When the client GC registers a stub, it in fact wraps it in a soft reference and keeps the latter in an internal table - associated with the corresponding object identifier. It checks at a predefined interval the thus created soft references, making sure that the wrapped references are not null; if so, its means that the client no longer has references on the stubs and must synchronize its state with the server - meaning that the reference counts at the latter have to be decremented - for each of the dereferenced objects.
  • At a predefined interval then, the client GC will notify the server GC about the object identifiers for which the corresponding reference count should be decremented. When receiving such a notification from the client, the server GC will decrement the total reference count for the remote object; it does so by substracting the count that is kept on a per-client basis from the total reference count. If the latter reaches zero, it means that no client has references on the remote object anymore. When this occurs, the remote object is removed from the object table - and eventually garbage collected.

As part of the DGC, client are responsible for signaling their presence at a regular interval to the server. The server GC keeps track of all "current" clients, and will unregister them from its internal table if it detects that a client has not "pinged" for a predefined amount of time - decrementing the total reference counts accordingly. This is necessary since eventually, if a client stops, the reference counts will be decremented appropriately on the server-side, and the server's memory will be freed. Not taking this precaution would inevitably introduce distributed memory leaks.

The Command Pattern at Work

The "logic" protocol between Ubik RMI clients and servers is implemented with the Command design pattern. Command objects are sent from clients to servers, and executed at the latter. Many commands have been implemented, including the one that carries remote method calls over the wire. Eventually, developers can benefit from this pattern by implementing their own commands, thus working one level "under" the usual one.

The Transport Layer

The transport layer as been cleanly separated from the Ubik RMI core in order to allow for plugging different transport implementations. The TransportManager (in the org.ubik.rmi.server.transport package) is the entry point into the transport layer. It allows for the registration of TransportProviders which handle the low-level communication issues. The default transport provider is implemented by the SocketTransportProvider class, which, as its name implies, handles communication over sockets. It is also the default transport provider of Ubik RMI.

As was mentioned previously, a Ubik RMI server implements the Server interface, which specifies methods for starting and stopping, and completely hides server internals from the core. A server's responsibility is restricted to receiving incoming requests (or, more precisely, commands), dispatching them internally so that they can be properly executed, and returning the result to the client. This behavior somewhat follows the principles set forth by the Acceptor-Connector pattern, which cleanly separates request reception from request handling; in our case, the Acceptor would be the server, and the connector the Ubik RMI command execution mechanism.

The transport layer also makes a clean distinction between server and client behavior; as was seen, for the server part, a specific interface was designed. For the client-side, another interface comes into play: the Connection interface. This interface models a client end-point, and specifies the behavior that allows to send and receive objects over the wire. When a Ubik RMI stub sends a remote invocation command to its server, it uses an instance of this interface to do so.

An instance of the TransportProvider interface is used by the Ubik RMI core to create a new server instance, and to acquire client connections. The core selects the proper provider to use based on the export(...) method that is called by the application; one of these methods allows to pass in the port on which the server should be started. Internally, this method will use the default provider - which creates socket servers. On the other hand, an object can be exported as a server throught the export method that takes a ServerAddress instance as a parameter. Internally, the method selects the transport provider based on the return value of the getTransportType() method on the address object; the method returns a logical type (as a character string) that maps to a registered transport provider.