/home/projects

Magnet

Overview

Do you hate writing start-up scripts for multiple platforms (.bat, .sh, etc)? Are you tired of writing a different script for all possible environments and all possible usages? Would you like to spare the user of having to pass multiple command-line arguments? Magnet brings a solution to the shell script nightmare: start-up parameters and logic are configured using XML. Magnet brings to the run-time what Ant brought to the build-time.

Features

Here are Magnet's features:

  • start-up parameters and logic are configured through XML;
  • allows to start multiple Java apps in the same VM;
  • profile-aware: a single start-up configuration can be used in a variety of contexts;
  • shell scripts no more required.

Need

The need for Magnet arises from exasperation: we were tired of writing shell scripts. And we were tired of writing variations of the same script for every environment our applications were running on. You have probably experienced this: you develop an application, and test it locally as you go along; then the application is deployed on some development environment so that other teams can use it, test drive their own applications against it; and then your app goes to the QA environment, and then to prod, etc.

At each stage, your shell scripts change, your bootstrap configuration changes - for example the database you are connecting to does not have the same URL in dev, QA or prod.

Or sometimes, it is rather the "usage mode" that varies: your application can be used in standalone mode, with everything it needs on the user's workstation; it can also be used in client-server mode, where only the client part sits on the user's workstation. There are certainly other "modes" you have come accross (swing vs web, etc). Take JBoss: you can start it up in DEFAULT, MINIMAL or STANDARD mode, with different runtime parameters for each mode.

We are guessing that your solution to the "multiple startup profiles" problem was to write different startup scripts, or to maintain multiple bootstrap config files, or both; at least, this WAS our solution.

And so Magnet was born.

Concepts

First, let's define a couple of concepts that we will refer to as we go along...

Profile

By profile, we mean runtime or startup profile: a given application might be started with different startup parameters, depending on its usage (see previous section for what we mean). For a given type of usage (or profile, from now on), these parameters may vary or remain the same. Let's take JBoss as an example: referring to what we mentioned in the previous section, JBoss has 3 profiles - according to our terminology: default, minimal, and standard. In default mode, JBoss requires a given set of parameters; in minimal mode, it requires only some of the parameters that the default mode needs - thus it is a subset of the default mode. From our experience (and we include you in this), profiles can extend, overlap, override each other; an application can have one to many profiles - as our JBoss example demonstrates.

Profile-Sensitive Parameters

So we have seen that a profile relates to the different runtime/startup properties an application may be ran under. In fact, a profile is just this: it is mainly a logical group of startup parameters. This means that startup parameters will vary according to the profile - which is specified by the user. In other words, the user specifies the profile under which the application should run, and then the startup parameters are "internally" chosen according to the passed in profile. Startup parameters thus become "profile-sensitive".

Architecture

Magnet was designed with the above-defined concepts in mind. In a few points, here is what Magnet is:

  • A single command-line script that starts whatever application you wish;
  • an XML format that pretty much looks like Ant's, except that it has been designed with run-time in mind (rather than build-time): classpaths can be defined dynamically with corresponding XML elements; the format supports variable interpolation (you know, the ${variable_name} thing?);
  • a multiple-application-per-VM scheme, with each application having its own specific classpath;
  • centrally defined startup parameters (rather than hard-coding them in shell scripts).

And here is what it does, or what we do with it:

  • startup parameters (and the application(s) to start) are defined using XML;
  • the user starts a magnet (an application or set of applications whose startup is configured using Magnet's XML format) with the built-in startup script that comes bundled with the Magnet distribution. This single startup script (magnet.bat or magnet.sh) is the only shell script you will ever need. The user passes a) the name of the Magnet startup file that is used to start the magnet application(s); b) the name of the profile under which the the magnet should be started.
From now on, we will use "start a magnet" instead of "start a magnet-configured application". It's way cooler. Plus the more conventional approach does not help suggest the notion of "multiple applications per VM" that Magnet supports.

Usage

Installation

To install Magnet, follow the steps below:

  • Download and install the Magnet distribution on your box, and extract it in a directory of your choice.
  • Make that directory the MAGNET_HOME (define the MAGNET_HOME environment variable as the path to that directory).
  • Add the MAGNET_HOME/bin directory to your PATH environment variable.

Learning By Example

Coding the App

Enough talk, let's get to the meat. Let's start with the most basic and universal application one can think of: the "Hello World" application. In fact, let's say that it's the "Hello XXXXXX" application; the "XXXXXX" will vary according to the profile under which the application is started. Our application is a basic Java app (a class with a main method):

  package org.sapia.magnet.example;
  
  public class HelloWorldApp{
  
    static final String DEFAULT_MSG = "Everyone";
    
    public static void main(String[] args){
      String msg;
      if(args.length == 0){
        msg = DEFAULT_MSG;
      }
      else{
        msg = args[0];
      }
      
      System.out.println("Hello " + msg);
    }
  }

Writing the Startup Script

Nothing to complicated thus far: the main method expects some token as an argument - it uses a default one if that token is not specified. The token is used to display a "Hello <some_token>" message.

Now, normally, what we would do is write a .sh/.bat script to start our childish application. That's a complication already: two startup scripts for a single app... With, Magnet, this operation is replaced by writing a magnet file. The following is our magnet's content:

        

<magnet xmlns="http://schemas.sapia-oss.org/magnet/core/"
  name="HelloMagnet" 
  description="A very simple Hello application">
        
  <classpath id="hello_cp">
    <path directory="${magnet.home}/examplelib">
      <include pattern="*.jar" />
    </path>
  </classpath>
        
  <launcher type="java" name="helloWorld" 
    mainClass="org.sapia.dynamo.magnet.examples.HelloWorldApp"
    args="${msg}" default="english" isDaemon="false">
            
      <profile name="english">
          <parameters>
              <param name="msg" value="world" />
          </parameters>
          <classpath parent="hello_cp"/>
      </profile>
      
      <profile name="spanish">
          <parameters>
              <param name="msg" value="el mundo" />
          </parameters>
          <classpath parent="hello_cp"/>      
      </profile>
      
      <profile name="french">
          <parameters>
              <param name="msg" value="le monde" />
          </parameters>
          <classpath parent="hello_cp"/>
      </profile>
      
  </launcher>
</magnet>

The script deserves some explanations. The following describes the above magnet, element by element - look at the /etc directory of the distribution for more information. More exhaustive documentation about Magnet's XML markup can be found in the Magnet Reference Manual

magnet

The root element in any magnet file is magnet. This element takes the following attributes

  • name: specifies the magnet's name. This is required for eventual display/information purposes.
  • description: specifies the magnet's description. Again required for display/information purposes.
  • extend: This optional attribute indicates from which magnet the current one inherits. This is an inclusion mechanism that allows a magnet file to inherit all the properties, applications, etc., of a given parent magnet. The attribute's value must be the full path to an existing magnet file. When such an inclusion is defined, the "parent" magnet is rendered before the current one.

classpath

The classpath element, as its name implies, allows to define a classpath. At runtime, a ClassLoader is instantiated for every specified classpath. The element takes nested elements that allow to include or exclude jar files into/from the classpath - in a manner identical to Ant's classpath task. A classpath element that appears directly under the root element must specify an id attribute that allows to refer to the classpath later on - see further below.

launcher

This element is where the bulk of our application's startup configuration is defined. The launcher element takes a "type" attribute. This type attribute specifies what type of launcher to use to start the application. The "type of launcher" concept is simple: we said to ourselves: why not be able to launch not only Java apps, but also executables? Or java applications that do not necessarily have a main method? Custom launchers can thus be used to start different types of application. For example, the "system" launcher is used to start a native process. In our case, we are using the "java" launcher: our application has a main method.

A magnet can have more than one launcher. This is why we are saying that with Magnet, multiple Java applications can share the same VM.

The launcher has a type attribute that must be set and identifies what type of launcher to use. The current supported types are "system" and "java". This attribute must be set first. Then, in the case of the "java" launcher, the following attributes are specified:

  • name: specifies "name" of the launcher. This is required for eventual display/debugging purposes.
  • mainClass: the class of the application to start - the "java" launcher expects the class to have a main() method.
  • args: this optional attribute specifies the arguments that must be passed to our application's main() method. If it is not specified, the method is called with an empty array of strings. As can be seen, the attribute in this case corresponds to a variable whose value is determined dynamically, at the magnet's rendering time. Look in the parameters element defined further below for a preview on how the value is specified...
  • default: the default profile to use if the current launcher has no profile element - see below - that matches the profile name that was specified when invoking magnet. This attribute is optional; if omitted, the application will not be started provided no corresponding profile element matches the profile name specified at magnet's command line.
  • daemon: if "true", indicates that our application's thread should be a daemon.

profile

Now, pay attention, this is an important part. We spent quite some time on the profile concept. This is where it is implemented in magnet: an application can have multiple profiles, and thus the "java" launcher can contain multiple profile elements. The profile element encapsulates runtime "characteristics", materialized by a classpath element - whose content has been explained above, and zero-to-many parameter element(s). The parameter element allows to define runtime parameters; a defined parameter's name can later be used for variable interpolation, through the ${param_name} notation. A given profile element will only be rendered if its name (specified through the name attribute) matches the "current" profile (i.e.: the profile that was specified at the magnet command-line).

Thus, in this case, we could pass the "english", "french", or "spanish" profile name, and our application would be started with the corresponding classpath and runtime parameters. In our example, parameters are only defined within profile elements. Yet, we could also have defined parameter elements right under the magnet file's root element. In such a case, we are talking about "global" parameters - they are visible from all launchers. As for the profile parameters, they are only visible from within there encapsulating launcher.

Note that we can use system properties for variable interpolation - thus we could refer to user.dir, java.home, etc.

At runtime, Magnet starts our application in its own thread; the thread is set a classloader whose classpath consists of the jars that are specified in the profile that matches the current profile name; the runtime parameters are also rendered on a per-profile basis, and made available to the current launcher. In the case of the "java" launcher, these parameters can be recuperated in its args attribute.

The profiles that we have defined have their classpaths "point" to the global classpath defined at the beginning of the file. Our application's classloader will thus have the global classpath's classloader as a parent. A global classpath - matching to a global classloader - is visible from all launchers. Thus, if starting multiple Java applications in the same VM, you can share libraries amongst these multiple applications. This is convenient if you want to implement "true" singletons, or if you want some in-VM communication between your applications without risking ClassCastExceptions.

Lastly, notice the default attribute defined on the launcher; it is given the name of an existing profile has a value. Imagine that we start our magnet with the "german" profile; nothing matches that in our configuration. In this case, Magnet will use the profile that is specified by the attribute - in this case, the "english" profile.

Starting the Magnet

Invoke the magnet command line: cd to Magnet's /etc directory and type:

magnet -magnetfile helloWorld.xml english

Note that the magnet executable provides command-line help:

magnet -h

Magnet allows you to pass in VM properties (-Dprop=name), -X options, etc.

Advanced Issues

Rendering and Startup

Magnet uses the Confix API to parse a magnet file and create an object representation of it. This dynamically generated object graph is then traversed; each object performs its "rendering"; it is passed a context that holds, among other things, the parameters that were previously declared through parameters elements. Rendering thus consists of variable interpolation and initialization.

If a given magnet inherits from another one, the latter is rendered first; the parent magnet's global parameters and classpaths are accessible from the child magnet, and so on.

Once a magnet has been properly rendered and initialized, it is "started": each launcher is started in its own thread, in the order in which it appears.

More on Parameters and Variable Interpolation

Magnet parameters can be declared at the "global" level, or (in the case of the "java" launcher), at the "profile" level. When declaring parameters at the global level, one can opt to "export" parameters to the VM's system properties. In such a case, the name/value pair of an exported parameter becomes the name/value pair of a system property, and is thus accessible VM-wide. The following shows how to export a parameter to the system properties:

<parameters>
  <param name="naming.port" 
         value="8200"
         scope="system" />
</parameters>

By default, all global parameters have "magnet" scope: they are visible from within the current magnet only (and from children magnets, if such is the case). The scope attribute of the param element takes either "system" or "magnet" as a value - we'll let you guess which corresponds to what...

Global parameters can also be defined on a per-profile basis. Take the following:

<parameters profile="dev">
  <param name="naming.port" 
         value="8200"
         scope="system" />
</parameters>

The above example demonstrates the use of the profile attribute. The latter's value must in this case correspond to the current profile for the encapsulated parameters to be rendered.

If a given parameter is declared twice, the second declaration overrides the first (this rule also applies to system properties: a magnet parameter that is exported to the system properties can potentially override an existing system property, so handle with care). Parameters that have "local" scope - defined at the profile level - will not be exported to the system properties; a local parameter can override (for a given launcher) a global parameter. More clearly, an override defined "locally" will only take effect locally; all other launchers will still "see" the global parameter in its original state.

As was explained in the Hello example, parameters are used to perform variable interpolation. The interpolation mechanism is tightly coupled to parameter scope; when Magnet tries to substitute a variable by its value, it will search for that value in the current scope, and then "up" if no value is found. For example, given the following snippet:

<launcher type="java" name="variableExample" 
    mainClass="org.sapia.dynamo.magnet.examples.DummyApp"
    args="-logDir ${user.dir}/log" />

The value of the user.dir variable will be searched first in the local scope (in the parameters defined within the current launcher); if no value is found, then the global parameters are searched; if the search remains unsuccessful, the system properties are searched. Indeed, user.dir maps to a system property...

Scripting

Sometimes, it is necessary to perform some operations prior to the application(s) being started: create some directories under the app root ("log", "db", "tmp", etc.), conditionnally set some system properties, etc. In such a case, Magnet offers scripting support, through the script element. The following is a typical script:

<script type="bsh" isAbortingOnError="true">
  File aFile = new File(System.getProperty("user.dir"),
         String.valueOf(System.currentTimeMillis()));
  System.out.println("Creating temp directory " + 
         aFile.getName());
  aFile.mkdirs();
</script>

The script element takes the text of a script and passes it to an interpreter. Currently, Magnet supports Beanshell. Beanshell is a Java interpreter that can be used in a magnet by specifying the "bsh" value for the type attribute on the script element. The isAbortingOnError attribute indicates if the current magnet's execution should be stopped provided the script generates an error.

The script element goes under the root element. Magnet will execute the script - if any - as the first step in the rendering proceess.

Starting System Processes

So far we talked about Magnet as being a tool to start Java programs, but this is not entirely true. The fact is that we created Magnet having in mind that the XML configuration could be used to start any type of process. If you look again at the previous "Hello XXXXXX" you can see that the launcher element has a type attribute with the "java" value; in addition, Magnet provides the "system" laucher type, that allows one to execute native system commands. So for example if the Java application you wish to start in your Magnet is dependent on a MySQL database, you could first execute a system command to start the database and then start your Java application. Another good example is when you start a web application with Magnet; you may wish to automatically open a web browser to the URL of your application. Hey, guess what, the following Magnet script indeed opens internet Explorer on a given URL:

        

<magnet xmlns="http://schemas.sapia-oss.org/magnet/core/"
  name="SystemMagnet" description="Test system launcher">

  <parameters>
    <param name="SystemRoot" value="C:\Windows" />
    <param name="iexplorePath"
      value="&quot;C:\Program Files\Internet Explorer&quot;" />
  </parameters>

  <environment id="windows_env">
    <variable name="ComSpec"
      value="${SystemRoot}\system32\cmd.exe" />
    <variable name="SystemRoot"
      value="${SystemRoot}" />
    <variable name="windir"
      value="${SystemRoot}" />
  </environment>

  <launcher type="system" name="StartBrowser" default="yahoo"
            command="cmd /C start /B iexplore.exe ${url}"
            workingDirectory="${user.home}">
    <profile name="yahoo">
      <parameters>
        <param name="url" value="http://www.yahoo.com" />
      </parameters>
      <environment parent="root">
        <variable name="path"
          value="${SystemRoot}\system32;${iexplorePath}" />
      </environment>
    </profile>
    <profile name="google">
      <parameters>
        <param name="url" value="http://www.google.com" />
      </parameters>
      <environment parent="root">
        <variable name="path"
          value="${SystemRoot}\system32;${iexplorePath}" />
      </environment>
    </profile>
  </launcher>
</magnet>

The previous example shows how we can start Internet Explorer on a Windows machine using a system launcher. This magnet would start the browser with the Yahoo or Google start page - depending on the passed in profile. Let's examine this example to understand how it works: you can see that the child elements of this launcher sightly differ from the ones of the "java" launcher. The environment element is used to define environment variables that will be passed to the system process. Because the information required to start a Java program differs from the one required to start a system process, the attributes of launcher element are different: the command attribute replaces the mainClass used in the Java launcher and a workingDirectory attribute is added.

See the /etc directory of the distribution for more examples.

Conclusion

As you can see, Magnet can free you from the start-up script nightmare. It also allows you to start multiple applications per VM, and supports the "profile" concept in order to greatly simplify bootstrap configuration. To learn more, well, download and try! You will find examples in the /etc directory of Magnet's distribution. Have fun!