Magnet

Overview

Magnet is a tool that spares writing platform-specific Java application startup files (.sh, .bat, ...) by supporting a neutral XML format describing application runtime parameters and bootstrap configuration. To top it off, Magnet also supports launching multiple standalone Java applications in the same JVM, each with its own classloader that operates in isolation from the others. Magnet's goal is to provide for the run-time what Ant provides for the build time.

Features

At a glance, here are Magnet's features:

  • start-up parameters and logic are configured through XML;
  • allows starting multiple Java apps in the same JVM;
  • profile-aware: a single start-up configuration can be used in a variety of contexts;
  • shell scripts no more required;
  • the flexibility of the design gives the ability to launch any type of process

Need

The need for Magnet arises from exasperation: we were tired of writing shell scripts. And we were tired of writing variations of the same script for every environment our applications were running on. You have probably experienced this: you develop an application, and test it locally as you go along; then the application is deployed on some development environment so that other teams can use it, test drive their own applications against it; and then your app goes to the QA environment, and then to prod, etc.

At each stage, your shell scripts change, your bootstrap configuration changes - for example the database you are connecting to does not have the same URL in dev, QA or prod.

Or sometimes, it is rather the "usage mode" that varies: your application can be used in standalone mode, with everything it needs on the user's workstation; it can also be used in client-server mode, where only the client part sits on the user's workstation. There are certainly other "modes" you have come accross (swing vs web, etc). Take JBoss: you can start it up in DEFAULT, MINIMAL or STANDARD mode, with different runtime parameters for each mode.

We are guessing that your solution to the "multiple startup profiles" problem was to write different startup scripts, or to maintain multiple bootstrap config files, or both; at least, this WAS our solution.

Thus, Magnet.

Concepts

First, let's define a couple of concepts that we will refer to as we go along...

Profile

By profile, we mean runtime or startup profile: a given application might be started with different startup parameters, depending on its usage (see previous section for what we mean). For a given type of usage (or profile, from now on), these parameters may vary or remain the same. Let's take JBoss as an example: referring to what we mentioned in the previous section, JBoss has 3 profiles - according to our terminology: default, minimal, and standard. In default mode, JBoss requires a given set of parameters; in minimal mode, it requires only some of the parameters that the default mode needs - thus it is a subset of the default mode. From our experience (and we include you in this), profiles can extend, overlap, override each other; an application can have one to many profiles - as our JBoss example demonstrates.

Profile-Sensitive Parameters

So we have seen that a profile relates to the different runtime/startup properties an application may be ran under. In fact, a profile is just this: it is mainly a logical group of startup parameters. This means that startup parameters will vary according to the profile - which is specified by the user. In other words, the user specifies the profile under which the application should run, and then the startup parameters are "internally" chosen according to the passed in profile. Startup parameters thus become "profile-sensitive".

Architecture

Magnet was designed with the above-defined concepts in mind. In a few points, here is what Magnet is:

  • A single command-line script that starts whatever application you wish;
  • an XML format that pretty much looks like Ant's, except that it has been designed with run-time in mind (rather than build-time): classpaths can be defined dynamically with corresponding XML elements; the format supports variable interpolation (you know, the ${variable_name} thing?);
  • a multiple-application-per-VM scheme, with each application having its own specific classpath;
  • centrally defined startup parameters (rather than hard-coding them in shell scripts).

And here is what it does, or what we do with it:

  • startup parameters (and the application(s) to start) are defined using XML;
  • the user starts a magnet (an application or set of applications whose startup is configured using Magnet's XML format) with the built-in startup script that comes bundled with the Magnet distribution. This single startup script (magnet.bat or magnet.sh) is the only shell script you will ever need. The user passes a) the name of the Magnet startup file that is used to start the magnet application(s); b) the name of the profile under which the the magnet should be started.

Usage

This section describes Magnet's usage in a summarized form.

Installation

To install Magnet, follow the steps below:

  • Download and install the Magnet distribution on your box, and extract it in a directory of your choice.
  • Make that directory the MAGNET_HOME (define the MAGNET_HOME environment variable as the path to that directory).
  • Add the MAGNET_HOME/bin directory to your PATH environment variable.

Learning By Example

Coding the App

Enough talk, let's get to the meat. Let's start with the most basic and universal application one can think of: the "Hello World" application. In fact, let's say that it's the "Hello XXXXXX" application; the "XXXXXX" will vary according to the profile under which the application is started. Our application is a basic Java app (a class with a main method):

package org.sapia.magnet.example;
  
  public class HelloWorldApp{
  
    static final String DEFAULT_MSG = "Everyone";
    
    public static void main(String[] args){
      String msg;
      if(args.length == 0){
        msg = DEFAULT_MSG;
      }
      else{
        msg = args[0];
      }
      
      System.out.println("Hello " + msg);
    }
  }

Writing the Startup Script

Nothing too complicated thus far: the main method expects some token as an argument - it uses a default one if that token is not specified. The token is used to display a "Hello <some_token>" message.

Now, normally, what we would do is write a .sh/.bat script to start our childish application. That's a complication already: two startup scripts for a single app... With, Magnet, this operation is replaced by writing a magnet file. The following is our Magnet configuration content:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE magnet PUBLIC "-//SAPIA OSS//Magnet DTD 2.0//EN"
                 "http://www.sapia-oss.org/dtd/magnet-2.0.dtd"> 

<magnet xmlns:magnet="http://schemas.sapia-oss.org/magnet/"
        name="HelloMagnet" 
        description="A very simple Hello application">
        
  <classpath id="hello_cp">
      <path directory="${magnet.home}/examplelib">
          <include pattern="*.jar" />
      </path>
  </classpath>
        
  <launcher type="java" name="helloWorld" 
            mainClass="org.sapia.magnet.examples.HelloWorldApp"
            args="${msg}" default="english" isDaemon="false">
            
      <profile name="english">
          <parameters>
              <param name="msg" value="world" />
          </parameters>
          <classpath parent="hello_cp"/>
      </profile>
      
      <profile name="spanish">
          <parameters>
              <param name="msg" value="el mundo" />
          </parameters>
          <classpath parent="hello_cp"/>      
      </profile>
      
      <profile name="french">
          <parameters>
              <param name="msg" value="le monde" />
          </parameters>
          <classpath parent="hello_cp"/>
      </profile>
      
  </launcher>
</magnet>

The script deserves some explanations. The following describes the above configuration, element by element - look at the /etc directory of the distribution for more information. More exhaustive documentation about Magnet's XML markup can be found in the Magnet Reference

magnet

The root element in any magnet file is magnet. This element takes the following attributes

  • name: specifies the Magnet configuration name. This is required for eventual display/information purposes.
  • description: specifies the configuration's description. Again required for display/information purposes.
  • extend: This optional attribute indicates from which Magnet configuration the current one inherits from. This is an inclusion mechanism that allows a Magnet file to inherit all the properties, applications, etc., of a given parent one. The attribute's value must be the full path to an existing Magnet file. When such an inclusion is defined, the "parent" configuration is rendered before the current one.

classpath

The classpath element, as its name implies, allows to define a classpath. At runtime, a classloader is instantiated for every specified classpath. The element takes nested elements that allow to include or exclude jar files into/from the classpath - in a manner identical to Ant's classpath task. A classpath element that appears directly under the root element must specify an id attribute that allows to refer to the classpath later on - see further below.

launcher

This element is where the bulk of our application's startup configuration is defined. The launcher element takes a type attribute. This attribute specifies what type of launcher to use to start the application. The "type of launcher" concept relates to being able to launch not only Java apps, but also other types of executables - or Java applications that do not necessarily have a main method. Custom launchers can thus be used to start different types of applications. For example, the "system" launcher is used to start a native process. In our case, we are using the "java" launcher: our application has a main() method.

A Magnet config can have more than one launcher. This is why we are saying that with Magnet, multiple Java applications can share the same JVM.

In the case of the "java" launcher, the following attributes are also required:

  • name: specifies "name" of the launcher. This is required for eventual display/debugging purposes.
  • mainClass: the class of the application to start - the "java" launcher expects the class to have a main() method.
  • args: this optional attribute specifies the arguments that must be passed to our application's main() method. If it is not specified, the method is called with an empty array of strings. As can be seen, the attribute in this case corresponds to a variable whose value is determined dynamically, at configuration rendering time. Look in the parameters element defined further below for a preview on how the value is specified...
  • default: the default profile to use if the current launcher has no profile element - see below - that matches the profile name that was specified when invoking Magnet. This attribute is optional; if omitted, the application will not be started provided no corresponding profile element matches the profile name specified at Magnet's command line.
  • daemon: if "true", indicates that our application's thread should be started as a daemon.

profile

Now, pay attention, this is an important part. We spent quite some time on the profile concept. An application can have multiple profiles, and thus the "java" launcher can contain multiple profile elements. The profile element encapsulates runtime "characteristics", materialized by a classpath element - whose content has been explained above, and zero-to-many parameter element(s). The parameter element allows to define runtime parameters; a defined parameter's name can later be used for variable interpolation, through the ${param_name} notation. A given profile element will only be rendered if its name (specified through the name attribute) matches the execution profile (i.e.: the profile that was specified at the Magnet command-line).

Thus, in this case, we could pass the "english", "french", or "spanish" profile name, and our application would be started with the corresponding classpath and runtime parameters. In our example, parameters are only defined within profile elements. Yet, we could also have defined parameter elements right under the Magnet file's root element. In such a case, we are talking about "global" parameters - they are visible from all launchers. As for the profile parameters, they are only visible from within their encapsulating launcher.

Note that we can use system properties and environment variables for variable interpolation - thus we could refer to user.dir, java.home, JAVA_HOME, etc. Magnet's parameters are first looked up when peforming interpolation, then a fallback is done on the JVM's system properties, and environment variables are searched as a last resort.

At runtime, Magnet starts our application in its own thread; the thread is assigned classloader whose classpath corresponds to the current execution profile - as explained above; the runtime parameters are also rendered on a per-profile basis, and made available to the current launcher. In the case of the "java" launcher, these parameters can be recuperated in the args attribute.

The profiles that we have defined have their classpaths "point" to the global classpath defined at the beginning of the file. Our application's classloader will thus have the global classpath's classloader as a parent. A global classpath - matching to a global classloader - is visible from all launchers. Thus, when starting multiple Java applications in the same JVM, you can share libraries among them. This is convenient if you want to implement "true" singletons, or if you want some in-JVM communication between your applications without risking ClassCastExceptions.

Lastly, notice the default attribute defined on the launcher; it is given the name of an existing profile has a value. Imagine that we start our Magnet config with the "german" profile; nothing matches that in our configuration. In this case, Magnet will use the profile that is specified by the attribute - i.e.: the "english" profile.

Starting the Magnet

Invoke the magnet command line: cd to Magnet's /bin directory and type:

magnet -magnetfile helloWorld.xml -profile english

this alternate command line does the same:
magnet -f helloWorld.xml -p english

Note that the magnet executable provides command-line help:

magnet -help or magnet -h

Magnet allows passing in VM properties (-Dprop=name), -X options, etc.

Advanced Issues

Rendering and Startup

Magnet uses the Confix API to parse a Magnet file and create an object representation of it. This dynamically generated object graph is then traversed; each object performs its "rendering"; it is passed a context that holds, among other things, the parameters that were previously declared through parameters elements. Rendering thus consists of variable interpolation and initialization.

If a given Magnet config inherits from another one, the latter is rendered first; the parent Magnet's global parameters and classpaths are accessible from the child Magnet, and so on.

Once a Magnet config has been properly rendered and initialized, it is "started": each launcher is started in its own thread, in the order in which it appears.

More on Parameters and Variable Interpolation

Magnet parameters can be declared at the "global" level, or (in the case of the "java" launcher), at the "profile" level. When declaring parameters at the global level, one can opt to "export" parameters to the JVM's system properties. In such a case, the name/value pair of an exported parameter becomes the name/value pair of a system property, and is thus accessible VM-wide. The following shows how to export a parameter to the system properties:

<parameters>
  <param name="naming.port" 
         value="8200"
         scope="system" />
</parameters>

By default, all global parameters have "magnet" scope: they are visible from within the current magnet only (and from children magnets, if such is the case). The scope attribute of the param element takes either "system" or "magnet" as a value - we'll let you guess which corresponds to what...

Global parameters can also be defined on a per-profile basis. Take the following:

<parameters profile="dev">
  <param name="naming.port" 
         value="8200"
         scope="system" />
</parameters>

The above example demonstrates the use of the profile attribute. The latter's value must in this case correspond to the current profile for the encapsulated parameters to be rendered.

If a given parameter is declared twice, the second declaration overrides the first (this rule also applies to system properties: a Magnet parameter that is exported to the system properties can potentially override an existing system property, so handle with care). Parameters that have "local" scope - defined at the profile level - will not be exported to the system properties; a local parameter can override (for a given launcher) a global parameter. More clearly, an override defined "locally" will only take effect locally; all other launchers will still "see" the global parameter in its original state.

As was explained in the Hello example, parameters can be used in the context of variable interpolation. The interpolation mechanism is tightly coupled to parameter scope; when Magnet tries to substitute a variable by its value, it will search for that value in the current scope, and then "up" if no value is found. For example, given the following snippet:

<launcher type="java" name="variableExample" 
    mainClass="org.sapia.magnet.examples.DummyApp"
    args="-logDir ${user.dir}/log" />

The value of the user.dir variable will be searched first in the local scope (in the parameters defined within the current launcher); if no value is found, then the global parameters are searched; if the search remains unsuccessful, the system properties are searched. Indeed, user.dir maps to a system property...

Using Application Arguments

Application arguments are parameters that can be passed to a main() method of a Java class when invoking the Java virtual machine (list of arguments after the main class name when calling the java command). Magnet provides a way to access those arguments. Through the usage of variable interpolation, these arguments could be used within your Magnet file. See the running magnet page for more details on how to pass application arguments when running a Magnet config.

For each application argument received, Magnet will create a system property in the JVM. The property name uses the pattern magnet.args.n where 'n' is the number of the argument, starting with 1.

Magnet will also define the system property magnet.args.* which will contain a concatenated string of all the application arguments received. This can be handy if you need all the arguments at once and not a list of each individual argument.

For example, here's the list of system property that Magnet would create if you would run the following command: magnet -f MetricConversion.xml -p weight 10 50 125

  • magnet.args.1 = 10
  • magnet.args.2 = 50
  • magnet.args.3 = 125
  • magnet.args.* = 10 50 125

Scripting

Sometimes, it is necessary to perform some operations prior to the application(s) being started: create some directories under the app root ("log", "db", "tmp", etc.), conditionnally set some system properties, etc. In such a case, Magnet offers scripting support, through the script element. The following is a typical script:

<script type="bsh" isAbortingOnError="true">
  File aFile = new File(System.getProperty("user.dir"),
         String.valueOf(System.currentTimeMillis()));
  System.out.println("Creating temp directory " + 
         aFile.getName());
  aFile.mkdirs();
</script>

The script element takes the text of a script and passes it to an interpreter. Currently, Magnet supports Beanshell. Beanshell is a Java interpreter that can be used in a magnet by specifying the "bsh" value for the type attribute on the script element. The isAbortingOnError attribute indicates if the current Magnet's execution should be stopped provided the script generates an error.

The script element goes under the root element. Magnet will execute the script - if any - as the first step in the rendering proceess.

Starting System Processes

So far we talked about Magnet as being a tool to start Java programs, but this is not entirely true. The fact is that we created Magnet having in mind that the XML configuration could be used to start any type of process. If you look again at the previous "Hello XXXXXX" you can see that the launcher element has a type attribute with the "java" value; in addition, Magnet provides the "system" laucher type, that allows one to execute native system commands. So for example if the Java application you wish to start in your Magnet is dependent on a MySQL database, you could first execute a system command to start the database and then start your Java application. Another good example is when you start a web application with Magnet; you may wish to automatically open a web browser to the URL of your application. Hey, guess what, the following Magnet script indeed opens internet Explorer on a given URL:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE magnet PUBLIC "-//SAPIA OSS//Magnet DTD 2.0//EN"
                 "http://www.sapia-oss.org/dtd/magnet-2.0.dtd"> 

<magnet xmlns:magnet="http://schemas.sapia-oss.org/magnet/"
  name="SystemMagnet" description="Test system launcher">

  <parameters>
    <param name="SystemRoot" value="C:\Windows" />
    <param name="iexplorePath"
      value="&quot;C:\Program Files\Internet Explorer&quot;" />
  </parameters>

  <environment id="windows_env">
    <variable name="ComSpec"
      value="${SystemRoot}\system32\cmd.exe" />
    <variable name="SystemRoot"
      value="${SystemRoot}" />
    <variable name="windir"
      value="${SystemRoot}" />
  </environment>

  <launcher type="system" name="StartBrowser" default="yahoo"
            command="cmd /C start /B iexplore.exe ${url}"
            workingDirectory="${user.home}">
    <profile name="yahoo">
      <parameters>
        <param name="url" value="http://www.yahoo.com" />
      </parameters>
      <environment parent="root">
        <variable name="path"
          value="${SystemRoot}\system32;${iexplorePath}" />
      </environment>
    </profile>
    <profile name="google">
      <parameters>
        <param name="url" value="http://www.google.com" />
      </parameters>
      <environment parent="root">
        <variable name="path"
          value="${SystemRoot}\system32;${iexplorePath}" />
      </environment>
    </profile>
  </launcher>
</magnet>

The previous example shows how we can start Internet Explorer on a Windows machine using a system launcher. This Magnet config would start the browser with the Yahoo or Google start page - depending on the passed in profile. Let's examine this example to understand how it works: you can see that the child elements of this launcher sightly differ from the ones of the "java" launcher. The environment element is used to define environment variables that will be passed to the system process. Because the information required to start a Java program differs from the one required to start a system process, the attributes of the launcher element are different: the command attribute replaces the mainClass used in the Java launcher and a workingDirectory attribute is added.

See the launcher directory of the distribution for more examples.

Conclusion

As you can see, Magnet can free you from the start-up script nightmare. It also allows you to start multiple applications per JVM, and sports a "profile" feature that simplifies application startup across different environments. To learn more, download and try. You will find examples in the /etc directory of Magnet's distribution. Have fun!