Legacy Caplin Transformer Java Module SDK 8.0.2-290852-a608fcd3

The Transformer Module SDK for Java provides Java developers with the tools necessary to perform business rules and calculations on data updates being passed between applications connected to the DataSource API (DataSource peers such as Caplin Liberator servers).

For example, you can use the SDK to do the following:

  • calculate the top 10 shares in terms of price, yield, or number of transactions per hour;
  • log incoming data to a database, and if a corrected or marked trade appears in a feed determine whether that is passed on or not;
  • simplify updates so that if multiple updates are contained within the update packet, only the last (and hence most significant) is sent out;
  • perform predetermined tasks according to market conditions, such as sending initialisation messages when markets open and end-of-day calculations when they close;
  • process suspensions in trading and determine what to do with the data in alignment with market rules.

Each calculation or set of calculations is performed by individual modules connected to a central core. The Transformer Core distributes incoming data to the modules that have requested it. It is also a DataSource peer.

A module for Caplin Transformer takes the form of a dynamic library which is loaded on startup or on receipt of a UDP command. A Java module is loaded into a VM started up by the Transformer Core, configured from a file. When a module requests data, the data sink element of the Transformer Core forwards the request to that peer or set of peers which can provide it. When an update occurs, the Core extracts the returned data from the DataSource API and sends it to the module for processing.

The module can then output the transformed data to the Core, for distribution in two possible ways:

  • publishing to the DataSource API, where it can be picked up by any suitably-configured application connected to the DataSource API which has requested that data.
  • delivering to a Caplin Liberator, which can then publish the information to clients over the web.

There are two main steps to using the Transformer Module SDK:

  1. Edit the configuration file transformer.conf so that the Transformer Core can communciate with its modules and other applications connected to the DataSource API.
  2. Use the Java classes provided by the SDK to create the individual business modules to process the data supplied through the Transformer Core.

Using the Transformer SDK for Java

Follow the links on the table below for information on the following topics:

Steps to create a module
Create the main module object
Request the data needed for calculations
Respond to requests from peers
Manipulate incoming data
Publish the transformed data
Configure the Core so that it can load the module
Using UDP commands

Create the main module object

The TransformerModule is the main interface for the Transformer Module Java SDK. This interface must be implemented by a developer, and the resulting class should be the one referenced in the transformer.conf file.

In addition, a reference to the TransformerAccessor passed into the Transformer module initialise method must be kept in order to access the main interfaces needed to communicate with the Transformer Core.

TOP

Request the data required for calculations

In order for Caplin Transformer to supply updated information to a module, the module has to tell the Core that it has interest in a particular set of symbols. The Core then makes an active request for the symbols from DataSource applications connected to the DataSource API, unless a requested symbol is already within the its memory (i.e. another module has requested it), in which case your module will receive the data without it being rerequesting from the data source.

The Subscriber object allows a module to register and deregister subscriptions with the Transformer Core. Subscriptions can be made for both object name patterns and ObjectTypes.

If you specify wildcards in the object name pattern (for example /LO/* for all symbols on the London Stock Exchange), the data cannot be requested from any peers: your module must wait for the information to come into the Transformer Core. Only if the exact symbol is known can an active request be made to a DataSource peer (for example /LO/BAY). Wildcards are particularly useful for listening for broadcast data however, which will come into the Transformer Core without being actively requested by any module.

Example:

SubscriptionListener subListener = new LondonSubscriptionListener();
transformerAccessor.getSubscriber().addSubscriptionListener("/LO/*", subListener);

public class LondonSubscriptionListener implements SubscriptionListener
{
   public void objectDeleted(String objectName)
    {
       // code to handle object deletion...
    }

   public void update(DataSourceUpdateEvent event)
    {
       // code to handle update to an object...
   }
}

Having requested data for a particular symbol, whenever any values associated with that symbol change, the Transformer Core will receive a packet of information containing an array of values. Individual values can then be extracted and processed.

TOP

Respond to requests from peers

The DataProviderRegistrar class allows a module to register and deregister itself as a provider of data. The module should register itself as a provider of data for a particular object name pattern. When a DataSource peer makes a request for an object that matches the pattern, the module will be asked if it can provide data for that DataSource object.

Example:

DataProvider chartingProvider = new ChartingDataProvider();
transformerAccessor.getDataProviderRegistrar().register("/CHART/*", chartingProvider);

public class ChartingDataProvider implements DataProvider
{
    public DataProviderResponse request(int dataSourcePeerId, String objectName)
   {
      // code to create a thread to start contributing data when
      // this method has finished executing...

       return DataProviderResponse.OK;
   }
   public DataProviderResponse discard(int dataSourcePeerId, String objectName)
    {
      // code to stop contribution of data for the specified
      // object...

       return DataProviderResponse.OK;
    }
}

TOP

Using object name patterns

Every exchange uses a different set of symbols to identify the various financial instruments that are traded. Object name patterns enable you to search all available symbols and return a list of those fitting your search criteria.

Patterns may contain asterisk "*" and question mark "?" wildcards. The asterisk will match any number of characters, whilst the question mark will match any one character.

For example:

  • a search string of "/LO/*" will return all symbols traded on the London Stock Exchange and a search string of "/*/MSFT" will return the symbols of Microsoft stocks on all available exchanges (assuming constant symbology).
  • the pattern /LO/* will match /LO/VOD, /LO/CAPLIN and /LO/DIR/OBJ1, but will not match /NA/VOD, /DIR/LO/VOD or /LO2/CAPLIN.
TOP

Manipulate data

The DataCache class provides a module access the data that is cached within the Transformer core. This data might be useful for helping with calculations based on an update that has just been received.

Finding updates to manipulate

The latest values for an object can be retrieved from the Transformer Core using get(objectName) method. To find out which objects are available in the Core, the getObjectNames(objectNamePattern) method can be used.

Example:

TransformerData vodafoneData = transformerAccessor.getDataCache().get("/LO/VOD");
if (vodafoneData != null)
{
    // code to process the cached data...
}

Finding the time of an update

The method getLastUpdateTime(objectName) returns the last time an update was received for this symbol. This can be used to create modules which output data regarding time periods for use in charts.

Copying updates

Once data has been sent to the Transformer core its memory is freed. If the update needs to be sent, then further changes made to the object, either the DSFactory.createTransformerData(TransformerData) or the DSFactory.createTransformerRecord(TransformerRecord) methods can be used to create a copy of the object before the update is sent.

Updating manipulated fields

In order to process a field value of an update, you must get a reference to the field (DSField) using one of the following methods:

The field value or field number/name can be modified using the following methods:

Example:

public class FieldModificationSubscriptionListener implements SubscriptionListener
{
   public void objectDeleted(String objectName)
   {
      // code to handle object deletion...
   }

   public void update(DataSourceUpdateEvent event)
   {
      // get the field with field number 17 and, if it exists,
      // round it value the the nearest whole number
      DSField field = event.getTransformerData().getFieldByFieldNumber(17);
      if (field != null)
      {
         int roundedValue;
         try
         {
               float actualValue = Float.valueOf(field.getValue()).floatValue();
               roundedValue = Math.round(actualValue);
         }
         catch (NumberFormatException e)
         {
               roundedValue = 0;
         }

         field.setValue(Long.toString(roundedValue));
      }
   }
}

TOP

Publish the transformed data

The Caplin Transformer core can output data in the following ways:

  • publish data to DataSource peers connected to the DataSource API; or
  • send responses to requests for data from Caplin Liberators. When the core receives a request, it checks its list of providing modules for a match (modules tell the core which symbols they handle), and if a match is made calls the module. If the module does not handle that symbol then a request is sent out to the data sources and Transformer filters the results and sends them back to Liberator.

Example:

String objectName = "/LO/CPLN";
try
{
   // add a place holder for the object within the Core
   transformerAccessor.getDataCache().put(objectName, DataCache.SILENT_SEND);

   // send an update for the object
   TransformerRecord record = DSFactory.createTransformerRecord(objectName);
   record.addRecordData("CompanyName", "Caplin Systems Limited");
   record.addRecordData("Currency", "GBP");
   record.addRecordData("Volume", 0);
   record.send();
}
catch (DataCachePutException e)
{
   // code to handle exception...
}

TOP

Configure Transformer Core to load the module

To load a Transformer Java Module, the following configuration settings should be added to the Core's configuration file etc/transformer.conf:

  • add-module
    Tells the Core to load a module. Should contain the name of the .so file (jtm.so) and can also set a flag to determine whether the Core API can be extended to be used within the module.
  • jvm-location
    Location of the libjvm.so file. Should contain the complete path and include the .so/.dll suffix.
  • jvm-global-classpath
    Location of the global classpath. Default is %r/lib/java. To include several classpaths, separate jvm-global-classpath entries should be added for each one.
  • add-javaclass
    Identifies the module to be loaded.

Note: For more information on these and other parameters, and how the Transformer Core is configured, please refer to the Transformer Core Administrator's Guide.

The example configuration below loads a Transformer Module implemented in a Java class, examples.first.FirstTransformerModule.

add-module       jtm.so

# Configuration of Java class to be loaded
add-javaclass
    class-name   examples.first.FirstTransformerModule
    class-id     jtm
    classpath    %r/lib/java/examples.jar
end-javaclass

# Configuration of the JVM
jvm-location          /usr/local/jdk/jre/lib/sparc/libjava.so
jvm-global-classpath  %r/lib/java/transformermodule.jar

To add other startup options for the JVM, multiple configuration lines beginning jvm-options may be specified. For example, to enable socket debugging on port 9955 the following configuration options could be added:

jvm-options    -Xdebug
jvm-options    -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=9955

To load a second Transformer Java Module, make a copy of jtm.so (found in the lib folder), and add a second add-javaclass entry. The class-id parameter must be set to the name of the copied file (jtm2 in the example below).

For example, if jtm.so were copied as jtm2.so, a second module could be loaded by:

add-module       jtm2.so

add-javaclass
    class-name   examples.second.SecondTransformerModule
    class-id     jtm2
    classpath    %r/lib/java/examples2.jar
end-javaclass

TOP

Using UDP commands

The UDPAccessor allows a module to send UDP commands and to add or remove its own listeners for UDP commands.

Listeners associated with a certain UDP message will be informed each time that command is executed.

If a particular listener is added multiple times for the same command, then it will be informed of a single execution of that command multiple times. When a listener that has been added multiple times for the same command is deleted, only the first occurrence Listeners will be removed. If one particular listener is used to listen to multiple commands, it can identify which command was executed from the first command argument (which is always the command itself).

TOP

UDP commands can be used to change logging levels for debugging purposes and the writing of lists of watched objects to files.

Example:

// tell the Transformer Core to set the debug level to warn
transformerAccessor.getUDPAccessor().send("debug-level " + ServerLevel.WARN_LEVEL.intValue());

Other UDP commands that may be useful to a Transformer module include the following:

  • memory_write
    tells the Core to write its memory to file. This will be read back when the Transformer is restarted.
  • symbol_publish arg1 arg2 ... argN -
    tells the Core to publish all of the data for the specified objects. The object arguments can either refer to exact object names, or can be object name patterns, including wildcards.
  • source_up peerId
    a command sent from the Core to inform its modules that the specified peer is now up. If your module needs to know about which peers are up, then it should register a listener for the source_up command.
  • source_down peerId
    a command sent from the Core to inform its modules that the specified peer is now down. If your module needs to know about which peers are down, then it should register a listener for the source_up command.

Other commands can be sent by a Transformer module; the Transformer core will send these to any other modules that are listening for them.

Example:

// tell the Transformer Core to write its memory out to disk
transformerAccessor.getUDPAccessor().send("memory_write");

TOP

Logging

TransformerAccessor provides a getLogger() method which returns a java.util.logging.Logger that can be used to write messages to the Transformer module's log file.

ServerLevel provides the different levels at which a message can be debugged.

Value Description
DEBUG Reports all errors and events
INFO Reports events and information regarding normal operation and all errors included in the WARN, NOTIFY, ERROR and CRITICAL debug levels.
WARN Reports minor errors and all errors included in the NOTIFY, ERROR and CRITICAL debug levels.
NOTIFY Reports errors regarding data corruptions and all errors included in the ERROR and CRITICAL debug levels.
ERROR Reports serious errors regarding network connections and all errors included in the CRITICAL debug level.
CRITICAL Reports critical errors that prevent the module running.

Example:

transformerAccessor.getLogger().log(ServerLevel.ERROR_LEVEL, "This is an error message");

TOP