Deploying core components

The Caplin platform consists of various components: Liberator (the streaming server) and Transformer (the data transformation engine) form a part of the Caplin Platform; and Integration Adapters are adapters which sit between Liberator and Transformer and your banking system. Integration Adapters are created using the Caplin Integration Suite, which will be covered later on.

The Deployment Framework enables you to deploy and run these various components via an easy-to-use command line interface.

Objectives

In this tutorial you are going to take a brisk leap into the Caplin Platform. You will:

  1. Install the Deployment Framework

  2. Use it to deploy the Caplin Platform components, Liberator and Transformer.

  3. Change the default Liberator configuration.

  4. Install the Caplin Management Console, a utility for viewing information about your backend components.

Installing the Deployment Framework

Download Caplin’s Deployment Framework. Extract DeploymentFramework-<version>.zip to a desired location on your hard drive. If you are using Windows, be sure to extract this zip file using the unzip command in Cygwin rather than using Windows Explorer, to avoid issues with the unzipped files having Windows line endings.

The Deployment Framework looks like this:

tutorial dpfw deployment fw dir 2

Installing Liberator

Liberator is one of the main components of the Caplin Platform. Its role is to stream data from integration adapters to client applications.

Follow the steps below:

  1. Download the Liberator installation appropriate for your operating system and copy it to the kits directory of your Deployment Framework. (Note: Do not extract the contents of this zip file. This applies to any blade you will copy to your kits directory before deploying.)

  2. Open Cygwin (if using Windows) or your command line tool and navigate to the Deployment Framework’s directory.

  3. Run the following script to deploy Liberator.

    ./dfw deploy

  4. You should see the following output:

    Boot-strapping the Deployment Framework
  Unpacking Liberator kit Liberator-<version>.zip
  Liberator-<version>.zip successfully unpacked and stored in kits/archive
  Deploying evaluation license from Liberator kit
  Deploying example HTTPS keys from Liberator kit

  5. At this point you may wish to replace the evaluation licence with your own development licence file. You can do this by replacing the Liberator licence file (license-rttpd.conf) in the global_config\licenses folder in the Deployment Framework.

Upgrading the Liberator licence

Liberator is supplied with an evaluation licence which causes Liberator to automatically shut down after 30 minutes, among other limitations. You may have been given a replacement licence as part of this course which does not have the 30 minute shutdown. To replace the evaluation licence with your training licence:

Follow the steps below:

  1. Navigate to the directory <Deployment Framework Root>/global_config/licenses.

  2. Rename the existing license-rttpd.conf file to license-rttpd.conf.bk.

  3. Copy the license-rttpd.conf file that has been provided as part of your training material into this directory.

Running the Backend

Navigate to the Deployment Framework directory and run the following command: ./dfw start

This will start up Liberator which is currently the only deployed component. If you are running on Windows, you may be prompted to make a firewall exception for Liberator.

Use your web browser to open up http://localhost:18080. If this page loads then it’s an indication that Liberator has started up correctly. This is the Liberator home page hosted on Liberator’s HTTP server and contains some handy resources related to working with Liberator. We’ll use some of these resources in later tutorials.

Liberator Configuration

In this part of the tutorial you are going to alter some configuration which affects Liberator. First you will change the logging level and later change the HTTP port number.

Any configuration changes for the backend components should always be made in the global_config directory (and not inside the "kits" or "servers" directories). It is important to note that the global_config directory is the only editable part of the framework. This directory contains an "overrides" directory where you can override any component’s default configuration.

Changing the Log Level

Ensure that Liberator has started. Go to the following location servers/Liberator/var directory and open event-rttpd.log. Here you will notice that each log line has a log level associated with it like INFO or NOTIFY.

The log levels available (starting with the most critical) are: CRITICAL, ERROR, NOTIFY, WARN, INFO and DEBUG. By default Liberator’s logging level is set to INFO. (Notice that event-rttpd.conf has no DEBUG logs.) In this tutorial you are going to change the logging level to DEBUG so that more log messages appear in the log file.

Follow the steps below:

  1. Open the file global_config/overrides/servers/Liberator/etc/rttpd.conf

    INFO

  2. Change the log-level property to DEBUG

    DEBUG

  3. Save the file, then stop all components (./dfw stop), clear the logs (./dfw clean) and start the components again (./dfw start) . You can also achieve the same thing by just running ./dfw clean, which automatically stops the components before clearing the logs, and then ./dfw start. In Deployment Framework version 6.0.4 and later, you can perform all three tasks by running the single command ./dfw cleanstart.

  4. Reopen the file servers/Liberator/var/event-rttpd.log. This should now contain DEBUG messages which did not appear before.

    log

  5. Note that setting the log level to DEBUG tends to produce very large log files and is not ideal for production. Revert this configuration to INFO after having tried it out.

Changing Port Numbers

Another common customisation is to change a port number. Here we’ll change Liberator’s default HTTP port number:

The global_config directory contains two files, environment-defaults.conf and environment.conf. The Deployment Framework reads the contents of both of these files when it starts up.

The difference is that customers are instructed to never make any changes to environment-defaults.conf, because it will be replaced every time they take a new release of the Deployment Framework. Any configuration customisations that customers want to make should be placed in environment.conf instead. This file is used to override the default configuration, and is not touched by the upgrade process.

So, when you want to change the value of a configuration option in environment-defaults.conf, the correct procedure is to copy and paste it into your own file environment.conf and change the value, rather than edit it in place.

In some cases though, you won’t find the default macro definition in environment-defaults.conf. This is the case for the definition of Liberator’s HTTP port macro, which is now included in Liberator’s HTTP configuration blade.

Follow the steps below:

  1. Open the configuration file for the Liberator’s HTTP blade: kits/HTTP/Liberator/etc/rttpd.conf. Locate the default HTTP port macro:

    File: kits/HTTP/Liberator/etc/rttpd.conf
    define LIBERATOR${THIS_LEG}_HTTPPORT ${THIS_LEG}8080

    Notice that before this file assigns the value of the macro to Liberator’s http_port configuration item, it includes the file bootstrap.conf. If you open bootstrap.conf, then you will find that it, in turn, includes the file environment.conf. The ultimate inclusion of the environment.conf file gives you an opportunity to redefine the macro without editing the HTTP blade’s default configuration file (something you should never do).

  2. Copy the definition of the HTTP port macro and paste it at the end of the file global_config/environment.conf.

  3. Edit the macro definition, changing the last digit of the port number to an 8:

    File: global_config/environment.conf
    define LIBERATOR${THIS_LEG}_HTTPPORT ${THIS_LEG}8088

  4. Restart the Deployment Framework.

  5. Notice that the Liberator home page is no longer accessible at http://localhost:18080 (you may need to clear your browser cache to see it disappear), but it is now available at http://localhost:18088.

The tutorials in this course expect Liberator’s HTTP port to be 18080. So before you continue, revert the port change you made above. You can do this in one of two ways:

  • Edit the line in the file environment.conf. Assign a port of {$THIS_LEG}8080 rather than {$THIS_LEG}8088.

  • Delete the line in the file environment.conf. The Deployment Framework will fall back to the default value in environment-defaults.conf.

Restart the Deployment Framework and confirm that the Liberator status page is available at http://localhost:18080.

Installing Transformer

Transformer is another Caplin Platform component. It is a real-time data transformation engine that manages modules which manipulate the data being transferred to and from the Platform and connected data sources.

Follow the steps below:

  1. First stop all the components running in your Deployment Framework.

  2. Place the Transformer zip file in the kits directory and deploy (./dfw deploy) Transformer as you did for Liberator.

  3. Restart the components in the Deployment Framework. If you are running on Windows you may be prompted to make a new firewall exception for Transformer.

JMX Monitoring via the Caplin Management Console

It is possible to monitor the components running in the Deployment Framework by using an application called the Caplin Management Console (CMC). This provides a visualisation of the running components and how they are connected to each other, and also allows you to monitor the JMX objects exposed by each component.

All of the Caplin Platform components expose information about how they are running using Java Management Extensions (JMX). For example, the embedded JVM in Liberator uses JMX to expose information such as how many users are currently connected, the current throughput in terms of bytes read and written per second, CPU and memory usage and other information useful for monitoring.

The CMC is a Java application that connects to the components in the Caplin Platform using the JMX interface and provides a GUI for displaying and interacting with the information published by the components. It is useful for administrators of the Caplin Platform for both passive monitoring and also performing actions, such as ejecting users from Liberator and changing the log level of running components without restarting them.

Before connecting the CMC to Liberator and Transformer you need to activate the LiberatorJMX and TransformerJMX blades. These are standard blades provided in the Deployment Framework which contain the necessary snippets of configuration to turn on the JMX publishing feature of Liberator and Transformer.

Activating Liberator JMX and Transformer JMX

Follow the steps below:

  1. Open a command prompt (or Cygwin if you are using Windows) and navigate to the Deployment Framework main directory.

  2. Activate the LiberatorJMX blade: ./dfw activate LiberatorJMX

  3. Activate the Transformer JMX blade in the same way: ./dfw activate TransformerJMX

  4. Restart the components in the Deployment Framework: ./dfw start.

The LiberatorJMX blade uses a Java Virtual Machine. If you see something like the following in the output, you need to go and configure your JVM location:

Error: No JVM at the configured location: <path>/jre7/bin/client/jvm.dll
The JAVA_HOME environment variable is not set. Is this intentional?
If it is, review JVM_LOCATION in the file global_config/environment-defaults.conf.

To configure the JVM location, run the ./dfw java command.

Here’s an example of how to do this when the Deployment Framework is running under Microsoft Windows:

./dfw java "C:/Program Files (x86)/Java/jre7/bin/client/jvm.dll"

Installing and using the CMC

Follow the steps below:

  1. Place the CaplinManagementConsole-<version>.zip into the Deployment Framework’s kits directory.

  2. Run the ./dfw deploy command to deploy the CMC.

  3. Next execute ./dfw mon to create a CMC configuration file. This will find the JMX ports and authentication details for all deployed components which are JMX-enabled. This is used to configure the components within the CMC so that they can be monitored.

  4. Run ./dfw cmc to launch the CMC. (Note: If you are using ssh to run your Deployment Framework on Linux or Mac, then you will need to make the necessary configuration changes to run X11 apps.)

  5. The CMC will start up in the Overview screen. Liberator and Transformer will appear in the centre of the CMC panel. (Blue means that the component is up and running. Red means that the component has not started.)

    tutorial dpfw CMC.JPG

  6. Start up the Deployment Framework components to watch Liberator and Transformer turn on in the CMC.

When the CMC is running you will notice that the Deployment Framework takes a lot longer to shut down when you run ./dfw stop. From this point on, remember to close the CMC by clicking the close button before you stop the Deployment Framework.

Review

You should now have a good idea of how to deploy and run backend components using the Deployment Framework. You should also be aware that you can monitor your network of components using the CMC application, and also that you can change components' default configurations in the Deployment Framework’s global_config directory. In the subsequent tutorials you are going to be creating and deploying more Caplin Platform components, so the tasks covered in this tutorial are ones which you will be repeating several times.

You may find the following developer resources useful: