Installing the Refiner Service

In Transformer, data transformation services are implemented in Transformer modules. Caplin provides a Transformer module called Refiner that allows large lists of data (containers) to be efficiently filtered and sorted in real time on behalf of client applications. Refiner is supplied as a Service blade that’s deployed in the Caplin Platform Deployment Framework. This tutorial shows you how to deploy the Refiner blade and use it to sort and filter container responses.

Once you’ve completed the steps on this page, you should be familiar with how to deploy Refiner and how to request filtered and sorted data via Transformer.

In these instructions you’ll be using the dfw command of the Deployment Framework. Before entering any dfw command as ./dfw <command-name>, make sure your current (working) directory is set to the Deployment Framework’s topmost directory. In file paths, we’ve called this topmost directory <Framework-root>. For a list of dfw commands, click here.

Requirements

Refiner is a Transformer Java module and has the following requirements:

  • Oracle Java 8 (<= 8u202), Red Hat OpenJDK 8, or Red Hat OpenJDK 11

  • A Transformer that is licensed to run Java Transformer modules. Check your Transformer licence contains the string below:

    module jtm transformer

    For questions regarding licensing, contact Caplin Support.

Installation instructions

Follow the instructions below on all Caplin Platform servers in your Caplin deployment. The Refiner installation kit includes configuration for other components besides Refiner.

  1. Stop all components

    ./dfw stop

  2. Copy the installation kit CPB_RefinerService-<version>-<build_number>.zip to your Deployment Framework’s kits directory.

  3. Run the command below to deploy the Refiner blade:

    ./dfw deploy

  4. Increase Transformer’s JVM heap size to a value tuned to your production workload for Refiner.

    Refiner’s memory requirements are workload sensitive. If you have not yet tuned Transformer’s JVM heap size, increase Transformer’s JVM heap size to at least 2048MB (2GB). For more information on tuning a JVM heap, see Configure a DataSource application’s JVM heap.

  5. Start all components

    ./dfw start

Testing the installation

Follow the instructions below to check the Refiner Service is available:

  1. Stop all components:

    ./dfw stop

  2. Deactivate the LiberatorDemoDataSource blade, if you have activated it:

    ./dfw deactivate LiberatorDemoDataSource

  3. Activate the Transformer DemoDataSource blade:

    ./dfw activate TransformerDemoDataSource

  4. Start all components:

    ./dfw start

  5. Open a web browser and navigate to the Liberator server’s built in web page (for the URL, run the command ./dfw info in the root directory of your Deployment Framework).

  6. Click View Status.

    The Liberator should recognise the following data services and DataSources. In particular, the status page should show a data service for the Refiner Service, as highlighted in red here:

    Refiner service status shown on Liberator status page

  7. Examine the log file <Framework-root>/servers/Transformer/var/jtm.log

    If the Transformer is working correctly, this log contains the version number of the Transformer Module library:

    <YYYY/MM/DD-HH:MM:SS.NNN> +0000: INFO: Java Transformer module (6.2.0-297741) starting up
<YYYY/MM/DD-HH:MM:SS.NNN> +0000: INFO: Loading user class with identifier jtm
<YYYY/MM/DD-HH:MM:SS.NNN> +0000: CRIT: Thread [main/1]: Product: Java Transformer Module
Version        : 6.2.3-bbbbbb
Build Date     : dd-Mon-yyyy
Build Time     : hh:mm
Build Number   : bbbbbb
Copyright      : Copyright 1995-2014 Caplin Systems Ltd

    In the same log file, you should also see Caplin Refiner registering as a provider for the namespace it uses:

    2011/12/16-15:41:05.703 +0000: INFO: Registering as a provider of </FILTER/*>

  8. Examine the log file <Framework-root>/servers/Transformer/var/refiner.log

    If Caplin Refiner is working correctly, this log contains its version number:

    <YYYY/MM/DD-HH:MM:SS.NNN> +0000 - Thread [main/1]: SEVERE: Refiner 6.2.3-bbbbbb

  9. Open a web browser and navigate to the web page for the Liberator.

  10. Click Diagnostics > Liberator Explorer

    For more about using Liberator Explorer, see How can I…​ Use Liberator Explorer to request and send data.

  11. Follow the instructions in the sections below to test the filtering and sorting capabilities of Refiner.

Requesting an unfiltered container

The DemoDataSource adapter can supply a container called /EXAMPLES/PRICING/CONTAINERS/EQUITIES that’s populated with stock prices.

Use Liberator Explorer to request /EXAMPLES/PRICING/CONTAINERS/EQUITIES. This returns the unfiltered and unsorted container:

Unfiltered container of stock prices

Requesting a filtered container

In Liberator Explorer, request /FILTER/EXAMPLES/PRICING/CONTAINERS/EQUITIES?filter=(BestBid>20)

Liberator routes this request to Transformer, because its RefinerService1 data service contains the include-pattern ^/FILTER/ (have a look in the Liberator configuration file for the Refiner Service blade at <Framework-root>/kits/CPB_RefinerService/CPB_RefinerService-<version>/Liberator/etc/rttp.conf).

The /FILTER prefix directs Transformer to pass the request on to its Refiner module, which then returns just those instruments whose BestBid field is greater than 20

Stock prices filtered by Refiner

Requesting a sorted container

In Liberator Explorer, request /FILTER/EXAMPLES/PRICING/CONTAINERS/EQUITIES?sort=FullName:text:ascending

Now Refiner sorts the instruments in ascending alphabetical order of the FullName field:

Stocks sorted by name by Refiner
Avoid filtering or sorting on record fields that are subject to frequent updates. Frequent updates can have an adverse affect on the performance of Caplin Refiner, Liberator, and the requesting clients.

See also: