Creating a RET Adapter

The Refinitiv Electronic Trading (RET) Adapter Toolkit contains a set of classes that you can extend to create a suite of adapters that integrate with a Refinitiv Electronic Trading system and supply RET data to the Caplin Platform.

Download the presentations for this course.

Overview

The Caplin RET Adapter Toolkit provides classes to create six types of adapter:

  • Calendar Adapter

  • LBN Adapter

  • Orders Adapter

  • Permissions Adapter

  • Pricing Adapter

  • Trading Adapter

The role of each adapter is to respond to outgoing requests from the Caplin Platform on RET related data or actions, and to act accordingly by communicating with the RET system and providing a response. This communication between an adapter and a RET system is managed by the Refinitiv Trading API, as well as other RET specific APIs that are used to integrate external systems.

Objectives

In this training module we will take an in depth look at the example Calendar Adapter. We will use this adapter as an example for configuring a connection to live data streamed via RET. You will need to configure the connection to the Refinitiv Trading API for the Calendar Adapter to establish a connection to a RET system.

You are going to learn to set up an operational Calendar Adapter, where you will:

  • Configure and connect the adapter to the Refinitiv Trading API

  • Integrate the adapter with the Caplin Platform, and deploy it as a blade into the Deployment Framework

  • Test that the adapter is working as expected

  • Take what you have learned in the Calendar Adapter example and repeat the steps for all other adapters

Prerequisites

You will need:

  1. The following Caplin libraries:

    • Java DataSource API 7 (datasource-java-version-jar-with-dependencies.jar)

    • Java Permissioning API 7 (permissioning-datasource-version.jar)

    • Java Trading API 7 (trading-datasource-version.jar)

  2. Caplin’s front-end web trading application: Caplin FX Professional

  3. Caplin’s RET Adapter Toolkit (RETAdapterToolkit-version.zip)

  4. The following Refinitiv libraries:

    • trapi-lib

    • trapi-rates-plugin

    • lbn

    • admin

  5. Refinitiv licences: trapi.lic and lbn.lic

  6. The following user accounts in Refinitiv Electronic Trading (RET):

    Each learner on the course requires their own set of RET user accounts, so the usernames below are prefixed with an arbitrary namespace, namespace_, that must be unique to each learner.
    Adapter accounts
    Adapter Username User Type

    Calendar Adapter

    namespace_cal_client

    Proxy client

    namespace_cal_admin

    Administrator

    LBN

    namespace_lbn_proxy

    Bank Node

    namespace_lbn_admin

    Administrator

    Pricing

    namespace_rates_proxy

    Proxy Client

    namespace_rates_admin

    Administrator

    Trading

    namespace_fx_proxy_1

    Proxy Client

    namespace_fx_proxy_2

    Proxy Client

    namespace_fx_admin

    Administrator

    Orders

    namespace_monitor_proxy

    Proxy Client

    namespace_order_proxy_1

    Proxy Client

    namespace_order_proxy_2

    Proxy Client

    namespace_order_admin

    Administrator

    Permissioning

    namespace_perms_proxy

    Proxy Client

    namespace_perms_admin

    Administrator
    Client accounts
    Username Modifier

    namespace_client_1

    Yes

    namespace_client_2

    Yes

    namespace_client_3

    Yes

    namespace_client_4

    Yes

    namespace_client_5

    Yes

    namespace_client_6

    Yes

    namespace_client_7

    Yes

    namespace_client_8

    Yes

    namespace_client_9

    Yes

    namespace_client_10

    Yes
    Sales trader accounts
    Username Modifier

    namespace_client_st

    No

    The 'Modifier' option is set in the RET Admin applet and adds a margin to rates to cover costs and risks to the bank.

Creating a Workspace

You will need to set up a development environment by preparing a workspace and tools. The first step is to create an adapter blade — an adapter packaged in a blade — that includes all the necessary RET configuration. As a starting point, we can simply copy a ready-made example that is supplied with the RET Adapter Toolkit.

Follow the steps below:

  1. Open up your IDE and create a new Gradle Project with the name CalendarAdapter. It is important that you use this particular name as it is referenced in the start.jar file. Click Finish when you are done.

  2. Copy the contents of <RETAdapterToolkit>/examples/CalendarAdapter/ into your newly created project.

  3. Copy all files in the <RETAdapterToolkit>/lib/ and <RETAdapterToolkit>/lib/third-party directories to the <workspace>/CalendarAdapter/Blade/DataSource/lib/ directory.

    The <workspace>/CalendarAdapter/Blade/DataSource/lib/ directory must not contain any sub-directories.

  4. Copy the following Caplin software libraries to the <workspace>/CalendarAdapter/Blade/DataSource/lib/ directory: FXIntegrationAPI, datasource-java, permissioning-common, permissioning-datasource, and trading-datasource.

  5. Copy the following Refinitiv libraries to the <workspace>/CalendarAdapter/Blade/DataSource/lib/ directory: trapi-lib, trapi-rates-plugin, admin.

  6. Copy the Refinitiv licence file, trapi.lic, to the <workspace>/CalendarAdapter/Blade/overrides/CalendarAdapter/etc directory.

  7. Right-click your project and click Properties > Java Build Path to open the Build Path dialog.

  8. Under the Source tab of the Properties dialog box, change the source folder from src to src/main/java. Do this by clicking on the Add Folder button, expanding the src folder down to the Java folder and then enabling the Java folder by selecting the checkbox. Finally, remove the old src folder and click OK when you are done.

    We use a Maven-based directory layout.

  9. Under the Libraries tab in your Java Build Path, click the Add Jars button, and select all files in the directory <workspace>/CalendarAdapter/Blade/DataSource/lib/.

  10. Finally, you can begin compiling your project using two methods:

    1. In your IDE, select the build project command:

      1. Eclipse: Menu > Project > Build

      2. IntelliJ: Menu > Build > Build Project

    2. You can also use the project’s Gradle tasks to compile your project. Do this by opening a command-line interface and using the project’s Gradle build command or run it in your IDE:

      $ ./gradlew clean build

Once your adapter has been built, you will then configure it to connect to a live RET system via TrAPI - see the next section.

Connecting to Refinitiv Trading API (TrAPI)

The Calendar Adapter must be configured to connect to Refinitiv Trading API (TrAPI) in order to communicate with a RET system.

Usernames and passwords used in the following connection configuration files are examples only, and each user may only connect once.

For the calendar adapter, you can enable live Calendar data by replacing the contents of the Blade/overrides/CalendarAdapter/etc/trapi-connection.properties file with the following values.

Blade/overrides/CalendarAdapter/etc/trapi-connection.properties
[common]
host=<SCS Relay Host>
port=<SCS Relay Port>

[CalendarConnection]
username=<namespace>_cal_proxy
password=<password>

[AdminConnection]
username=<namespace>_cal_admin
password=<password>
order_type=admin
host and port must refer to the host and port of your SCS Relay, and any usernames must match the RET username/passwords provided to you.

Connecting to the Caplin Platform

After we have configured the TrAPI connectivity and set up the RET connection, we will need to configure the adapter so that it can communicate RET data to Liberator or Transformer (Caplin’s back-end systems).

Open the file <workspace>/CalendarAdapter/global_config/hosts-defns.conf and set the name of each hostname to the location of your Caplin Platform deployment. The hostnames by default should be localhost unless your Platform deployment is on a remote machine, then use the appropriate host name.

The Deployment Framework file hosts-defns.conf is used to generate the file hosts.conf. Both files reside in the same location. If you have used the Project templates to generate a new project follow the instruction under: Running your adapter within an IDE. This will not be a problem in deployments that use the Deployment Framework.

Exporting the Platform Configuration

As we have previously configured the RET adapters, you will need to export all configuration and deploy the adapters into the Caplin Platform. The Platform is managed by the Deployment Framework. The Deployment Framework enables you to deploy and run these various components via an easy-to-use command line interface.

To deploy your adapter’s configuration to your Deployment Framework (DFW), follow the steps below:

  1. From the root directory of your DFW, run the command below to stop all running Platform components:

    ./dfw stop

  2. Start by copying the file <RETAdapterToolkit>/examples/config into <DeploymentFramework>/global_config.

    This will change the Deployment Framework’s default configuration to the configuration required by the RET Adapter Toolkit.

  3. Now export the new configuration:

    Execute the following Gradle command either from your IDE or from your console:

    ./gradlew assemble CalendarAdapter --Pconfig-only

    The new configuration blade is created under your project’s build/distributions directory.

    Ensure that your overrides configuration was deployed correctly, otherwise you will have to manually copy your Blade/overrides/CalendarAdapter over to your DeploymentFramework/global_config/overrides/CalendarAdapter.

  4. Copy the blade ~/src/<adapter>/build/distributions/<adapter>-version.zip to your DFW’s kits directory:

    cp ~/src/build/distribution/<adapter>-version.zip ~/DeploymentFramework-version/kits

  5. Deploy the configuration into your Deployment Framework. Do this by opening a Cygwin console on your machine and navigating to your Deployment Framework’s topmost directory. Then execute the following command:

    ./dfw deploy

    This will have configured Liberator to send any requests such as calendar data request directly to the Calendar Adapter.

  6. Now start your Deployment Framework from a Cygwin terminal. Do this by navigating to the topmost directory of the Deployment Framework and executing the following command:

    ./dfw start

Running the adapter in your IDE

To run the adapter in your IDE, follow the steps below:

  1. In the Arguments tabe of the Run Configurations dialog box, look under Working directory. In the Other field, type the correct working directory: ${workspace_loc:CalendarAdapter/Blade/DataSource}

  2. In the Environments tab, add the following variables:

    • CONFIG_BASE = ../../global_config/ (points to the right configuration directory)

    • LOG_USE_PARENT_HANDLERS = true (to display the logs in the IDE’s console)

Testing the Adapter

Now that we have our Calendar Adapter deployed into the Caplin Platform (the back end) - let’s now test the adapter by running it in your IDE.

Follow the steps below:

  1. Using the blade tool’s run command that is available with the Caplin Integration Suite, we will run the adapter. Open a command-line tool and execute the following:

    java -jar <cis-blade-toolkit> run CalendarAdapter -m com.caplin.example.ret.CalendarAdapterExample

You can use the above step to confirm that the adapter is working. Finally, use Ctrl+C to shut-down the adapter.

Troubleshooting the Adapter

If at any point you have trouble testing the adapter, the debugging steps outlined below can be helpful.

Checklist:

  1. Use the Liberator status page on http://localhost:18080/status to determine whether the adapter is UP or DOWN.

  2. Check the console output; unrecoverable exceptions will be displayed here.

  3. Check the adapter logs in <workspace>/CalendarAdapter/Blade/DataSource/var/, in particular the all-calendar-adapter.log (you can find an all-<adapter_name>.log). Any working adapter should log the message Set status to <UP>. If the adapter status is <DOWN> or no message has been logged, this means the adapter is not ready to receive requests. This could be the result of an error, or the adapter may still be initialising. An analysis of the logs will reveal the cause of the problem. Some common problems include:

    • A failed connection to Refinitiv Trading API (TrAPI)

    • The network is unavailable

    • The TrAPI licence is invalid - these issues can be identified through any exceptions that have been logged in the log files.

  4. Test if the Calender Adapter is sending data by opening the Liberator Explorer. Navigate to http://localhost:18080/ and click on Diagnostics above, then click on the liberatorexplorer URL link. Now, in the field labelled 'Record' type /CALENDAR/TENORDATES/GBPUSD and click Go.

    You should now see some columns and a single row. View the 'Tenor' column to see how the values are displayed. In the next training module we will look at ways of modifying these values.

Deploy the other Adapters

To complete training, you will need to deploy and configure the other Adapters so that we have the entire Adapter Suite running.

Follow the steps below:

  1. Unzip the provided RETAdapterSuite.zip file

  2. Copy all files from RETAdapterSuite/blades, except for the Calendar Adapter, into your DeploymentFramework/kits folder.

  3. From the root directory of your Deployment Framework, run the command below to deploy the adapters:

    ./dfw deploy

Deploy RET Licences

For each of the adapters, copy the trapi.lic file into the overrides folder. Additionally, for the LBNAdapter you will also need to copy lbn.lic as well.

Configure the connection properties for Refinitiv Trading API

You will need to configure your adapters within your Deployment Framework. This is usually done in the overrides folder of each adapter. You can find this folder under DeploymentFramework/global_config/overrides.

Follow the steps below:

  1. To enable live data capture for the Execution and Historic Blotter, update the contents of the file global_config/overrides/LBNAdapter/etc/trapi-connection.properties with the following values:

    [common]
host=<SCS Relay Host>
port=<SCS Relay Port>

[LBNConnection1]
username=<namespace>_lbn_proxy
password=<password>
order_type=LBN

[AdminConnection]
username=<namespace>_lbn_admin
password=<password>
order_type=admin

  2. To enable live pricing data, update the contents of the file global_config/overrides/PricingAdapter/etc/trapi-connection.properties with the following values:

    [common]
host=<SCS Relay Host>
port=<SCS Relay Port>

[RatesConnection]
username=<namespace>_rates_proxy
password=<password>

[AdminConnection]
username=<namespace>_rates_admin
password=<password>
order_type=admin

  3. To enable live trading, update the contents of the file global_config/overrides/FxTradingAdapter/etc/trapi-connection.properties with the following values:

    [common]
host=<SCS Relay Host>
port=<SCS Relay Port>

[TradingConnection1]
username=<namespace>_fx_proxy_1
password=<password>
use_for_trading=true

[AdminConnection]
username=<namespace>_fx_admin
password=<password>
order_type=admin

  4. To enable the retrieval of live permissions from RET, update the contents of the file global_config/overrides/PermissioningAdapter/etc/trapi-connection.properties with the following values:

    [common]
host=<SCS Relay Host>
port=<SCS Relay Port>

[AdminConnection]
username=<namespace>_perms_admin
password=<password>
order_type=admin

  5. To enable live Orders trading, update the file global_config/overrides/LimitOrderAdapter/etc/trapi-connection.properties with the following values:

    [common]
host=<SCS Relay Host>
port=<SCS Relay Port>

[TradingConnection1]
username=<namespace>_order_proxy_1
password=<password>
use_as_monitor=false
order_type=limit

[MonitorConnection]
username=<namespace>_monitor_proxy
password=<password>
use_as_monitor=true
order_type=limit

[AdminConnection]
username=<namespace>_order_admin
password=<password>
order_type=admin

You should now be able to start all the adapters using the command ./dfw start

Troubleshooting the Adapters

If at any point you have trouble testing an adapter, the debugging steps outlined below can be helpful.

Checklist:

  1. Use the Liberator status page on http://localhost:18080/status to determine whether the adapter is UP or DOWN.

  2. Check the console output; unrecoverable exceptions will be displayed here.

  3. Check the adapter logs in <workspace>/Adapter/Blade/DataSource/var/, in particular the all-*-adapter.log (you can find an all-<adapter_name>.log). Any working adapter should log the message Set status to <UP>. If the adapter status is <DOWN> or no message has been logged, this means the adapter is not ready to receive requests. This could be the result of an error, or the adapter may still be initialising. An analysis of the logs will reveal the cause of the problem. Some common problems include:

    • A failed connection to Refinitiv Trading API (TrAPI)

    • The network is unavailable

    • The TrAPI licence is invalid - these issues can be identified through any exceptions that have been logged in the log files.

Review

The techniques covered in this training module can be used in the development of all RET Adapters.