DataSource configuration syntax
This page describes the configuration syntax used by C DataSource API applications from version 5 and Java DataSource API applications from version 6.
Earlier DataSource API versions used an XML configuration format that was deprecated in the DataSource 6 APIs and removed in the DataSource 7 APIs.
Introduction
DataSource applications can be configured by settings stored in one or more configuration files, or by settings stored in a database and supplied to the DataSource application by a configuration script.
This document describes:
-
The syntax of the different types of configuration item used to configure a DataSource application.
-
How variables, conditionals, and macros can be used in a configuration file.
-
How configuration settings supplied by a web server or configuration script can be used to configure a DataSource application.
For a description of the configuration items that you can use to configure a particular DataSource application (such as Caplin Liberator, Caplin Transformer, or an Integration Adapter), refer to the Administration Guide for the application that you want to configure.
For a description of the DataSource communications infrastructure and how it links Caplin components together, see Caplin Platform > DataSource.
Types of configuration item
Four different types of configuration item can be used to configure a DataSource application:
Some of these configuration items allow you to specify a list of one or more values (see Lists).
Boolean configuration items
A boolean configuration item does not take any value, but is set to true if present in the DataSource configuration, and to false (the default) if omitted from the configuration.
An example of a boolean configuration item is latency-chain-enable, which enables or disables latency chaining.
latency-chain-enable
latency-chain-enable TRUE
Single-parameter configuration items
Single parameter configuration items take one parameter:
The parameter can be assigned a literal, a configuration variable, an environment variable, or an expression.
An example of this type of configuration item is datasrc-port, which specifies the network port that the application listens on for DataSource messages and connection requests.
datasrc-port 22001
In this case the listening port is set to 22001.
If present in the configuration file, a single value configuration item must define the value to be assigned. If not present in the configuration file, the configuration item is assigned a default value.
Multiple-parameter configuration items
Multiple-parameter configuration items take two or more parameters:
Each parameter can be assigned a literal, a configuration variable, an environment variable, or an expression.
An example of this type of configuration item is add-field, which maps a field name (a string) to a field number (an integer):
add-field Bid 22
In this case the Bid field is mapped to field number 22.
If a multiple-parameter configuration item is present in the configuration file, values must be defined for all parameters that are not optional (such as the field name and field number in the example above).
If a parameter is described as optional, the value for that related item can be omitted from the configuration. The parameter will be assigned its default value.
Values are assigned to parameters according to the position of the value in the list of ordered arguments.
If the configuration item is not present in the configuration file, related items are assigned default values.
Configuration items with nested options
This type of configuration item takes nested options, where each nested option can be any one of the four types of configuration item (boolean, single parameter, multiple parameter, and nested).
An example of this type of configuration item is add-peer, which specifies options for connecting to a remote DataSource application (see Data services).
add-peer addr liberator.example.com port 25000 end-peer
In this case the address of the remote DataSource is set to liberator.example.com, and the port that it listens on to 25000.
Lists
Some configuration items and options allow you to specify a list of one or more values, where each value is the same type.
An example is addr, a nested option of add-peer, which defines a list of addresses that the DataSource application will attempt to connect to:
addr 192.168.1.1 192.168.1.2 192.168.1.3
In the reference sections of the DataSource documentation, the syntax definition specifies the types that can be listed, as shown in the following table.
| List Type | Description | Example |
|---|---|---|
string list |
A space separated list (array) of one or more strings. |
A space separated list of IP addresses: |
integer list |
A space separated list (array) of one or more integers. |
A space separated list of port numbers: |
Literals
Literal values in a configuration file can be numbers or strings.
If a literal value contains one or more spaces, it must be enclosed in single or double quotes. To escape a quote character in a literal value, precede it with a backslash (\).
Examples:
application-name TradingAdapter
application-name "Trading Adapter"
Configuration variables
This section covers defining, undefining, and referencing custom variables in configuration files.
Defining a variable
You define a configuration variable using the define configuration directive:
You can set a configuration variable to a literal, a configuration variable, an environment variable, or an expression.
By convention, variable names in Caplin configuration files are in upper case.
The scope of the variable is the configuration file in which it is defined, and any files that include this configuration file using the include-file directive (see Including multiple configuration files).
For example, to define the variable HTTP_PORT with the value 8080, use the configuration below:
define HTTP_PORT 8080
Undefining a variable
You undefine a configuration variable using the undefine configuration directive:
For example, to undefine the variable HTTP_PORT, use the configuration below:
undefine HTTP_PORT
Referencing a variable
The following notation shows how a variable is used to set the value of a configuration item.
When a variable is used to set the value of a configuration item, the name of the variable must be enclosed in curly braces and preceded by the $ character. If an undefined variable is used in the configuration, the value of the configuration item is set to an empty string and a warning is logged to the console and log file.
The following example defines a variable, and sets the value of a configuration item using that variable.
define HTTP_PORT 8080
http-port ${HTTP_PORT}
In this case, the value of Liberator’s http-port configuration item is set to 8080 (using the HTTP_PORT variable).
Variables can also be concatenated with surrounding text and other variables.
define HTTP_PORT 8080
http-port 1${HTTP_PORT}
In this case, the value of Liberator’s http-port configuration item is set to 18080.
Predefined variables
The following variables are predefined by DataSource libraries:
| Variable | Description |
|---|---|
|
The name of the DataSource application, as defined by |
|
Current configuration directory. The full path to the directory containing the configuration file. |
|
The home directory of the process owner. |
|
The name of the host machine |
|
The major version of the DataSource library (for example, |
|
The minor version of the DataSource library (for example, |
|
The name of the operating system that the DataSource library was
built for. Examples are |
|
The GNU configuration triplet (of the form cpu-manufacturer-operating_system). Examples are |
| Variable | Description |
|---|---|
|
The name of the DataSource application, as defined by |
|
Current configuration directory. The full path to the directory containing the configuration file. |
|
The architecture of the Java virtual machine: |
|
The name of the host machine, as returned by |
|
The name of the host operating system: |
Environment variables
This section describes how to reference an environment variable in a DataSource configuration file.
Environment variables are referenced using the syntax below:
The name of the environment variable must be preceded by the text ENV: and enclosed in curly braces, and the braces must be preceded by the $ character.
Environment variables that contain spaces are interpolated as a single value, not multiple values. As such, this syntax is not suitable for use with configuration items that accept multiple, space-separated values. For example, given a shell environment variable of USERS that is set to Alice Bob, the configuration setting rttp-log-users ${ENV:USERS} would be interpolated as rttp-log-users "Alice Bob", not rttp-log-users "Alice" "Bob" as rttp-log-users expects.
If the environment variable has not been defined, the syntax returns an empty string and a warning is logged to the console and log file.
The following example sets the runtime-user configuration item to the value of the USER environment variable:
runtime-user ${ENV:USER}
Expressions
This section describes how to use evaluated expressions in a configuration file. Expressions enable you to set a configuration item to a calculated value.
The expression macro uses the syntax below:
Each value in the expression can be a literal, a configuration variable, an environment variable, or a single-parameter configuration item.
All operators work with numeric operands, but the addition operator (+) is the only operator that works with string operands. If the subtraction (-), division (/), and multiplication (*) operators are used with a string operand, the DataSource raises an error.
When the addition operator (+) is used with string operands it concatenates the strings. If the addition operator is used with a numeric operand and a string operand, then the numeric operand is implicitly converted to a string.
Operators observe the rules of precedence described in the table below. You can use parentheses to group parts of an expression (not shown in the syntax diagram above).
| Operators | Description |
|---|---|
|
Parentheses (grouping) |
|
Multiplication and division |
|
Addition and subtraction |
Examples
The example below sets http-port to the sum of the PORT_BASE variable and 80 (=20080):
define PORT_BASE 20000
http-port ${EVAL:${PORT_BASE} + 80}
The example below sets https-port to the sum of the http-port configuration item and 1 (=8081):
http-port 8080
https-port ${EVAL:@http-port + 1}
Conditional flow control
DataSource libraries support conditional flow control constructions that have
if/else/elseif/endif statements in the DataSource configuration.
The if statement uses the syntax below:
if conditional ... elseif conditional (1) ... else (2) ... endif
| 1 | Optional (0..n) |
| 2 | Optional (0..1) |
Simple conditionals
A simple conditional runs an application-defined test against a value.
The value to be tested can be a literal, a configuration variable, an environment variable, or an expression.
DataSource applications support three tests by default:
| Test | Description |
|---|---|
|
Returns TRUE if the test-value is equal to the value of the |
|
Returns TRUE if the test-value is equal to the hostname. |
|
Returns TRUE if the test-value is equal to the name of the operating system the application was compiled for. The test supports the following operating system names: |
| Test | Description |
|---|---|
|
Returns TRUE if the test-value is equal to the value of the |
|
Returns TRUE if the test-value is equal to the hostname as returned by the method |
|
Returns TRUE if the test-value is equal to the name of the host operating system. The test supports the following operating system names: |
Additional simple conditional tests can be added to a DataSource application using the ds_config_set_test() function in the C DataSource API’s Configuration API. Currently no Caplin DataSource applications do so.
Example simple conditional
In the example below, the C DataSource configuration item jvm-location is set to a Windows path on a Windows host and to a Linux path on a Linux host:
if os win64 # Microsoft Windows with Oracle Java 1.8.0 jvm-location "C:\Program Files\Java\jre1.8.0_131\bin\server\jvm.dll" elseif os linux-EL6-gnu # Red Hat Enterprise Linux 6 with Oracle Java 1.8.0 jvm-location "/usr/java/jdk1.8.0_131/jre/lib/amd64/server/libjvm.so" endif
|
The Caplin Deployment Framework automatically sets a jvm-location ${JVM_LOCATION}
|
Complex conditionals
A complex conditional consists of a single value or an expression that evaluates to a boolean value.
The syntax for a complex conditional is illustrated below:
When writing a complex conditional with more than two operators, you can use parenthesis to control operator precedence (not shown in the syntax diagram above).
The following operators can be used inside a complex conditional:
| Operators | Description |
|---|---|
|
Parenthesis (grouping) |
|
Multiplication and division |
|
Addition and subtraction |
|
Comparisons: less than, less than or equal to, more than, more than or equal to. |
|
Comparisons: equal to, not equal to |
|
Logical AND ( |
|
Logical OR ( |
If a complex conditional does not evaluate to a boolean value, the result is automatically converted to a boolean according to the rules below:
| Type | Value | Examples | Effective boolean value |
|---|---|---|---|
Numeric |
> 0 |
|
|
Numeric |
< 0 |
|
|
Numeric |
0 |
|
|
String |
Empty |
|
|
String |
Not empty |
|
|
Example complex conditionals
In the example below, https-port is set to 443 if the boolean configuration item https-enable has the value true:
if @https-enable
https-port 443
endif
In the example below, https-port is set to 443 if the configuration item http-port is set to the standard HTTP port of 80:
if @http-port == 80
https-port 443
endif
In the example below, https-enable is set to true and https-port is set to 443 if the configuration variable HTTPS has the value 1 and the configuration variable HOST_NAME has the value host1 or host2:
if ${HTTPS} == 1 and (${HOST_NAME} == host1 or ${HOST_NAME} == host2) https-enable https-port 443 endif
Including multiple configuration files
The include-file directive specifies that another configuration file has configuration settings for the DataSource application. The * wild card can be used in the relative path of the included file to include multiple configuration files.
The example include-file directive below includes all files with a .conf extension in the FX directory:
include-file FX/*.conf
Relative file paths are searched for on the configuration include path. To add a path to the configuration include path, use include-dir:
include-dir "${ccd}"
Obtaining configuration from a webserver
The configuration for a DataSource application can be obtained from a web server by specifying the URL of the required configuration file in an include-file directive.
In the example below, a configuration file is downloaded from URL http://configurationserver/rttpd.conf:
include-file http://configurationserver/rttpd.conf
Only the HTTP protocol is supported.
By default, the file is retrieved using the wget command, which must be installed on the DataSource host. To use a different command to wget, see Changing the download command below.
Changing the download command
The default command used to download configuration files from webservers is wget -nv -O -. To use a different download command, specify a new value for the http-download-command configuration directive.
You can use any download utility that accepts the URL to download as its first parameter and writes the downloaded resource to STDOUT.
When specifying the name of the command to http-download-command, include any options required to suppress other output to STDOUT, such as progress indicators, error messages, and status codes.
For example, to set curl as the download utility, use the configuration below:
http-download-command "curl -s"
Note that curl’s silent option (-s) has been specified, which stops curl from writing progress and error messages to STDOUT.