Charting: cache and validate intraday data

This page provides information on the configuration items for caching intraday data, including options for conditional caching based on validation rules.

max-age

Maximum amount of time (in days) to keep caching a user-requested symbol after it was last discarded.

Default value: 10

cache-directory

Directory to store the cached charting data files. In the default configuration for the Charting Module, these files are stored in a directory charts relative to Transformer’s working directory. For a Transformer deployed under a Caplin Deployment Framework, this value resolves to servers/Transformer/charts.

Default value: ${ccd}/charts

Example: cache-directory ${ccd}/var/charts

add-fieldset

Defines how a Transformer handles a set of instruments and which fields should be cached for that type. See here for examples showing add-fieldset in use.

There are two ways of matching a fieldset to an instrument: using a set of regular expressions that match the subject of the instrument (using include-pattern and exclude-pattern), or using the contents of the "RECORDTYPE" field.

The RECORDTYPE field is a field available on RICs (Refinitiv Instrument Code) requested from Refinitiv that identifies the asset class and type of instrument. See record type codes.

Syntax:

add-fieldset
    exclude-pattern  [value]
    include-pattern  [value]
    recordtype       [value]
    add-field        [field] [size] [flags] [source-field] [group-code]
    cache-field      [value]
    rules            [value]
    pipeline-file    [value]
    pipeline-func    [value]
end-fieldset
Option Type Default Description

recordtype

int

 — 

The code identifying the record type. See record type codes.

add-field

string array

 — 

A calculated field that is based on the value of a field in the update (source-field, below).

You can define multiple add-field options per add-fieldset stanza.

This option takes 5 parameters:

  • field — The name of the field

  • size — The size of the field in characters (used in calculating the required disk storage). If a field is bigger than this it will be truncated.

  • flags — Flags to specify which type of caching should be used. For a list of valid flags, see Charting: add-fieldset options.

  • source-field — The name of the field that is the source of the data.

  • group-code — The name of the group in which this field is included.

cache-field

string array

 — 

Space-separated list of fields holding details such as the instrument’s full name and permissioning information.

exclude-pattern

string

 — 

Regex (first match)

include-pattern

string

 — 

Regex (first match)

rules

string array

 — 

A Reverse Polish notation (RPN) expression to validate an incoming price update before the fieldset caches it. If the RPN expression returns a non-zero value, the price update is not cached.

You can define multiple rules options per add-fieldset stanza.

You can also write validation rules using Lua and JavaScript. See the pipeline-file and pipeline-func options below.

For more information on writing RPN expressions for rules, see Charting: add-fieldset option values.

For example validation rules in RPN and Lua, see Charting: add-fieldset examples.

Do not include both rules and pipeline-* options in the same add-fieldset stanza.

pipeline-file

string

 — 

Name of a Transformer pipeline script file (Lua or JavaScript) used to validate an incoming price update before the fieldset caches it. Requires the pipeline-func option.

For example validation rules in RPN and Lua, see Charting: add-fieldset examples.

Do not include both rules and pipeline-* options in the same add-fieldset stanza.

pipeline-func

string

 — 

Name a Transformer pipeline function (Lua or JavaScript), used to validate an incoming price update before the fieldset caches it. Requires the pipeline-file option.

For example validation rules in RPN and Lua, see Charting: add-fieldset examples.

Do not include both rules and pipeline-* options in the same add-fieldset stanza.

Defining the instruments to cache

The add-fieldset stanza defines the fields to cache for an instrument or set of instruments.

To define the instruments that an add-fieldset stanza applies to, use the following three configuration options:

  • recordtype: the instrument’s record-type

  • include-pattern: the instrument’s RTTP subject

  • exclude-pattern: the RTTP subject of the instrument

Validating instrument values

You can refine the applicability of an add-fieldset stanza by specifying validation rules for specific fields in an update. If the validation rules are not met, then the fields defined in the add-fieldset stanza are not cached.

For example, you could reject updates in which the value of a specific field is zero or outside a percentage difference from the previously cached value.

To define validation rules for an add-fieldset stanza, use one of two sets of configuration options below:

  • rules: a reverse Polish notation (RPN) expression.

  • pipeline-func and pipeline-file: the name of a pipeline function and the name of the script file in which the function is defined.

For example validation rules in RPN and Lua, see Charting: add-fieldset examples.

add-interval

add-interval defines a caching period.

Syntax:

add-interval
    name                 [string]
    cache-interval       [integer]
    cache-size           [integer]
    historic-routing     [string] [string]
end-interval
Option Type Default Description

name

string

-

-

cache-size

integer

-

Number of data points in the cache

cache-interval

integer

-

Interval, in seconds, between data-points. Specify an interval of 0 to cache every change in value.

historic-routing

string array

-

-

Examples

In the example below, all requests for 1m (one minute) will be satisfied by the intraday cache within the Charting blade. The cache will contain 1000 data points and will be updated every 60 seconds.

add-interval
    name                 1m
    cache-interval       60
    cache-size           1000
    historic-routing     %1 file
end-interval

In the example below, any requests to chart an instrument under the /I directory for 1d (one day) will be forwarded onto a datasource request. So for example, when requesting /I/VOD.L with a 1d period from the charting blade, a request will be made for /TS1/dVOD.Ld to obtain the charting data.

add-interval
   name                  1d
   historic-routing      /I/%1  /TS1/d
end-interval