Cache and Validate Intraday Data

Caching Data

max-age

Maximum amount of time (in days) to keep caching a user-requested symbol after it was last discarded.

Default value: 10

cache-directory

Directory to be used to store the cached charting data files. ‘0’ (zero) is representative of the root directory of the Transformer core application.

Default value: [current directory]

Example:

cache-directory       0/or/charts

Validating Data

Incoming updates can be checked before being cached in order to ascertain whether they are needed for charting purposes or not.

For example, you can check a last trade update is within a certain percentage of the previous value (those that are not are taken as erroneous trades), or reject a last trade price that has a value of 0 (which indicates start-of-day initialisation). All fields that are to be requested by the Charting blade must be specified.

The specific instruments required are identified using the configuration parameter 'charts-prefix' (see here).

add-fieldset

Defines how a Transformer handles a set of instruments and which fields should be cached for that type. See here for examples showing add-fieldset in use.

There are two ways of matching a fieldset to an instrument: using a set of regular expressions that match the subject of the instrument (using include-pattern and exclude-pattern), or using the contents of the "RECORDTYPE" field.

The RECORDTYPE field is a field available on RICs (Refinitiv Instrument Code) requested from Refinitiv that identifies the asset class and type of instrument, see here for the available values of RECORDTYPE.

The entry must use the following format:

add-fieldset
      exclude-pattern
      include-pattern
      recordtype       [value]
      add-field        [value]   [value]
      cache-field      [value]
      rules            [value]
end-fieldset

The options in this entry are:

Parameter Type Default Description

recordtype

int

[no default]

The code identifying the record type. See here for a list of codes.

add-field

string array

[no default]

Space separated list of fields that should be requested and cached. Takes 5 parameters:

  • field — The name of the field

  • size — The size of the field in characters (used in calculating the required disk storage). If a field is bigger than this it will be truncated.

  • flags — Flags to specify which type of caching should be used. Options are listed here.

  • source-field — The name of the field that is the source of the data.

  • group-code — The name of the group in which this field is included.

cache-field

string array

[no default]

Space separated list of fields holding details such as the instrument’s full name and permissioning information.

rules

string array

[no default]

List of rules for validating incoming values. The rules should give a result ZERO or NON-ZERO. If

NON-ZERO then the update is rejected—see here for using RPN rules.

exclude-pattern

 — 

 — 

Regex, first match

include-pattern

 — 

 — 

Regex, first match

Configuring Periods

In terms of the periods, this is configured using the add-interval directive.

The options for the add-interval configuration item are:

Parameter Type Default Description

name

string array

-

-

cache-size

int

-

-

cache-interval

int

-

Interval, in seconds, between data-points. Specify an interval of 0 to cache every change in value.

historic-routing

[from subject] [to subject]

-

-

Example 1

In this example, all requests for 1m (one minute) will be satisfied by the intraday cache within the Charting blade. The cache will contain 1000 data points and will be updated every 60 seconds.

add-interval
    name                 1m
    cache-interval       60
    cache-size           1000
    historic-routing     %1 file
end-interval

Example 2

Here, any requests to chart an instrument under the /I directory for 1d (one day) will be forwarded onto a datasource request. So for example, when requesting /I/VOD.L with a 1d period from the charting blade, a request will be made for /TS1/dVOD.Ld to obtain the charting data.

add-interval
   name                  1d
   historic-routing      /I/%1  /TS1/d
end-interval