Timestamp resolution
From version 7.1 of the Caplin Platform, DataSource timestamps are recorded to a resolution of one microsecond or finer. This change aligns the Caplin Platform with the financial industry’s use of high-resolution timestamps to satisfy regulatory requirements for best execution, such as the revision to the Markets in Financial Instruments Directive due to come into effect on 3 January 2018 (MiFID II).
For more information on MiFID II’s requirements for the accuracy and resolution of business clocks, see ESMA’s RTS 25 and RTS 25: Annex 1.
Requirements
Caplin Platform supports high-resolution timestamps from version 7.1.
High-resolution timestamps are supported by the following operating systems:
-
Red Hat Enterprise Linux 6+
-
Windows 7+
-
Apple macOS 10.12 (Sierra) and higher
| Caplin Platform is supported on Microsoft Windows and Apple macOS for development use only. Caplin Platform components are tested on the latest version of macOS available at the time of testing (see the release notes for the specific version). |
To measure latency with high-resolution timestamps, all components in the latency chain must be 7.1 or higher. Measuring latency across different versions of Caplin Platform components is not supported.
Breaking changes in Caplin Platform 7.1
The following breaking changes relating to timestamps were introduced with Caplin Platform 7.1.
New method and function signatures
The following DataSource for Java methods now take a java.time.Instant object as a parameter instead of a long:
-
RecordMessage.setInitialLatencyChainTime(java.time.Instant instant) -
RecordMessage.addLatencyChainPoint(String pointName, java.time.Instant instant)
The following DataSource for C (DSDK) functions now take a timespec struct instead of an int64_t:
-
void ds_add_latency_chain(ds_data_t* dsdata, char suffix, struct timespec* tp) -
void ds_add_latency_chain2(ds_data_t* dsdata, char suffix, struct timespec tp, char suffix2, struct timespec* tp2)
New packet-log binary format
The binary format of packet logs has changed in Caplin Platform 7.1 to accommodate microsecond timestamps. Caplin Platform 7.1 packet logs can only be read by logcat 7.1 and later.
New microsecond timestamp format
Timestamps in log files and logcat output now have a fractional microsecond component. This may break log parsers that expect timestamps to have a fractional component fixed at three figures.
| DataSource Version | Example |
|---|---|
>= 7.1 |
|
< 7.1 |
|
DataSource for Java event logs use a microsecond formatted timestamp, but log to millisecond precision.
Timestamp accuracy in Caplin Platform 7.1
The accuracy of timestamps recorded by Caplin Platform can only be as accurate as the timestamps supplied to Caplin Platform by the operating system.
To accurately measure latency in the Caplin Platform, all Caplin Platform servers must be synchronised with the same time source.
For Caplin Platform timestamps to be comparable with backend timestamps, Caplin Platform servers must be synchronised with the same time source as backend servers. Backend servers that are compliant with MiFID II requirements for business clock accuracy will be synchronised with a time source that is directly traceable to Coordinated Universal Time (UTC). For an introduction to high-resolution clock synchronisation in the context of MiFID II, see Time Traceability for the Financial Sector by the UK’s National Physical Laboratory.
Timestamp resolution in Caplin Platform 7.1
Timestamp resolution has been increased to microsecond granularity or finer.
DataSource for Java 7.1
The table below documents timestamp granularity in DataSource for Java (DS4J) 7.1.
| Timestamp Granularity | |
|---|---|
Packet log |
Microsecond |
Event log |
Millisecond (see note below) |
Latency chaining |
Nanosecond |
The DataSource for Java event log is written using the java.util.logging package, which does not support high-resolution timestamps.
DataSource for Java uses Caplin’s open-source Nanotime JNI extension for Java 8 to provide high-resolution timestamps for packet logs. For more information, see the Nanotime repository on Caplin’s GitHub site.
The method signatures for RecordMessage.setInitialLatencyChainTime and RecordMessage.addLatencyChainPoint have changed. See Breaking changes, above.
DataSource for C (DSDK) 7.1
The table below documents timestamp granularity in DataSource for C 7.1 and all applications based on DataSource for C 7.1 (Liberator 7.1 and Transformer 7.1).
| Timestamp Granularity | |
|---|---|
Packet log |
Microsecond |
Event log |
Microsecond |
Latency chaining |
Nanosecond |
The function signatures for ds_add_latency_chain and ds_add_latency_chain2 have changed. See Breaking changes, above.
Liberator 7.1
See DataSource for C (DSDK) 7.1 above.
For information on the precision of Liberator’s optional timestamp field, see the documentation for the global timestamp-field configuration item.
Known issues
Java 8 does not include native support for high precision timestamps. To provide high precision timestamps in Java 8, the Java DataSource API includes Caplin’s open-source Nanotime native library.
By default, the Caplin Nanotime native library is extracted to, and run from, the /tmp directory. If the /tmp filesystem has been mounted with the noexec flag, then Java cannot link to the Caplin Nanotime library. To resolve this, edit the java command used to launch the DataSource application and set the Java system property java.io.tmpdir to a directory with executable permissions:
$ java -Djava.io.tmpdir=/home/caplin/tmp existing_arguments
See also: