About the DataSource for C SDK

The DataSource for C SDK (Software Development Kit) is a library of C functions. It enables developers to create applications that can send financial market data (in the form of fields, structured records, pages, news headlines and news stories) to other DataSource-enabled applications, using Caplin's DataSource protocol.

This DataSource SDK supports Linux and Microsoft Windows platforms. DataSource SDKs for Java and .NET are also available.

The kit has the following components:

several APIs to perform DataSource-specific tasks within a program;
a configuration file to use as a template for any others you need;
a sample start-up script;
example programs created using the kit to demonstrate how it can be used;
Makefiles to simplify using the example applications.

What's new in DataSource for C 6.0

DataSource version 5 API
The DataSource version 5 API allows DataSource applications to be written in modular manner.When developing a DataSource application with this version of the API, you no longer have to code logic for managing DataSource peers.

Note: The DataSource for C version 4 API is deprecated. You are strongly recommended to develop new C-based DataSource applications using the version 5 API.

Note: In the rest of this document the DataSource version 5 API is referred to for short as the DSv5 API.
More DataSource peer connections.
DataSource applications built with this SDK can now handle up to 1023 DataSource peer connections (the previous limit was 63).

Peer connections are defined using the add-peer configuration item, so you can have up to 1023 such items in the configuration for a DataSource application. (See "Configuration: Datasource Peers" in the "Related Pages" topics.)

DataSource for C SDK features

The DataSource for C SDK incorporates a number of advanced features that enable you to add greater flexibility and reliability to the way your DataSource applications operate.

Operational features

Combined source and sink functionality
DataSource can act as both a source of data and a destination for data. Other applications to which it is connected which also use the DataSource protocol are known as DataSource peers.
Failover
When a connection is lost the side responsible for the connection will attempt to reconnect using a degrading retry algorithm. A number of DataSource peers can be configured to be used as alternative or failover sources.
Message queues
If a DataSource peer loses its connection to your DataSource, messages will be queued until the connection can be reestablished. The queue is flushed when a reconnection is successful. The length of the queue is configurable on a per-peer basis.
Multithreading
The DataSource library can be used with multi-threaded programs.
There are two considerations when using the DataSource library in a multi-threaded application.
- You may start your own threads which run independently of the DataSource event manager. When ds_send_data is called DataSource will insert this update into the queue. Using threads in this way there is no need to call ds_thread_init().
- You may also use multiple threads making use of the DataSource event manager. In this case you would need to call ds_thread_init() before calling ds_loop().
SSL support
DataSource is capable of communicating with its peers over SSL, providing an encrypted channel over which the data sources can publish their data.
Sending binary data
Any value can be encoded as binary in Base 64 and then sent to Caplin Datasource SDK and DataSource peers, where they can be converted back to their original state. See Adding binary data to objects.

Active request features

Active DataSources
An active DataSource is one that will keep track of which objects have been requested and send updates for those objects only.
Active source mapping
This allows you to define which DataSource peer or set of peers an active request for an object will be sent to. Wildcards are used to match object names which are then sent to the relevant DataSource peer.

Each mapping can have many DataSource peers or groups of peers defined – a group of peers is regarded as one peer and requests will be sent to them in a roundrobin fashion.
Non-active object flag for active sources
You can set an object to be non-active, whether the DataSource is an active source or not. Updates for non-active objects will always be sent to Caplin Datasource SDK even if they have not been requested by a particular RTTP client.

Active Cache

Active Cache enables a broadcast-style data feed to be turned into an active request-based one. This is done by caching all data in DataSource and handling requests for that data internally.

Active Cache allows a DataSource programmer to send data in a simple way instead of having to write a database or caching mechanism. Handling active requests and keeping track of what objects are being viewed is all handled by the DataSource library.

This feature is useful when one DataSource is feeding multiple DataSource peers and the peers are only interested in a small subset of the overall data.

Arbitrary page fields

DataSource SDK enables you to add arbitrary fields to pages. These can be used to add extra meta data to page objects.

Clearing cached entries

DataSource SDK enables you to clear cached type 3 entries for an object on receipt of a flag in an update for that object.

Advanced formatting functions

DataSource SDK gives you control over the format of fields, including specifying the number of decimal places.

Data handling features

Name mapping

DataSource can be configured to map names passed into it into a different format. This can either be used to simply make it a valid object name, or to create a complex directory structure of objects.

News functions

DataSource can contribute news objects, including news headlines and associated stories, and enable filtering of headlines based on either text or codes.

Status functions

DataSource SDK provides callbacks which provide information indicating the status of the connection to DataSource peers.

Logging configuration errors

DataSource SDK enables the developer to log errors which occur when configuring an application.

Enumerated data types for configuration options

For configuration options types INT and FLOAT and arrays of INT and FLOAT you can define enumerated data types to allow configuration files to become more readable.

An enumerated data type enables you to give a non-numeric and more meaningful name to a number. The names can then be used in the configuration file. For example if an INT option can take the values 1, 2 or 3, you can define 1 as "online", 2 as "offline" and 3 as "reconnecting".

Using condition testing when configuring

You can use condition testing configuration files. This enables you to change configuration settings dependent on factors such as differences in application or host.

Interfaces

To use the DataSource version 5 API, you must implement a number of interfaces. Each interface is defined as a C structure containing a set of function pointers. To implement such an interface, for each function pointer in the interface structure, code a function that matches the prototype defined in the structure and has the behaviour you require. then initialise an instance of structure to point to each of your function implementations.

The following example shouls how a simple interface is implemented:

ds5_connectionlistener_t example_connection_listener;
void  example_service_status(void *context, const char *service_name, int state);
void  example_peer_status(void *context, int peer_index, const char *peer_name, int state);
void setup_connection_listener()
{
   example_connection_listener.service_status = example_service_status;
   example_connection_listener.peer_status = example_peer_status;
   ds5_add_connectionlistener(&example_connection_listener, NULL);
}
void example_service_status(void *context, const char *service_name, int state)
{
   printf("example_service_status %s %d\n", service_name, state);
}
void example_peer_status(void *context, int peer_index, const char *peer_name, int state)
{
   printf("example_peer_status %s %d\n", peer_name, state);
}