Use timers in DataSource applications

Here we explain what timers are used for in a DataSource application, what they’re for, and how to configure them.

Timing out requests to DataSource peers

The global request timeout for data services

This facility isn’t available to Java-based DataSource applications, which don’t support data services.

If your DataSource application obtains data from peers through data services (which is the recommended way to do it), requests sent to a data service, such as a subscription to a subject, are timed out. If the DataSource peers providing the service don’t respond within the timeout period, the DataSource application assumes the requested item isn’t available, so the subscription to it is discarded.

This timeout is defined by the configuration item service-request-timeout, which applies to all data services defined in the DataSource application.

If you don’t explicitly define a value for service-request-timeout in your application’s configuration, the default timeout of 10 seconds is applied to all the application’s data service requests.

Timing out requests to individual data services

This facility isn’t available to Java-based DataSource applications, which don’t support data services.

You can override the data services global service-request-timeout for a selected data service. Just add a request-timeout option to the add-data-service configuration item for the data serice:

service-request-timeout 12.0

add-data-service
   service-name    fx-prices
   include-pattern ^/FX/
#
# Override service-request-timeout
# for the fx-prices service only.
#
   request-timeout 9.0
   ...
   add-source-group
   ...
   end-source-group
end-data-service

If you don’t want to time out the responses from a particular data service, set the service’s request-timeout option to -1:

service-request-timeout 12.0

add-data-service
   service-name    fx-prices
   include-pattern ^/FX/
#
# Disable data service request timeouts
# for the fx-prices service only
#
   request-timeout -1
   ...
   add-source-group
   ...
   end-source-group
end-data-service
If you disable the request timeout on a data service, make sure that you’ve defined a request timeout for each of the peers that supply data for the service, otherwise the DataSource application could wait indefinitely for the service to satisfy a request. See Timing out requests made to individual peers below.

Retrying a request to a data service

This facility isn’t available to Java-based DataSource applications, which don’t support data services.

If a DataSource application finds that none of the peers defined by the add-priority options in a data service’s source group have responded to a request, the DataSource application waits for a configurable time before reissuing the request to the group. It repeats this sequence until the master timeout for the service, as defined by service-request-timeout, or the request-timeout option of add-data-service, expires. You set this retry timer with the retry-time option of the add-source-group configuration item:

service-request-timeout 12.0

add-data-service
   service-name    fx-prices
   include-pattern ^/FX/
#
# Override service-request-timeout
# for the fx-prices service only.
#
   request-timeout 9.0
   ...
   add-source-group
      required   TRUE
#
# If none of this service's peers
# respond to a request, wait 3.0 seconds
# before retrying.
#
      retry-time 3.0
      add-priority
         remote-label fxpriceadapter1
      end-priority
      add-priority
         remote-label fxpriceadapter2
      end-priority
   end-source-group
end-data-service

What happens if I don’t explicitly specify any timers for my data services?

The default timer settings would apply. The default value of service-request-timeout is 10 seconds, so all data service requests requests would time out after 10 seconds. Since the default setting of retry-time is 30 seconds, there wouldn’t be any retries, because the service request timeout would take effect well before the DataSource application could reissue the request to the group.

Timing out requests made to individual peers

This facility isn’t available to Java-based DataSource applications.

The data service request timeouts described above apply to all the peers that belong to a data service. You can also specify a timeout on response to requests sent to a particular peer. This timeout applies to requests sent to the peer from any of the DataSource application’s data services.

Firstly, there’s a global request timeout called source-request-timeout that applies to all the DataSource application’s peers. If you don’t explicitly define it, peer level request timeouts aren’t enabled.

#
# Peers configuration
#
#
# Set a timeout on the response to
# any request sent to any peer.
#
source-request-timeout 3.0
add-peer
   remote-name fxpriceadapter1
   ...
end-peer

add-peer
   remote-name fxpriceadapter2
   ...
end-peer

#
# Data services configuration
#
service-request-timeout 12.0

add-data-service
   service-name    fx-prices
   include-pattern ^/FX/
   request-timeout 9.0
   ...
   add-source-group
      required   TRUE
      retry-time 3.0
      add-priority
         remote-label fxpriceadapter1
      end-priority
      add-priority
         remote-label fxpriceadapter2
      end-priority
   end-source-group
end-data-service

You can also set a timeout on the request reponses made by a specific peer, using the request-timeout option of add-peer. This overrides the source-request-timeout setting for that peer only:

#
# Peers configuration
#
#
# Set a timeout on the response to
# any request sent to any peer.
#
source-request-timeout 3.0
add-peer
   remote-name fxpriceadapter1
#
# Set a different request response
# timeout for the fxpriceadapter1
# peer only.
#
   request-timeout 2.0
   ...
end-peer

add-peer
   remote-name fxpriceadapter2
   ...
end-peer

#
# Data services configuration
#
service-request-timeout 12.0

add-data-service
   service-name    fx-prices
   include-pattern ^/FX/
   request-timeout 9.0
   ...
   add-source-group
      required   TRUE
      retry-time 3.0
      add-priority
         remote-label fxpriceadapter1
      end-priority
      add-priority
         remote-label fxpriceadapter2
      end-priority
   end-source-group
end-data-service

Timing out connections to DataSource peers

When a DataSource application requests to connect to a DataSource peer, the peer may not exist or there might be no current network route to the peer. The operating system will attempt to connect for a time that’s dependent on various operating system parameters; it could typically be up to 4 minutes before the OS gives up and returns a connection error to the application. This delay’s usually too long for a typical Caplin Platform installation, so the DataSource application overrides it, using the connect-timeout option of the add-peer configuration item:

#
# Peers configuration
#
#
add-peer
   remote-name fxpriceadapter1
#
# Set a timeout (in seconds) on the
# connection request for this peer.
#
   connect-timeout 5
   ...
end-peer

connect-timeout has a default value of 10 seconds, so if you don’t explicitly set it in the add-peer configuration, connection requests to that peer will time out after 10 seconds - rather less than the typical OS default!

You’d typically want to define connect-timeouts in the configuration of an Integration Adapter that connects to one or more Liberators and/or Transformers, setting the timeout in the add-peer items that define the Liberator and Transformer connections.

Note that if you’ve set up a list of alternative peers in the add-peer configuration (see the addr and port options), and the timeout expires when attempting to connect to one of the peers, the DataSource application attempts to fail over to the next peer connection defined in the addr option’s list and applies the timeout again.

Other data service timeouts

The cleanup-stale-timeout and discard-timeout configuration items aren’t available to Java-based DataSource applications, which don’t support data services.

In Timing out requests to DataSource peers, we discussed various timers that apply to requests made to peers through data services. There are also a couple of other timers that apply to data services: cleanup-stale-timeout and discard-timeout.

  • Define cleanup-stale-timeout when you want the DataSource application to delete stale objects from its cache.

  • Define the discard-timeout option of add-data-service when you want the DataSource application to discard the data for a subscribed subject once the last client application has unsubscribed from the subject. This option is particularly relevant to Liberator.

See also: