Configure threads for peer communication

A DataSource application uses threads to process the communication with its DataSource peers. Here we explain how to configure the allocation of threads to peers.

You can configure how your DataSource application allocates threads to peer connections by setting the global configuration item peer-thread-pool-size and the thread-name option of the add-peer configuration item. This helps to improve performance when the application is connected to several DataSource peers.

Specifying the thread-name option for a peer creates a named thread. You can then allocate the same named thread to other peers, so all the peers share this thread. For example:

add-peer
   remote-name  DataSource_1
   remote-label DataSource_1
   ...
   thread-name DS_Thread_1
end-peer

add-peer
   remote-name  DataSource_2
   remote-label DataSource_2
   ...
   thread-name DS_Thread_1
end-peer

In this example, the connections to DataSource_1 and DataSource_2 are handled on a single thread called DS_Thread_1.

Use peer-thread-pool-size to define a pool of threads that are shared between peer connections that don’t have a defined thread-name. For example:

peer-thread-pool-size 2
...
add-peer
   remote-name  DataSource_3
   remote-label DataSource_3
   ...
   # No thread-name defined
end-peer

add-peer
   remote-name  DataSource_4
   remote-label DataSource_4
   ...
   # No thread-name defined
end-peer

add-peer
   remote-name  DataSource_5
   remote-label DataSource_5
   ...
   # No thread-name defined
end-peer

In this example (continued from the previous one), a pool of two threads is allocated to three of the DataSource peer connections (DataSource_3, DataSource_4, and DataSource_5), so two of the connections must share a thread.

If you need to ensure that each peer connection that isn’t explicitly allocated a named thread nevertheless has its own unique thread (that is, not shared with any other connection), omit peer-thread-pool-size from the configuration.

If neither of the peer-thread-pool-size or thread-name configuration items is defined, the DataSource application creates a dedicated thread for each configured peer.

Consider allocating pool threads to DataSource peer connections with low update rates (typically 100 updates/sec or lower). In this case, there’s no real performance advantage from dedicating a separate thread to each connection.

If you expect a particular peer connection to be heavily loaded (typically when it must handle 10,000 or more updates/sec), we recommend that you allocate a dedicated thread to that connection. For example:

peer-thread-pool-size 30

...

add-peer
   remote-name  DataSource_1
   remote-label DataSource_1
   ...
   # No thread-name defined
end-peer
...
add-peer
   remote-name  DataSource_90
   remote-label DataSource_90
   ...
   # No thread-name defined
end-peer

add-peer
   remote-name  DataSource_91
   remote-label DataSource_91
   ...
   # This peer connection is heavily loaded
   # so we give it its own thread:
   thread-name DS_91_Thread
end-peer

If your DataSource application has just one DataSource peer to which you send, or which sends you, a high rate of updates, you may be able to improve performance by configuring more than one connection to the peer, and ensuring each connection uses a separate thread. The updates are then spread across multiple threads.

See also: