Pipeline threads configuration

The code within a Transformer pipeline module can run in multiple threads. The configuration items described here enable you to monitor how the threads handle the throughput for the various pipelines, and to tune the threads accordingly.

Some of these configuration items relate to pipeline messages. A message is posted to a pipeline whenever the pipeline needs to handle a subscription request, subscription discard, data update, nodata event, UDP signal message, monitoring message, or pipeline action message (add new pipeline, shut down pipeline, and so on).

Here’s a list of all the configuration items for monitoring and tuning pipeline threads; to get more detail, just click on the item you’re interested in.

num-threads

num-threads Specifies the number of worker threads that the pipeline module spawns. Worker threads are used to process updates and nodata events.

When the first update for a subject arrives inside the pipeline module, one of the worker threads is chosen in a round-robin manner and the update is assigned to it for processing. All subsequent updates are then passed to the same update thread; this ensures that updates for a particular subject are always processed in the order they were received.

Syntax: num-threads <number-of-threads>

Type: integer

Default value: 1 thread

Minimum value: 1 thread

threads-latency-enable

threads-latency-enable when set to TRUE, enables the latency of pipeline threads to be calculated. The latency is a measure of how fast a thread processes pipeline messages.

Syntax: threads-latency-enable <boolean>

Type: boolean

Default value: FALSE (thread latency is not calculated)

threads-max-messages-to-process

threads-max-messages-to-process specifies the maximum number of pipeline messages to process. It applies to all threads and is a tuning mechanism to stop any one thread being locked up by handling too many incoming messages. A thread processes no more than threads-max-messages-to-process messages at a time before breaking out of its processing loop to allow other events (such as timed events) to be serviced.

The default setting should normally be sufficient.

Syntax: threads-max-messages-to-process <max-number-of-messages>

Type: integer

Default value: 1000 messages

threads-queue-report-logfile

threads-queue-report-logfile specifies the name of the file to which the sizes of the thread queues are reported.

Syntax: threads-queue-report-logfile <file-name>

Type: string

Default value: (no log file is created, so thread queue sizes aren’t logged)

threads-queue-report-period

threads-queue-report-period specifies the interval in seconds at which thread queue sizes are reported to the theads-queue-report-logfile.

Syntax: threads-queue-report-period <interval-in-seconds>

Type: float

Default value: 5.0 seconds

See also: