Openmetrics

Capabilities

Capability	Support	Comment
Consume Openmetrics values		This feature is currently not supported. In case of interest please contact us at info@sdn.ch
Provide Openmetrics values		Most metrics provided by modules are available. The list of supported metrics is defined per module and detailed below. Refer to Modules capabilities for the complete set of supported modules

Concept

Overview

OpenMetrics is an open standard designed for exposing and exchanging metrics in a unified text format. It is directly inspired by the Prometheus exposition format but aims to be interoperable across different monitoring systems. The standard allows applications and services to expose their metrics in a universal way, without being tied to a single tool. It introduces improvements such as an explicit end-of-stream marker (# EOF) and clearer standardization of metric types. official source

Fetching metrics

By default every modules are using internal port 9100 to expose the metrics. This port is not exposed on external of the stack by default. The network need so to be configured to expose the port externally with a solution depending of your orchestrator.

The internal port can be set via an environment variable. See Modules capabilities for supported modules

Buckets

If not specified otherwise the histogram default bucket configuration follow this guideline

• 0.001 seconds

• 0.05 seconds

• 0.1 seconds

• 0.25 seconds

• 0.5 seconds

• 0.75 seconds

• 1.0 seconds

• 5.0 seconds

• 10.0 seconds

• 20.0 seconds

• 60.0 seconds

• 300.0 seconds

• inf

Available metrics

default-metrics

See Modules capabilities for the list of module who are supporting openmetrics export

Supervision Rabbit

Metric	Type	Description
rabbit_nacked_published_messages_total	counter	Number of messages published to RabbitMQ but not acknowledged by the server.
rabbit_connections	gauge	Current number of connections to the RabbitMQ server.
rabbit_acknowledged_messages_total	counter	Total number of messages acknowledged by RabbitMQ.
rabbit_channels	gauge	Current number of channels opened on the RabbitMQ server.
rabbit_published_messages_total	counter	Total number of messages published to RabbitMQ.
rabbit_unrouted_published_messages_total	counter	Number of published messages that could not be routed to any queue.
rabbit_consumed_messages_total	counter	Total number of messages consumed from RabbitMQ.

osp-alarms

Execution Time

Metric	Labels	Type	Description
pre_insert_rule_execution_seconds	ItemId	histogram	Pre-insert rule execution time in seconds
archive_execution_seconds	None	histogram	Archive execution time in seconds
history_execution_seconds	None	histogram	History execution time in seconds
alarm_action_execution_seconds	None	histogram	Alarm action execution time in seconds
insertion_execution_seconds	None	histogram	Time to insert into the buffer
processing_execution_seconds	step	histogram	Processing time per step in seconds

Buffer & Batch Metrics

Metric	Labels	Type	Description
insertion_count_total	None	counter	Total alarms inserted into the buffer
buffer_size	None	gauge	Buffer size before processing (insertion)
processing_batch_size	batch total_batch	gauge	Size of processing batch (insertion)

Processing Execution Created Timestamps

Metric	Labels	Type	Description
archive_execution_seconds_created	None	gauge	Timestamp of archive execution
history_execution_seconds_created	None	gauge	Timestamp of history execution
insertion_count_created	None	gauge	Timestamp of insertion count
insertion_execution_seconds_created	None	gauge	Timestamp of insertion execution
processing_execution_seconds_created	buffer total_batch batch pre-alarms alarm check aggregate post-alarms write	gauge	Timestamp of processing execution per step

Metric	Labels	Type	Description
action_rule_execution_seconds	itemId	histogram	Duration in seconds of the execution of each action rule, labeled by itemId.

alarm_action_execution_seconds

Metric	Labels	Type	Description
alarm_action_execution_seconds	acknowledge, escalate, lock, unlock, clear, tag, untag, journal, edit, create, insert	histogram	Duration in seconds of each alarm action execution, labeled by the corresponding action.

Front view

Metric	Labels	Type	Description
live_change_execution_seconds	step=find	histogram	Duration in seconds of the find operation on the database.
live_change_execution_seconds	step=compute	histogram	Duration in seconds to compute the difference between two alarm states, publishing only the changes to the front-end.
live_change_execution_seconds	filter	histogram	Label representing each applied filter.

An example for a Grafana dashboard : Dashboard

osp-collections

Write operations

• collection_write_request_execution_seconds: metrics for write operations on a collection. Comes with the following labels:

  • requestType: type of request. One of insert, updateDiff, updateDoc or delete

  • schemaId: ItemId of the concern collection schema

Read operations

• collection_read_request_execution_seconds: metrics for read operations on a collection. Comes with the following labels:

  • requestType: type of request. One of getRecordWithFilter, getRecordWithCustomFilter, listRecords or listRecordsCustomFilter

  • schemaId: ItemId of the concern collection schema

  • filter: Filter used for the query

Collections history

• collection_history_request_execution_seconds: metrics for read operations on a collection history. Comes with the following labels:

  • requestType: type of request. One of getHistoryVersions or historyGoBackUntil

  • schemaId: ItemId of the concern collection schema

• collection_history_adapter_execution_seconds: metrics for history adapter process (enable/disable history for a collection).

Rights generation

• collection_rights_generation_execution_seconds: metrics for collections rights generation.

osp-configuration-dispatcher

Module state supervision

Warning

OpenMetrics only allow a dedicated character set a-zA-Z0-9_; any other character is replaced with _.

Metric	Type	Description
module_id + _heartbeat_counter_total	counter	Number of messages published to RabbitMQ but not acknowledged by the server.
module_id + _heartbeat_state	gauge	Each module communicates with the dispatcher via a heartbeat. This metric reports the module’s state as computed by the dispatcher: - 0.0 = RUNNING - 1.0 = RUNNING_WITH_ERROR - 2.0 = CONFIG_OUTDATED - 3.0 = UNREACHABLE - 4.0 = REMOVED

Default Aggregator Execution Time Metrics

Warning

These metrics can be modified at any time and expose the internal state and behavior of the system, which is useful for debugging the stack.

Metric	Labels	Description
git_hook_api_seconds	gauge	Execution time in seconds for the git hook API. Labels: step
executor_file_hierarchy_change_seconds	gauge	File hierarchy session execution time for changes
get_values_seconds	gauge	Execution time in seconds for getting values
aggregator_main_aggregate_seconds	gauge	Aggregate execution time. Labels: module
default_aggregator_seconds	gauge	Default aggregator execution time in seconds. Labels: module, step
executor_aggregation_total_async_task	gauge	Aggregation execution time for every async task
executor_aggregation_async_task	gauge	Aggregation execution time for async tasks. Labels: module, aggregator

osp-scripts

Metric	Type	Labels	Description
script_execution_seconds	histogram	name: ItemId of the executed script	The time to execute the script.
script_backpressure_total	counter	name: ItemId of the script, or <script>::<trigger> when useTriggerAsId is enabled (see Priority scheduling) strategy: backpressure strategy applied	Number of script executions dropped due to backpressure.
script_enqueued_total	counter	name: ItemId of the script, or <script>::<trigger> when useTriggerAsId is enabled (see Priority scheduling)	Number of script executions accepted in the queue.
script_expired_total	counter	name: ItemId of the script, or <script>::<trigger> when useTriggerAsId is enabled (see Priority scheduling)	Number of script executions discarded because their validity expired while waiting in the queue.
script_queue_wait_seconds	histogram	name: ItemId of the script, or <script>::<trigger> when useTriggerAsId is enabled (see Priority scheduling)	Time a script execution waited in the queue before being dispatched.
script_pending_executions	gauge	priority: priority of the waiting executions	Number of script executions currently waiting to be dispatched, per priority. Use sum(script_pending_executions) for the total.
script_running_executions	gauge	None	Number of script executions currently running, all scripts taken together.