20-dns-syslog.conf
How to write a Stream Connector
Overview
Centreon Stream Connector is a feature introduced in Centreon 3.4.6. It allows one to export Centreon data (events and metrics) to an external storage or application such as ElasticSearch, Splunk, InfluxDB, files, etc.
In a Centreon platform, the component that carries information between the remote pollers and the Centreon central server is called Centreon Broker. This broker stores received data into the Centreon local storage: MariaDB and RRDtool.
The following diagram explains the transfer of collected data and insertion into storages:
The Stream Connector functionality is a new Centreon Broker output getting data from Centreon Broker Master (also known as Centreon Broker SQL) to aggregate and forward it to external storage:
This output loads a Lua script called a Stream Connector, which job is to handle, aggregate and enrich the data before forwarding it to the defined protocol:
Because it is an output of Centreon Broker, the principle of creating retention files upon interrupting external storage access is retained. In the same way, it is possible to filter input on the categories of flow to handle.
Requirements
To use the Centreon Stream connector functionality you need to update your Centreon platform to Centreon 3.4.6:
- Centreon Web >= 2.8.18
- Centreon Broker >= 3.0.13
- Lua >= 5.1.x
Creating a new Lua script
The complete technical documentation is available here. In this how-to, we will write two scripts:
- The first one, easy, that explains the basics of Stream Connectors. Its goal is to export data to a log file.
- The second one is more exigent for the reader ; it exports performance data to the TSDB InfluxDB but is easily adaptable to export to another TSDB.
Programming language
Centreon chose the Lua programming language to let you handle, aggregate and transfer data. Lua is a programming language that is easy to use. You can find more information with the Lua official documentation
Storage of Lua scripts
Broker's Lua scripts can be stored in any directory readable by the centreon-broker user.
We recommend to store them in /usr/share/centreon-broker/lua.
Note
In a near future, this directory will be in the default path of the Lua scripts launched by broker. It will then be easier to use user defined Lua libraries because you will just have to add your libraries there like stream connectors.