OpenTelemetry collector
The OpenTelemetry collector is a key service in the telemetry pipeline. It works as a processing hub for telemetry data, and it consists of four main types of components [7]:
Receivers: Ingest telemetry data from multiple sources and formats such as OTLP [8], Jaeger, Prometheus.
Processors: Modify the data by making use of batching and filtering, or by adding extra details to enrich the data.
Exporters: Export processed data to observability platforms or log management systems such as Azure Ap-plication Insights, ElasticSearch, Graylog [9].
Connectors: Links two pipelines. A connector works both as an exporter at the end of one pipeline and as a receiver at the beginning of another pipeline. It can be used to summarize consumed data, replicate it, or route it.
A collector configuration can contain multiple components of each type (receivers, processors, exporters, and connectors). Each component is represented as a module with a specific purpose and a configuration that is tai-lored to a particular technology or protocol. These modules are then connected to form pipelines, which are de-scribed in the service section in the collector configuration.
For example, a collector can be configured with an OTLP receiver and a Syslog exporter. Each of these compo-nents include settings specific to its role and protocol. When connected through a pipeline (defined in the service section of the collector configuration) the collector will receive logs using the OTLP protocol and export them via Syslog. This setup is flexible and can be extended. For instance, by adding an ElasticSearch exporter, the same logs can be sent to both Syslog and ElasticSearch simultaneously.
It is worth noting that this pipeline can also include processors to enrich or filter the data before it’s exported, and
it can include connectors to route or replicate the data between multiple pipelines.
Additionally, extensions can be defined to support extra functionalities that enhance other components’ capabil-ities, like authentication mechanisms [10], OIDC for bearer token validation [11], or persistent storage modules
[12] to ensure log integrity during outages, among others. The bearer tokens and persistent storage modules will be used in the collector configuration examples later in this document.
The collector can be deployed in two ways [13]:
Agent: Runs alongside application instances to collect local telemetry.
Gateway: Runs as a centralized service to aggregate telemetry from multiple sources.