Skip to main content
Skip to main content
Edit this page

Ingesting with OpenTelemetry

All data is ingested into ClickStack via an OpenTelemetry (OTel) collector instance, which acts as the primary entry point for logs, metrics, traces, and session data. We recommend using the official ClickStack distribution of the collector for this instance.

Users send data to this collector from language SDKs or through data collection agents collecting infrastructure metrics and logs (such OTel collectors in an agent role or other technologies e.g. Fluentd or Vector).

Installing ClickStack OpenTelemetry collector​

The ClickStack OpenTelemetry collector is included in most ClickStack distributions, including:

Standalone​

The ClickStack OTel collector can also be deployed standalone, independent of other components of the stack.

If you're using the HyperDX-only distribution, you are responsible for delivering data into ClickHouse yourself. This can be done by:

We recommend using the ClickStack OpenTelemetry collector

This allows users to benefit from standardized ingestion, enforced schemas, and out-of-the-box compatibility with the HyperDX UI. Using the default schema enables automatic source detection and preconfigured column mappings.

To deploy the connector, run the following docker command:

Note that we can overwrite the target ClickHouse instance with environment variables for CLICKHOUSE_ENDPOINT, CLICKHOUSE_USERNAME, and CLICKHOUSE_PASSWORD. The CLICKHOUSE_ENDPOINT should be the full ClickHouse HTTP endpoint, including the protocol and port—for example, http://localhost:8123.

These environment variables can be used with any of the docker distributions which include the connector.

The OPAMP_SERVER_URL should point to your HyperDX deployment - for example, http://localhost:4320. HyperDX exposes an OpAMP (Open Agent Management Protocol) server at /v1/opamp on port 4320 by default. Make sure to expose this port from the container running HyperDX (e.g., using -p 4320:4320).

Users should use a user with the appropriate credentials in production.

Sending OpenTelemetry data​

To send data to ClickStack, point your OpenTelemetry instrumentation to the following endpoints made available by the OpenTelemetry collector:

  • HTTP (OTLP): http://localhost:4318
  • gRPC (OTLP): localhost:4317

For most language SDKs and telemetry libraries that support OpenTelemetry, users can simply set OTEL_EXPORTER_OTLP_ENDPOINT environment variable in your application:

In addition, an authorization header containing the API ingestion key is required. You can find the key in the HyperDX app under Team Settings → API Keys.

For language SDKs, this can either be set by an init function or via anOTEL_EXPORTER_OTLP_HEADERS environment variable e.g.:

Agents should likewise include this authorization header in any OTLP communication. For example, if deploying a contrib distribution of the OTel collector in the agent role, they can use the OTLP exporter. An example agent config consuming this structured log file, is shown below. Note the need to specify an authorization key - see <YOUR_API_INGESTION_KEY>.