gotosocial/docs/advanced/tracing.md
Jochen Schalanda 56204cc2f4 [feature] add support for sending headers to tracing system
Add support for sending additional HTTP or gRPC headers which
can be used for authentication or other additional information
for the tracing system without having to set up a local instance
of the OpenTelemetry Collector to add these headers.

Example with Dash0:

```yaml
tracing-enabled: false
tracing-transport: "grpc"
tracing-endpoint: "ingress.eu-west-1.aws.dash0.com:4317"
tracing-headers:
  "Authorization": "Bearer DASH0_AUTH_TOKEN"
  "Dash0-Dataset": "gotosocial"
```

Example with Honeycomb:

```yaml
tracing-enabled: false
tracing-transport: "grpc"
tracing-endpoint: "api.honeycomb.io:443"
tracing-headers:
  "x-honeycomb-team": "YOUR_API_KEY"
  "x-honeycomb-dataset": "YOUR_DATASET"
```
2024-12-14 17:28:50 +01:00

2.5 KiB

Tracing

GoToSocial comes with OpenTelemetry based tracing built-in. It's not wired through every function, but our HTTP handlers and database library will create spans. How to configure tracing is explained in the Observability configuration reference.

In order to receive the traces, you need something to ingest them and then visualise them. There are many options available including self-hosted and commercial options.

We provide an example of how to do this using Grafana Tempo to ingest the spans and Grafana to explore them. Please beware that the configuration we provide is not suitable for a production setup. It can be used safely for local development and can provide a good starting point for setting up your own tracing infrastructure.

You'll need the files in example/tracing. Once you have those you can run docker-compose up -d to get Tempo and Grafana running. With both services running, you can add the following to your GoToSocial configuration and restart your instance:

tracing-enabled: true
tracing-transport: "grpc"
tracing-endpoint: "localhost:4317"
tracing-headers:
  "Authorization": "Bearer super-secret-token"
  "Dataset": "gotosocial"
tracing-insecure-transport: true

Querying and visualising traces

Once you execute a few queries against your instance, you'll be able to find them in Grafana. You can use the Explore tab and pick Tempo as the datasource. Because our example configuration for Grafana enables TraceQL, the Explore tab will have the TraceQL query type selected by default. You can switch to "Search" instead and find all traces emitted by GoToSocial under the "GoToSocial" service name.

Using TraceQL, a simple query to find all traces related to requests to /api/v1/instance would look like this:

{.http.route = "/api/v1/instance"}

If you wanted to see all GoToSocial traces, you could instead run:

{.service.name = "GoToSocial"}

Once you select a trace, a second panel will open up visualising the span. You can drill down from there, by clicking into every sub-span to see what it was doing.

Grafana showing a trace for the /api/v1/instance endpoint