Skip to content
This repository has been archived by the owner on Jun 16, 2024. It is now read-only.

Use humio as a backend for storing traces for jaeger

License

Notifications You must be signed in to change notification settings

chlunde/humio-jaeger-storage

Repository files navigation

OpenSSF Scorecard Go Report Card

humio-jaeger-storage

This is a storage plugin for Jaeger, an OpenTelemetry implementation. It allows using Humio for storage of spans/traces/logs, which can be queried using jaeger-ui or directly from Humio. By using jaeger-ui, you get a nice user interface showing the spans. By using humio, you can write advanced queries.

Why would you do this?

  • You may want to write more advanced queries to find traces, which is possible in Humio
  • Operating a stateful systems is a non-trivial task, so you may want to have as few stateful systems as possible

Deploying

The plugin is a Go binary, which you can add to the jaeger container images. Then you must specify the path to the storage plugin as a startup parameter (--grpc-storage-plugin.binary=/go/bin/humio-jaeger-storage), as show in the documentation.

With jaeger running on kubernetes, you can deploy the container provided in this repo as an init container to install the plugin. jaegertracing/jaeger-operator#1517

You should also be able to deploy it with dedicated storage nodes as of jaegertracing/jaeger#3835

To deploy a demo setup,

Implementation

We implement the StoragePlugin interface, which means we must provide implementations for the following methods:

type Reader interface {
    GetTrace(ctx context.Context, traceID model.TraceID) (*model.Trace, error)
    GetServices(ctx context.Context) ([]string, error)
    GetOperations(ctx context.Context, service string) ([]string, error)
    FindTraces(ctx context.Context, query *TraceQueryParameters) ([]*model.Trace, error)
    FindTraceIDs(ctx context.Context, query *TraceQueryParameters) ([]model.TraceID, error)
}

type Writer interface {
    WriteSpan(span *model.Span) error
}

See https://github.com/jaegertracing/jaeger/tree/master/plugin/storage/grpc

Storage strategies

Currently everything is stored with spans as events in humio. In the field payload, the JSON-serialized representation of the internal Jaeger span data structure is stored.

Query strategies

Check code for current implementation. It does it in two queries instead of a single query to avoid a slow and too-fancy humio query.

Issues

  • humio no longer provides a docker image which we can use to integration tests without embedding a license file in this repo
  • search for TODO
  • head()/tail()/limit= does not stop query when fulfilled. This is not trivial to implement for anything except simple queries. Humio is so fast this might be a non-issue.
  • No escaping of * in field queries

About

Use humio as a backend for storing traces for jaeger

Topics

Resources

License

Security policy

Stars

Watchers

Forks

Releases

No releases published

Packages