Description

SUSE Observability pre-processes topology data during ingestion and stores it in its custom graph database. An alternative approach is to treat topology data similar to all other telemetry signals, which are stored mostly as-is in an appropriate store (a VictoriaMetrics for metrics, ClickHouse for traces).

ClickHouse seems like a good choice for storing topology data. The query side of topology also needs to be modified to not just read from the different store, but also to implement some of the logic that was done during ingestion at query time instead.

Of course we don't want to lose the time-traveling functionality or any of the other features.

Goals

We don't expect to implement a full solution at all, but hope to get to a POC implementation that allows storing some of the data and implementing 1 or 2 query endpoints to support minimal query functionality. With this we want to answer some of these questions:

• How does the ingestion performance and resource usage compare to the existing setup?
• How does the disk usage behave for Clickhouse? Is the ClickHouse compression good enough or are more optimizations needed to limit disk space usage.
• How hard is it to implement queries on top of the new schema?
• How is the query performance.

Ideally, we manage to implement the topology querying for tag-based queries (and we simply ignore any other parts), this is one of the more heavy queries that is also used a lot throughout the app.

Resources