diff options
Diffstat (limited to 'docs/backends/clickhouse.md')
| -rw-r--r-- | docs/backends/clickhouse.md | 92 |
1 files changed, 92 insertions, 0 deletions
diff --git a/docs/backends/clickhouse.md b/docs/backends/clickhouse.md new file mode 100644 index 0000000..ad1b5f0 --- /dev/null +++ b/docs/backends/clickhouse.md @@ -0,0 +1,92 @@ +# ClickHouse + +Epimetheus can ingest metrics into ClickHouse in **watch mode** only. ClickHouse is optional: you can use it in addition to Prometheus or as the only backend (by setting `-prometheus=` to disable Prometheus ingestion). + +## Data flow (watch mode only) + +``` +┌─────────────────┐ poll (1s) ┌─────────────────────────────────────┐ +│ CSV file(s) │ ─────────────────▶ │ Epimetheus (watch mode) │ +│ (mtime = │ │ • Parse tabular CSV │ +│ timestamp) │ │ • -metric-name + columns → metrics │ +└─────────────────┘ └─────────────────────────────────────┘ + │ + ┌────────────────────┼────────────────────┐ + │ │ │ + ▼ ▼ │ + ┌───────────────┐ ┌───────────────┐ │ + │ Prometheus │ │ ClickHouse │ │ + │ (optional) │ │ (optional) │ │ + │ -prometheus= │ │ -clickhouse= │ │ + │ Remote Write │ │ HTTP insert │ │ + └───────────────┘ └───────────────┘ │ + │ + At least one of -prometheus or -clickhouse │ +``` + +## When It's Used + +- **Mode:** Watch only. Other modes (realtime, historic, backfill, auto) do not write to ClickHouse. +- **Flags:** + - `-clickhouse` – ClickHouse HTTP URL (e.g. `http://localhost:8123`). If empty, no ClickHouse ingestion. + - `-clickhouse-table` – Table name (default: `epimetheus_metrics`). + +At least one of `-prometheus` or `-clickhouse` must be set for watch mode. + +## Table Schema + +Epimetheus creates the table if it does not exist. Schema: + +```sql +CREATE TABLE IF NOT EXISTS epimetheus_metrics ( + metric String, + labels Map(String, String), + value Float64, + timestamp DateTime64(3) +) ENGINE = MergeTree() +ORDER BY (metric, timestamp) +``` + +- `metric` – metric name (e.g. from `-metric-name` and column headers in tabular CSV). +- `labels` – key-value map of label names and values. +- `value` – sample value. +- `timestamp` – sample time (millisecond precision). + +## Examples + +**Prometheus and ClickHouse:** + +```bash +./epimetheus -mode=watch -file=data.csv -metric-name=myapp \ + -prometheus=http://localhost:9090/api/v1/write \ + -clickhouse=http://localhost:8123 +``` + +**ClickHouse only:** + +```bash +./epimetheus -mode=watch -file=test-data/watch-clickhouse-test.csv \ + -metric-name=watch_test \ + -clickhouse=http://localhost:8123 \ + -prometheus= +``` + +**Custom table:** + +```bash +./epimetheus -mode=watch -file=data.csv -metric-name=myapp \ + -clickhouse=http://localhost:8123 \ + -clickhouse-table=my_metrics +``` + +## Verification + +Use the provided script to check that data landed in ClickHouse: + +```bash +./scripts/verify-clickhouse.sh +# Or with custom URL/table: +./scripts/verify-clickhouse.sh http://localhost:8123 epimetheus_metrics +``` + +The script checks connectivity, row count, distinct metrics, sample rows, and rows per metric. See [Setup: ClickHouse](../operations/setup-clickhouse.md) for getting ClickHouse running. |
