docs: restructure documentation and move scripts to scripts/

- Add docs/ hierarchy: guides, backends, operations, reference, design
- Slim root README; add documentation index and links to docs/
- Add missing docs: csv-format-flexibility, dns-resolution, dtail-metrics-example, magefile
- Document Prometheus/VictoriaMetrics and ClickHouse backends
- Move all helper shell scripts to scripts/; update Magefile and doc references
- Add ASCII diagrams for watch mode (CSV watcher), auto mode, and ingestion paths
- Add .gitignore

Co-authored-by: Cursor <cursoragent@cursor.com>

1 files changed, 76 insertions, 0 deletions

diff --git a/docs/backends/prometheus.md b/docs/backends/prometheus.md
new file mode 100644
index 0000000..f8d2a9b
--- /dev/null
+++ b/docs/backends/prometheus.md
@@ -0,0 +1,76 @@
+# Prometheus (and Prometheus-Compatible Backends)
+
+Epimetheus can ingest metrics into Prometheus via two paths. Any backend that exposes the Prometheus Remote Write API (including **VictoriaMetrics**) is supported by pointing `-prometheus=` at that backend's write URL (e.g. `http://victoriametrics:8428/api/v1/write`).
+
+## Ingestion paths (overview)
+
+```
+                    Epimetheus
+                         │
+         ┌───────────────┼───────────────┐
+         │               │               │
+         ▼               ▼               ▼
+   Realtime mode    Historic/Backfill   Watch mode
+   (current data)   (old data)          (CSV file mtime)
+         │               │               │
+         ▼               │               │
+   ┌───────────┐         │               │
+   │Pushgateway │         │               │
+   │ (HTTP POST)│         │               │
+   └─────┬─────┘         │               │
+         │ Scrape         │               │
+         │ (15–30s)       │               │
+         ▼               ▼               ▼
+   ┌─────────────────────────────────────────────┐
+   │  Prometheus / VictoriaMetrics               │
+   │  Remote Write API: /api/v1/write            │
+   │  (realtime: via Pushgateway scrape;         │
+   │   historic/watch: direct POST)              │
+   └─────────────────────────────────────────────┘
+```
+
+## Ingestion Paths
+
+### Realtime: Pushgateway
+
+- **Used by:** realtime mode, and auto mode for samples < 5 minutes old.
+- **Flow:** Epimetheus pushes to Pushgateway (HTTP POST); Prometheus scrapes Pushgateway on its schedule. Timestamps become "now" at scrape time.
+- **Flags:** `-pushgateway` (default `http://localhost:9091`), `-job`, `-continuous`.
+
+### Historic: Remote Write API
+
+- **Used by:** historic mode, backfill mode, auto mode for samples ≥ 5 minutes old, and watch mode (when `-prometheus` is set).
+- **Flow:** Epimetheus sends samples to the Remote Write endpoint (e.g. `/api/v1/write`). Timestamps from the data are preserved.
+- **Flags:** `-prometheus` (default `http://localhost:9090/api/v1/write`).
+
+The Remote Write receiver must be enabled on Prometheus for historic/watch/backfill/auto with old data. See [Setup: Prometheus](../operations/setup-prometheus.md).
+
+## Prometheus-Compatible Backends (e.g. VictoriaMetrics)
+
+Backends that implement the [Prometheus Remote Write](https://prometheus.io/docs/concepts/remote_write_spec/) API work with Epimetheus without any code changes. Use their write endpoint as the `-prometheus=` URL.
+
+**Example (VictoriaMetrics):**
+
+```bash
+./epimetheus -mode=watch -file=data.csv -metric-name=myapp \
+  -prometheus=http://victoriametrics:8428/api/v1/write
+```
+
+Replace host/port with your VictoriaMetrics (or other compatible) write URL. Realtime mode still uses Pushgateway (scraped by your Prometheus or VictoriaMetrics); for watch/historic/backfill/auto, only the `-prometheus=` target changes.
+
+## Time Ranges
+
+| Time range | Status | Method |
+|------------|--------|--------|
+| Current (< 5 min) | Supported | Pushgateway |
+| 1 hour old | Supported | Remote Write |
+| 1 day to 1 month old | Supported | Remote Write |
+| 6+ months | May be rejected (retention) | Remote Write |
+| Years old | Likely rejected; use `promtool tsdb create-blocks-from` | — |
+| Future (> 5 min ahead) | Rejected | — |
+
+Out-of-order samples (older than existing data for the same series) require out-of-order ingestion to be enabled on the backend, or use different labels. See [Troubleshooting](../operations/troubleshooting.md).
+
+## Retention and Configuration
+
+Check your backend's retention (e.g. Prometheus `retention`, VictoriaMetrics settings). For very old data you may need to increase retention or enable out-of-order ingestion. See [Setup: Prometheus](../operations/setup-prometheus.md) for Prometheus-specific options.