docs: restructure documentation and move scripts to scripts/

- Add docs/ hierarchy: guides, backends, operations, reference, design
- Slim root README; add documentation index and links to docs/
- Add missing docs: csv-format-flexibility, dns-resolution, dtail-metrics-example, magefile
- Document Prometheus/VictoriaMetrics and ClickHouse backends
- Move all helper shell scripts to scripts/; update Magefile and doc references
- Add ASCII diagrams for watch mode (CSV watcher), auto mode, and ingestion paths
- Add .gitignore

Co-authored-by: Cursor <cursoragent@cursor.com>

33 files changed, 1715 insertions, 1049 deletions

diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..c0f69ee
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,31 @@
+# Binaries
+./epimetheus
+prometheus-pusher
+
+# Test coverage
+coverage.out
+coverage.html
+*.prof
+
+# Logs
+*.log
+
+# OS files
+.DS_Store
+Thumbs.db
+
+# IDE
+.vscode/
+.idea/
+*.swp
+*.swo
+
+# Test data
+test-*.csv
+test-*.json
+
+# Temporary files
+tmp/
+benchmark-data-100mb.csv
+benchmark-results/
+OLD/
diff --git a/Magefile.go b/Magefile.go
index 3cce6e0..6a0b124 100644
--- a/Magefile.go
+++ b/Magefile.go
@@ -179,7 +179,7 @@ func Dev() error {
 // GenerateTestData creates test data files
 func GenerateTestData() error {
 	fmt.Println("Generating test data...")
-	return sh.RunV("./generate-test-data.sh")
+	return sh.RunV("./scripts/generate-test-data.sh")
 }
 
 // Backfill runs backfill for the last 48 hours
@@ -192,31 +192,31 @@ func Backfill() error {
 // Benchmark100MB runs the 100MB benchmark
 func Benchmark100MB() error {
 	fmt.Println("Running 100MB benchmark...")
-	return sh.RunV("./benchmark-100mb.sh")
+	return sh.RunV("./scripts/benchmark-100mb.sh")
 }
 
 // Benchmark1GB runs the 1GB benchmark
 func Benchmark1GB() error {
 	fmt.Println("Running 1GB benchmark...")
-	return sh.RunV("./benchmark-1gb.sh")
+	return sh.RunV("./scripts/benchmark-1gb.sh")
 }
 
 // CleanupBenchmarkData removes benchmark data from Prometheus
 func CleanupBenchmarkData() error {
 	fmt.Println("Cleaning up benchmark data...")
-	return sh.RunV("./cleanup-benchmark-data.sh")
+	return sh.RunV("./scripts/cleanup-benchmark-data.sh")
 }
 
 // CleanupBenchmarkMetrics removes benchmark metric files
 func CleanupBenchmarkMetrics() error {
 	fmt.Println("Cleaning up benchmark metric files...")
-	return sh.RunV("./cleanup-benchmark-metrics.sh")
+	return sh.RunV("./scripts/cleanup-benchmark-metrics.sh")
 }
 
 // DeployDashboard deploys the Grafana dashboard
 func DeployDashboard() error {
 	fmt.Println("Deploying Grafana dashboard...")
-	return sh.RunV("./deploy-dashboard.sh")
+	return sh.RunV("./scripts/deploy-dashboard.sh")
 }
 
 // Help prints available targets
diff --git a/README.md b/README.md
index ba10a76..10d7b97 100644
--- a/README.md
+++ b/README.md
@@ -4,993 +4,82 @@
 
 # Epimetheus
 
-A versatile Go tool for pushing metrics to Prometheus with support for both realtime and historic data ingestion.
+A versatile Go tool for pushing metrics to Prometheus (and Prometheus-compatible backends like VictoriaMetrics) and ClickHouse, with support for realtime and historic data ingestion.
 
 ## Why "Epimetheus"?
 
-In Greek mythology, [Epimetheus](https://en.wikipedia.org/wiki/Epimetheus_(mythology)) is Prometheus's brother, whose name means "afterthought" or "hindsight" (while Prometheus means "forethought"). This name cleverly captures the tool's purpose: bringing data to Prometheus **after** collection, whether it's historic data from hours, days, or weeks ago, or realtime data pushed on-demand.
-
-While Epimetheus is sometimes depicted as foolish in myths (he accepted Pandora's box despite warnings), this tool embraces the "afterthought" aspect productively - it's never too late to bring your metrics home to Prometheus!
-
-## Architecture
-
-```
-┌─────────────────────────────────────────────────────────────────────────┐
-│                            Epimetheus                                   │
-│                     (Metrics Ingestion Tool)                            │
-│                                                                         │
-│  Modes:                                                                 │
-│  • Realtime  - Current metrics (< 5 min old)                          │
-│  • Historic  - Historic metrics (≥ 5 min old)                         │
-│  • Backfill  - Range of historic data                                 │
-│  • Auto      - Automatic routing based on timestamp age               │
-└─────────────────────────────────────────────────────────────────────────┘
-         │                                           │
-         │ Realtime Data                            │ Historic Data
-         │ (via HTTP POST)                          │ (via Remote Write API)
-         │ Uses "now" timestamp                     │ Preserves timestamps
-         ▼                                           ▼
-┌─────────────────────┐                    ┌─────────────────────┐
-│   Pushgateway       │                    │    Prometheus       │
-│   (Port 9091)       │                    │    (Port 9090)      │
-│                     │                    │                     │
-│ • Buffers metrics   │                    │ Remote Write API:   │
-│ • Scraped by        │──── Scraped ─────▶ │ /api/v1/write       │
-│   Prometheus        │    every 15-30s    │                     │
-│ • No timestamp      │                    │ Feature Required:   │
-│   preservation      │                    │ --enable-feature=   │
-│                     │                    │  remote-write-      │
-│                     │                    │  receiver           │
-└─────────────────────┘                    └─────────────────────┘
-                                                      │
-                                                      │ Prometheus Query API
-                                                      │ /api/v1/query
-                                                      ▼
-                                           ┌─────────────────────┐
-                                           │     Grafana         │
-                                           │   (Port 3000)       │
-                                           │                     │
-                                           │ • Prometheus as     │
-                                           │   datasource        │
-                                           │ • Dashboards:       │
-                                           │   - Epimetheus      │
-                                           │     Test Metrics    │
-                                           │ • Auto-refresh      │
-                                           └─────────────────────┘
-```
-
-### Data Flow
-
-1. **Realtime Path** (for current data):
-   - Epimetheus → Pushgateway (HTTP POST)
-   - Prometheus scrapes Pushgateway periodically
-   - Timestamp = "now" when Prometheus scrapes
-
-2. **Historic Path** (for old data):
-   - Epimetheus → Prometheus Remote Write API (HTTP POST)
-   - Direct write to Prometheus TSDB
-   - Timestamp preserved from original data
-
-3. **Visualization**:
-   - Grafana queries Prometheus
-   - Displays metrics in dashboards
-   - Auto-refresh every 10 seconds
+In Greek mythology, [Epimetheus](https://en.wikipedia.org/wiki/Epimetheus_(mythology)) is Prometheus's brother—"afterthought" or "hindsight" (while Prometheus means "forethought"). This tool brings data to Prometheus **after** collection: historic data from hours or days ago, or realtime data pushed on-demand. It's never too late to bring your metrics home.
 
 ## Overview
 
-**epimetheus** is a standalone binary that:
-- **Generates** realistic example metrics simulating production applications
-- **Pushes** metrics via Pushgateway (realtime) or Remote Write API (historic)
-- **Automatically detects** timestamp age and chooses the optimal ingestion method
-- **Supports** multiple data formats (CSV, JSON) and all Prometheus metric types
-- **Provides** Grafana dashboard for visualizing test metrics
-
-## Quick Start
-
-### 1. Deploy Pushgateway (one-time setup)
-
-The Pushgateway Helm chart is available in the [conf repository](https://codeberg.org/snonux/conf) at `f3s/pushgateway/helm-chart`.
-
-```bash
-# Clone the conf repository if you haven't already
-git clone https://codeberg.org/snonux/conf.git
-cd conf/f3s/pushgateway/helm-chart
-
-# Deploy Pushgateway
-helm upgrade --install pushgateway . -n monitoring --create-namespace
-```
-
-Alternatively, deploy Pushgateway using the official chart:
-
-```bash
-helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
-helm install pushgateway prometheus-community/prometheus-pushgateway -n monitoring --create-namespace
-```
-
-### 2. Run in Realtime Mode
-
-```bash
-# Port-forward Pushgateway
-kubectl port-forward -n monitoring svc/pushgateway 9091:9091 &
-
-# Push test metrics continuously
-cd /home/paul/git/conf/f3s/epimetheus
-./epimetheus -mode=realtime -continuous
-```
-
-The binary pushes metrics every 15 seconds. Press Ctrl+C to stop.
-
-### 3. View Metrics
-
-```bash
-# Pushgateway UI
-open http://localhost:9091
-
-# Prometheus UI
-kubectl port-forward -n monitoring svc/prometheus-kube-prometheus-prometheus 9090:9090 &
-open http://localhost:9090
-```
-
-## Operating Modes
-
-### 👁️ Watch Mode
-Monitor CSV files for changes and push metrics to Prometheus with file modification timestamps.
-
-**Works with ANY CSV format** - automatically detects numeric vs string columns and sanitizes names.
-
-**NEW: Automatic DNS Resolution** - IP addresses are automatically resolved to hostnames for better observability in Grafana.
-
-```bash
-./epimetheus -mode=watch \
-    -file=mydata.csv \
-    -metric-name=myapp \
-    -prometheus=http://localhost:9090/api/v1/write
-```
-
-**Features:**
-- 🔍 **Format-agnostic**: Works with any tabular CSV structure
-- 📊 **Automatic detection**: Numeric columns → metrics, String columns → labels
-- 🏷️ **Name sanitization**: `min(potatoes)`, `avg(time)`, `p99(latency)` → valid metric names
-- 🌐 **DNS Resolution**: IP addresses → hostnames (e.g., `10.50.52.61` → `foo.example.lan`)
-- 💾 **Smart Caching**: In-memory cache prevents redundant DNS lookups
-- ⏱️ **Timestamp preservation**: Uses file modification time
-- 🔄 **Continuous monitoring**: Polls file every 1 second
-- 💪 **Error resilient**: Continues watching despite failures
-- 🎯 **Remote Write**: Pushes to Prometheus (preserves timestamps)
-
-**CSV Format:**
-Works with any tabular CSV:
-- First row: column headers (automatically sanitized)
-- Subsequent rows: data values
-- Column names can be anything: `min(x)`, `avg(y)`, `p99(latency)`, etc.
-
-**Example 1** - Web metrics:
-```csv
-avg(response_time),p99(latency),endpoint,method
-45.2,120.5,/api/users,GET
-52.1,135.8,/api/orders,POST
-```
-
-Generates:
-```promql
-web_avg_response_time{endpoint="/api/users",method="GET"} 45.2
-web_p99_latency{endpoint="/api/users",method="GET"} 120.5
-web_avg_response_time{endpoint="/api/orders",method="POST"} 52.1
-web_p99_latency{endpoint="/api/orders",method="POST"} 135.8
-```
-
-**Example 2** - Food metrics:
-```csv
-min(potatoes),last(coke),avg(price),country,store_type
-5.2,10.5,12.99,USA,grocery
-3.8,8.2,9.99,Canada,convenience
-```
-
-Generates:
-```promql
-food_min_potatoes{country="USA",store_type="grocery"} 5.2
-food_last_coke{country="USA",store_type="grocery"} 10.5
-food_avg_price{country="USA",store_type="grocery"} 12.99
-# ... etc
-```
-
-Each row generates N samples (N = number of numeric columns).
-
-See [CSV-FORMAT-FLEXIBILITY.md](CSV-FORMAT-FLEXIBILITY.md) for more examples.
-
-**Options:**
-- `-file` - CSV file to watch (required)
-- `-metric-name` - Base metric name (required, e.g., `food`, `network`, `database`)
-- `-prometheus` - Prometheus Remote Write URL (default: http://localhost:9090/api/v1/write)
-- `-clickhouse` - ClickHouse HTTP URL (e.g. http://localhost:8123) to also ingest metrics
-- `-clickhouse-table` - ClickHouse table name (default: epimetheus_metrics)
-- `-job` - Job name for metrics (default: example_metrics_pusher)
-- `-resolve-ip-labels` - Additional IP labels to resolve via DNS (default: ip is always resolved)
-
-**ClickHouse Support:**
-Watch mode can ingest to ClickHouse in addition to (or instead of) Prometheus:
-
-```bash
-# Ingest to both Prometheus and ClickHouse
-./epimetheus -mode=watch -file=data.csv -metric-name=myapp \
-    -prometheus=http://localhost:9090/api/v1/write \
-    -clickhouse=http://localhost:8123
-
-# ClickHouse only (use -prometheus= to disable Prometheus)
-./epimetheus -mode=watch -file=test-data/watch-clickhouse-test.csv \
-    -metric-name=watch_test -clickhouse=http://localhost:8123 -prometheus=
-
-# Verify data in ClickHouse
-./verify-clickhouse.sh
-```
-
-**DNS Resolution:**
-By default, the `ip` label is automatically resolved to a hostname. To resolve additional IP labels:
-
-```bash
-./epimetheus -mode=watch \
-    -file=network.csv \
-    -metric-name=network \
-    -resolve-ip-labels=source_ip,dest_ip
-```
-
-This will resolve: `ip` (default) + `source_ip` + `dest_ip`
-
-**Example:**
-- Input: `ip="10.50.52.61"`
-- Output: `ip="foo.example.lan"`
-- Failed lookups: IP remains unchanged
-
-**Documentation:**
-- [DNS-RESOLUTION-FEATURE.md](DNS-RESOLUTION-FEATURE.md) - Complete DNS resolution guide
-- [CSV-FORMAT-FLEXIBILITY.md](CSV-FORMAT-FLEXIBILITY.md) - Works with ANY CSV format
-- [DTAIL-METRICS-EXAMPLE.md](DTAIL-METRICS-EXAMPLE.md) - Detailed dtail.csv example
-
-### 🔄 Realtime Mode (Default)
-Push current metrics to Pushgateway with "now" timestamp.
-
-```bash
-./epimetheus -mode=realtime -continuous
-```
-
-**Options:**
-- `-pushgateway` - Pushgateway URL (default: http://localhost:9091)
-- `-job` - Job name (default: example_metrics_pusher)
-- `-continuous` - Keep pushing every 15 seconds
-
-### ⏰ Historic Mode
-Push a single datapoint from the past using Remote Write API.
-
-```bash
-# Port-forward Prometheus
-kubectl port-forward -n monitoring svc/prometheus-kube-prometheus-prometheus 9090:9090 &
-
-# Push data from 24 hours ago
-./epimetheus -mode=historic -hours-ago=24
-```
-
-**Options:**
-- `-prometheus` - Prometheus URL (default: http://localhost:9090/api/v1/write)
-- `-hours-ago` - Hours in the past (default: 24)
-
-### 📦 Backfill Mode
-Import a range of historic data points.
-
-```bash
-# Backfill last 48 hours with 1-hour intervals
-./epimetheus -mode=backfill -start-hours=48 -end-hours=0 -interval=1
-
-# Backfill last week with 6-hour intervals
-./epimetheus -mode=backfill -start-hours=168 -end-hours=0 -interval=6
-```
-
-**Options:**
-- `-start-hours` - Start time in hours ago
-- `-end-hours` - End time in hours ago (0 = now)
-- `-interval` - Interval between points in hours
-
-### 🤖 Auto Mode (Recommended!)
-Automatically detect timestamp age and route to the correct ingestion method.
-
-```bash
-# Generate test data
-./generate-test-data.sh
-
-# Import mixed current and historic data
-./epimetheus -mode=auto -file=test-all-ages.csv
-```
-
-**Detection Logic:**
-- Data < 5 minutes old → Pushgateway (realtime)
-- Data ≥ 5 minutes old → Remote Write (historic)
-
-**Options:**
-- `-file` - Input file path
-- `-format` - Data format: csv or json (default: csv)
-- `-pushgateway` - Pushgateway URL
-- `-prometheus` - Prometheus Remote Write URL
-
-## Data Formats
-
-### CSV Format
-
-```csv
-# Format: metric_name,labels,value,timestamp_ms
-# Labels: key1=value1;key2=value2
-epimetheus_test_requests_total,instance=web1;env=prod,100,1767125148000
-epimetheus_test_temperature_celsius,instance=web2,22.5,1767038748000
-
-# Timestamp is optional (uses "now" if omitted)
-epimetheus_test_active_connections,instance=web3,42,
-```
-
-### JSON Format
-
-```json
-[
-  {
-    "metric": "epimetheus_test_requests_total",
-    "labels": {"instance": "web1", "env": "prod"},
-    "value": 100,
-    "timestamp_ms": 1767125148000
-  },
-  {
-    "metric": "epimetheus_test_temperature_celsius",
-    "labels": {"instance": "web2"},
-    "value": 22.5,
-    "timestamp_ms": 1767038748000
-  }
-]
-```
-
-## Test Metrics
-
-All generated metrics use the `epimetheus_test_` prefix to clearly identify them as test data.
+Epimetheus is a standalone binary that:
 
-### Counter: `epimetheus_test_requests_total`
-- **Type:** Counter (monotonically increasing)
-- **Description:** Total number of requests processed
-- **Use case:** Counting total events, requests, errors
+- Pushes metrics via **Pushgateway** (realtime) or **Remote Write API** (historic, watch)
+- Optionally ingests to **ClickHouse** in watch mode
+- Supports **Prometheus-compatible backends** (e.g. VictoriaMetrics) by using their Remote Write URL
+- Offers modes: realtime, historic, backfill, auto, and watch (CSV file monitoring)
+- Accepts CSV and JSON input and provides a Grafana dashboard for test metrics
 
-### Gauge: `epimetheus_test_active_connections`
-- **Type:** Gauge (can increase or decrease)
-- **Description:** Current number of active connections (0-100)
-- **Use case:** Current state measurements, capacity
-
-### Gauge: `epimetheus_test_temperature_celsius`
-- **Type:** Gauge
-- **Description:** Current temperature in Celsius (0-50°C)
-- **Use case:** Environmental monitoring
-
-### Histogram: `epimetheus_test_request_duration_seconds`
-- **Type:** Histogram (distribution)
-- **Description:** Request duration distribution
-- **Buckets:** 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10 seconds
-- **Use case:** Latency measurements, SLO tracking
-
-### Labeled Counter: `epimetheus_test_jobs_processed_total`
-- **Type:** Counter with labels
-- **Description:** Jobs processed by type and status
-- **Labels:**
-  - `job_type`: email, report, backup
-  - `status`: success, failed
-- **Use case:** Categorized counting, multi-dimensional metrics
-
-## Grafana Dashboard
-
-A comprehensive dashboard is available showcasing all test metrics.
-
-### Dashboard Features
-
-- **8 Panels:**
-  1. Request Rate (line graph)
-  2. Total Requests (stat panel)
-  3. Active Connections (gauge with thresholds)
-  4. Temperature (gauge with thresholds)
-  5. Request Duration Histogram (p50, p90, p99)
-  6. Average Request Duration (stat)
-  7. Jobs Processed by Type (bar gauge)
-  8. Jobs Status Breakdown (table)
-
-- **Auto-refresh:** Every 10 seconds
-- **Time range:** Last 15 minutes (customizable)
-- **Dark theme optimized**
-
-### Deploy Dashboard
-
-#### Option 1: Helm/Kubernetes ConfigMap (Recommended)
-
-```bash
-# Deploy via Kubernetes ConfigMap
-kubectl apply -f ../prometheus/epimetheus-dashboard.yaml
-```
-
-The dashboard will be automatically discovered by Grafana.
-
-#### Option 2: Manual Import
-
-```bash
-# Port-forward Grafana
-kubectl port-forward -n monitoring svc/prometheus-grafana 3000:80
-
-# Open Grafana
-open http://localhost:3000
-
-# Go to Dashboards → Import → Upload grafana-dashboard.json
-```
-
-#### Option 3: Automated Script
-
-```bash
-# Deploy via API
-./deploy-dashboard.sh
-
-# Or with custom credentials
-GRAFANA_URL="http://localhost:3000" \
-GRAFANA_USER="admin" \
-GRAFANA_PASSWORD="yourpassword" \
-./deploy-dashboard.sh
-```
-
-## Example Queries
-
-### Basic Queries
-
-```promql
-# View total requests
-epimetheus_test_requests_total
-
-# View request rate over last 5 minutes
-rate(epimetheus_test_requests_total[5m])
-
-# View current active connections
-epimetheus_test_active_connections
-
-# View current temperature
-epimetheus_test_temperature_celsius
-```
-
-### Histogram Queries
-
-```promql
-# 95th percentile request duration
-histogram_quantile(0.95, rate(epimetheus_test_request_duration_seconds_bucket[5m]))
-
-# 50th percentile (median)
-histogram_quantile(0.50, rate(epimetheus_test_request_duration_seconds_bucket[5m]))
-
-# Average request duration
-rate(epimetheus_test_request_duration_seconds_sum[5m]) /
-rate(epimetheus_test_request_duration_seconds_count[5m])
-```
-
-### Labeled Counter Queries
-
-```promql
-# Failed jobs by type
-epimetheus_test_jobs_processed_total{status="failed"}
-
-# Job success rate
-rate(epimetheus_test_jobs_processed_total{status="success"}[5m]) /
-rate(epimetheus_test_jobs_processed_total[5m])
-
-# Total jobs by type
-sum by (job_type) (epimetheus_test_jobs_processed_total)
-```
-
-### Curl Examples
-
-```bash
-# Port-forward Prometheus
-kubectl port-forward -n monitoring svc/prometheus-kube-prometheus-prometheus 9090:9090 &
-
-# Query total requests
-curl -s "http://localhost:9090/api/v1/query?query=epimetheus_test_requests_total" | jq .
-
-# Query temperature
-curl -s "http://localhost:9090/api/v1/query?query=epimetheus_test_temperature_celsius" | jq .
-
-# Query request rate
-curl -s "http://localhost:9090/api/v1/query?query=rate(epimetheus_test_requests_total[5m])" | jq .
-
-# Query histogram p95
-curl -s "http://localhost:9090/api/v1/query?query=histogram_quantile(0.95,rate(epimetheus_test_request_duration_seconds_bucket[5m]))" | jq .
-```
-
-## Time Range Limitations
-
-### ✅ Supported Time Ranges
-
-| Time Range | Status | Method |
-|------------|--------|--------|
-| Current (< 5 min) | ✅ Works | Pushgateway |
-| 1 hour old | ✅ Works | Remote Write |
-| 1 day old | ✅ Works | Remote Write |
-| 1 week old | ✅ Works | Remote Write |
-| 1 month old | ✅ Works | Remote Write |
-
-### ⚠️ Potential Issues
-
-- **Future timestamps:** Rejected (> 5 minutes in future)
-- **Very old data (6+ months):** May be rejected depending on Prometheus retention
-- **Years old:** Likely rejected - use `promtool tsdb create-blocks-from` instead
-- **Out-of-order samples:** Can't insert older data into existing time series (use different labels)
-
-### Prometheus Configuration
-
-Check your retention settings:
-
-```bash
-# View retention
-kubectl get prometheus -n monitoring prometheus-kube-prometheus-prometheus \
-  -o jsonpath='{.spec.retention}'
-
-# Default is typically 15 days
-```
-
-For very old data:
-- Increase retention in Prometheus config
-- Enable out-of-order ingestion (experimental)
-- Use `promtool` for direct TSDB block creation
-
-## Project Structure
-
-```
-epimetheus/
-├── cmd/
-│   └── epimetheus/
-│       └── main.go              # Main entry point
-├── internal/
-│   ├── config/                  # Configuration
-│   ├── metrics/                 # Metric generators
-│   ├── parser/                  # CSV/JSON parsers (includes tabular CSV)
-│   ├── ingester/                # Pushgateway & Remote Write ingesters
-│   └── watcher/                 # File watcher for watch mode
-├── epimetheus            # Compiled binary
-├── grafana-dashboard.json       # Grafana dashboard definition
-├── deploy-dashboard.sh          # Dashboard deployment script
-├── generate-test-data.sh        # Test data generator
-├── run.sh                       # Helper script
-└── README.md                    # This file
-```
-
-## Setup Requirements
-
-### 1. Enable Prometheus Remote Write Receiver ⚠️ **REQUIRED for Historic Data**
-
-**IMPORTANT**: To use historic mode, backfill mode, or auto mode with old data, you **must** enable the Prometheus Remote Write receiver. Without this feature, Epimetheus can only push realtime data via Pushgateway.
-
-The Remote Write receiver is configured in the [conf repository](https://codeberg.org/snonux/conf) at `f3s/prometheus/persistence-values.yaml`:
-
-```yaml
-# In prometheus/persistence-values.yaml (from conf repository)
-prometheus:
-  prometheusSpec:
-    # Enable Remote Write receiver endpoint and Admin API (Prometheus 3.x syntax)
-    additionalArgs:
-      - name: web.enable-remote-write-receiver
-        value: ""
-      - name: web.enable-admin-api
-        value: ""
-
-    # Enable out-of-order ingestion for backfilling
-    # Allows writing data points older than existing data for the same time series
-    enableFeatures:
-      - exemplar-storage
-      - otlp-write-receiver
-
-    # Allow backfilling up to 31 days in the past (provides 1-day buffer for 30-day datasets)
-    tsdb:
-      outOfOrderTimeWindow: 744h  # 31 days
-```
-
-**What This Enables:**
-- **Remote Write API**: HTTP endpoint at `/api/v1/write` for ingesting metrics with custom timestamps
-- **Admin API**: HTTP endpoints at `/api/v1/admin/tsdb/*` for data deletion and management
-- **Out-of-Order Ingestion**: Allows writing data points older than existing data for the same time series
-- **31-Day Window**: Can backfill data up to 31 days in the past (provides 1-day buffer for 30-day datasets)
-
-After updating the configuration, upgrade your Prometheus installation:
-
-```bash
-cd conf/f3s/prometheus
-just upgrade  # Or manually:
-# helm upgrade prometheus prometheus-community/kube-prometheus-stack \
-#   -n monitoring -f persistence-values.yaml
-```
-
-Verify the features are enabled:
-
-```bash
-# Check Remote Write receiver flag
-kubectl get pod -n monitoring prometheus-prometheus-kube-prometheus-prometheus-0 \
-  -o jsonpath='{.spec.containers[0].args}' | grep -o "web.enable-remote-write-receiver"
-
-# Check out-of-order time window
-kubectl get prometheus -n monitoring prometheus-kube-prometheus-prometheus \
-  -o jsonpath='{.spec.tsdb.outOfOrderTimeWindow}'
-# Should output: 744h
-
-# Check admin API flag
-kubectl get pod -n monitoring prometheus-prometheus-kube-prometheus-prometheus-0 \
-  -o jsonpath='{.spec.containers[0].args}' | grep -o "web.enable-admin-api"
-```
-
-**Performance Considerations:**
-
-This configuration is designed for ad-hoc troubleshooting and development, **NOT production use**. Enabling these features has trade-offs:
-
-- **Increased Memory Usage**: Out-of-order ingestion requires additional memory for buffering and sorting time series
-- **Higher TSDB Overhead**: Prometheus TSDB needs to handle non-sequential writes, increasing disk I/O
-- **Query Performance**: Queries may be slower due to fragmented data blocks
-- **Storage Amplification**: Out-of-order samples can trigger additional compactions, increasing storage usage
-
-**Recommendation for Production:**
-- Keep `outOfOrderTimeWindow` as small as possible (or disabled)
-- Monitor Prometheus memory and disk usage closely
-- Use Remote Write only when necessary
-- Consider using dedicated testing/development Prometheus instances
-
-**Note**: The syntax changed in Prometheus 3.x - use `additionalArgs` with `web.enable-remote-write-receiver` instead of the deprecated `enableFeatures: [remote-write-receiver]`.
-
-### 2. Update Prometheus Scrape Config
+## Quick Start
 
-Ensure Pushgateway is in scrape targets:
+1. **Build:** `mage build` or `go build -o epimetheus cmd/epimetheus/main.go`
+2. **Realtime (Pushgateway):** Deploy Pushgateway and Prometheus, then run:
+   ```bash
+   ./epimetheus -mode=realtime -continuous
+   ```
+3. **Watch (Remote Write):** Enable [Remote Write receiver](docs/operations/setup-prometheus.md), then:
+   ```bash
+   ./epimetheus -mode=watch -file=mydata.csv -metric-name=myapp -prometheus=http://localhost:9090/api/v1/write
+   ```
+4. **View:** Prometheus at http://localhost:9090 (after port-forward if needed). For full steps see [Quick Start](docs/guides/quickstart.md).
 
-```yaml
-# additional-scrape-configs.yaml
-- job_name: 'pushgateway'
-  honor_labels: true
-  static_configs:
-    - targets:
-      - 'pushgateway.monitoring.svc.cluster.local:9091'
-```
+## Documentation
 
-Apply the configuration:
+Full documentation is in the [docs](docs/README.md) directory:
 
-```bash
-kubectl create secret generic addi