Add historic data ingestion support to prometheus-pusher

This commit extends prometheus-pusher to support ingesting historic data
with custom timestamps via Prometheus Remote Write API.

## Key Changes

1. New Historic Data Module (historic.go)
   - GenerateHistoricMetrics: Creates metrics for specific past timestamps
   - PushHistoricData: Sends single datapoint via Remote Write API
   - BackfillHistoricData: Backfills range of historic data
   - Uses Protobuf + Snappy encoding per Prometheus spec

2. Enhanced Main Binary (main.go, realtime.go)
   - Refactored to support multiple modes
   - Mode 1: realtime - Push to Pushgateway (original behavior)
   - Mode 2: historic - Push single historic datapoint
   - Mode 3: backfill - Backfill range of historic data
   - Command-line flags for configuration

3. Prometheus Configuration (persistence-values.yaml)
   - Added web.enable-remote-write-receiver flag
   - Enables Prometheus to accept timestamped samples via Remote Write API
   - Required for historic data ingestion

4. Documentation (HISTORIC.md)
   - Complete guide for historic data ingestion
   - Explains limitations and best practices
   - Examples for all three modes
   - Troubleshooting guide

## Technical Details

**Problem**: Pushgateway doesn't support custom timestamps - Prometheus
always uses "now" when scraping. This prevents backfilling historic data.

**Solution**: Use Prometheus Remote Write API which accepts timestamped
samples. Requires enabling --web.enable-remote-write-receiver flag.

**Data Format**: Protobuf (prompb.WriteRequest) with Snappy compression

**Use Cases**:
- Backfill missing data (e.g., during outage)
- Import historic data from other systems
- Testing with specific timestamps
- Data migration scenarios

## Usage Examples

```bash
# Realtime mode (original behavior)
./prometheus-pusher-historic -mode=realtime -continuous

# Push data from 24 hours ago
./prometheus-pusher-historic -mode=historic -hours-ago=24

# Backfill last 48 hours with 1-hour intervals
./prometheus-pusher-historic -mode=backfill -start-hours=48 -end-hours=0 -interval=1
```

## Dependencies Added
- github.com/prometheus/prometheus (for prompb package)
- github.com/golang/snappy (for compression)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

1 files changed, 5 insertions, 0 deletions

diff --git a/f3s/prometheus/persistence-values.yaml b/f3s/prometheus/persistence-values.yaml
index 5f832d4..4f0d9e4 100644
--- a/f3s/prometheus/persistence-values.yaml
+++ b/f3s/prometheus/persistence-values.yaml
@@ -26,6 +26,11 @@ kubeControllerManager:
 
 prometheus:
   prometheusSpec:
+    # Enable remote write receiver for accepting historic data with custom timestamps
+    # Pass as additional argument to Prometheus binary
+    additionalArgs:
+      - name: web.enable-remote-write-receiver
+        value: "true"
     additionalScrapeConfigsSecret:
       enabled: true
       name: additional-scrape-configs