summaryrefslogtreecommitdiff
path: root/f3s/prometheus-pusher/go.mod
diff options
context:
space:
mode:
authorPaul Buetow <paul@buetow.org>2025-12-30 21:36:48 +0200
committerPaul Buetow <paul@buetow.org>2025-12-30 21:36:48 +0200
commitde3e419a76873c8ac6c6415fbdbdd708fcecdf30 (patch)
tree2909220393a8099fc376492672458426db09bd76 /f3s/prometheus-pusher/go.mod
parent37a0ce2c6ad01a7439739c0794e74fd0b052bcae (diff)
Add historic data ingestion support to prometheus-pusher
This commit extends prometheus-pusher to support ingesting historic data with custom timestamps via Prometheus Remote Write API. ## Key Changes 1. New Historic Data Module (historic.go) - GenerateHistoricMetrics: Creates metrics for specific past timestamps - PushHistoricData: Sends single datapoint via Remote Write API - BackfillHistoricData: Backfills range of historic data - Uses Protobuf + Snappy encoding per Prometheus spec 2. Enhanced Main Binary (main.go, realtime.go) - Refactored to support multiple modes - Mode 1: realtime - Push to Pushgateway (original behavior) - Mode 2: historic - Push single historic datapoint - Mode 3: backfill - Backfill range of historic data - Command-line flags for configuration 3. Prometheus Configuration (persistence-values.yaml) - Added web.enable-remote-write-receiver flag - Enables Prometheus to accept timestamped samples via Remote Write API - Required for historic data ingestion 4. Documentation (HISTORIC.md) - Complete guide for historic data ingestion - Explains limitations and best practices - Examples for all three modes - Troubleshooting guide ## Technical Details **Problem**: Pushgateway doesn't support custom timestamps - Prometheus always uses "now" when scraping. This prevents backfilling historic data. **Solution**: Use Prometheus Remote Write API which accepts timestamped samples. Requires enabling --web.enable-remote-write-receiver flag. **Data Format**: Protobuf (prompb.WriteRequest) with Snappy compression **Use Cases**: - Backfill missing data (e.g., during outage) - Import historic data from other systems - Testing with specific timestamps - Data migration scenarios ## Usage Examples ```bash # Realtime mode (original behavior) ./prometheus-pusher-historic -mode=realtime -continuous # Push data from 24 hours ago ./prometheus-pusher-historic -mode=historic -hours-ago=24 # Backfill last 48 hours with 1-hour intervals ./prometheus-pusher-historic -mode=backfill -start-hours=48 -end-hours=0 -interval=1 ``` ## Dependencies Added - github.com/prometheus/prometheus (for prompb package) - github.com/golang/snappy (for compression) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Diffstat (limited to 'f3s/prometheus-pusher/go.mod')
-rw-r--r--f3s/prometheus-pusher/go.mod23
1 files changed, 16 insertions, 7 deletions
diff --git a/f3s/prometheus-pusher/go.mod b/f3s/prometheus-pusher/go.mod
index 4576c38..e5fbf08 100644
--- a/f3s/prometheus-pusher/go.mod
+++ b/f3s/prometheus-pusher/go.mod
@@ -1,16 +1,25 @@
module prometheus-pusher
-go 1.21
+go 1.24.0
-require github.com/prometheus/client_golang v1.20.5
+require (
+ github.com/golang/snappy v1.0.0
+ github.com/prometheus/client_golang v1.23.2
+ github.com/prometheus/prometheus v0.308.1
+)
require (
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
+ github.com/gogo/protobuf v1.3.2 // indirect
+ github.com/grafana/regexp v0.0.0-20250905093917-f7b3be9d1853 // indirect
+ github.com/kr/text v0.2.0 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
- github.com/prometheus/client_model v0.6.1 // indirect
- github.com/prometheus/common v0.55.0 // indirect
- github.com/prometheus/procfs v0.15.1 // indirect
- golang.org/x/sys v0.22.0 // indirect
- google.golang.org/protobuf v1.34.2 // indirect
+ github.com/prometheus/client_model v0.6.2 // indirect
+ github.com/prometheus/common v0.67.4 // indirect
+ github.com/prometheus/procfs v0.16.1 // indirect
+ go.yaml.in/yaml/v2 v2.4.3 // indirect
+ golang.org/x/sys v0.37.0 // indirect
+ golang.org/x/text v0.30.0 // indirect
+ google.golang.org/protobuf v1.36.10 // indirect
)