| Age | Commit message (Collapse) | Author |
|---|
|
This commit configures Prometheus to accept historic data via the Remote
Write API, enabling backfilling of test metrics for development and
troubleshooting purposes.
Changes:
- Enable Remote Write receiver (--web.enable-remote-write-receiver)
- Enable out-of-order ingestion with 30-day window (720h)
- Enable exemplar-storage and otlp-write-receiver features
- Add Epimetheus dashboard ConfigMap for Grafana provisioning
- Remove old prometheus-pusher directory (moved to separate repo)
- Document configuration, use cases, and performance considerations
Configuration allows backfilling data up to 30 days in the past, supporting
tools like Epimetheus for generating synthetic historic metrics.
Performance note: This is optimized for ad-hoc troubleshooting, not
production use. Out-of-order ingestion increases memory usage, TSDB overhead,
and may impact query performance.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
This commit extends prometheus-pusher to support ingesting historic data
with custom timestamps via Prometheus Remote Write API.
## Key Changes
1. New Historic Data Module (historic.go)
- GenerateHistoricMetrics: Creates metrics for specific past timestamps
- PushHistoricData: Sends single datapoint via Remote Write API
- BackfillHistoricData: Backfills range of historic data
- Uses Protobuf + Snappy encoding per Prometheus spec
2. Enhanced Main Binary (main.go, realtime.go)
- Refactored to support multiple modes
- Mode 1: realtime - Push to Pushgateway (original behavior)
- Mode 2: historic - Push single historic datapoint
- Mode 3: backfill - Backfill range of historic data
- Command-line flags for configuration
3. Prometheus Configuration (persistence-values.yaml)
- Added web.enable-remote-write-receiver flag
- Enables Prometheus to accept timestamped samples via Remote Write API
- Required for historic data ingestion
4. Documentation (HISTORIC.md)
- Complete guide for historic data ingestion
- Explains limitations and best practices
- Examples for all three modes
- Troubleshooting guide
## Technical Details
**Problem**: Pushgateway doesn't support custom timestamps - Prometheus
always uses "now" when scraping. This prevents backfilling historic data.
**Solution**: Use Prometheus Remote Write API which accepts timestamped
samples. Requires enabling --web.enable-remote-write-receiver flag.
**Data Format**: Protobuf (prompb.WriteRequest) with Snappy compression
**Use Cases**:
- Backfill missing data (e.g., during outage)
- Import historic data from other systems
- Testing with specific timestamps
- Data migration scenarios
## Usage Examples
```bash
# Realtime mode (original behavior)
./prometheus-pusher-historic -mode=realtime -continuous
# Push data from 24 hours ago
./prometheus-pusher-historic -mode=historic -hours-ago=24
# Backfill last 48 hours with 1-hour intervals
./prometheus-pusher-historic -mode=backfill -start-hours=48 -end-hours=0 -interval=1
```
## Dependencies Added
- github.com/prometheus/prometheus (for prompb package)
- github.com/golang/snappy (for compression)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
|
This commit adds a complete Prometheus data ingestion solution:
1. Pushgateway Helm Chart (f3s/pushgateway/)
- Standalone helm chart for Prometheus Pushgateway
- Deployed to monitoring namespace
- Receives pushed metrics via HTTP POST on port 9091
2. Prometheus Pusher (f3s/prometheus-pusher/)
- Standalone Go binary (12MB) for pushing metrics to Pushgateway
- Demonstrates all Prometheus metric types:
* Counter (app_requests_total)
* Gauge (app_active_connections, app_temperature_celsius)
* Histogram (app_request_duration_seconds)
* Labeled Counter (app_jobs_processed_total)
- Pushes metrics every 15 seconds
- Includes comprehensive documentation and examples
3. Prometheus Configuration
- Updated additional-scrape-configs.yaml to scrape Pushgateway
- Uses honor_labels to preserve pushed metric labels
Architecture:
Go Binary → Pushgateway → Prometheus → Grafana
The pusher binary generates realistic example metrics and pushes them
to Pushgateway in Prometheus text format. Prometheus then scrapes the
Pushgateway and makes the metrics available for querying and alerting.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
