diff options
| author | Paul Buetow <paul@buetow.org> | 2025-12-30 22:07:28 +0200 |
|---|---|---|
| committer | Paul Buetow <paul@buetow.org> | 2025-12-30 22:07:28 +0200 |
| commit | 5023df58f2d1a37aed50cb5f3b00f510cb186d53 (patch) | |
| tree | 865988d91e234cf6a9d5bcf628c4b9bf7668a4ac /frontends/scripts | |
| parent | de3e419a76873c8ac6c6415fbdbdd708fcecdf30 (diff) | |
Add AUTO mode with automatic timestamp detection
This commit adds intelligent auto-detection of data age, automatically
choosing the appropriate ingestion method without user intervention.
## New Features
1. **AUTO Mode** (-mode=auto)
- Automatically detects timestamp age from input data
- Routes realtime data (< 5min) → Pushgateway
- Routes historic data (> 5min) → Remote Write API
- No manual timestamp calculation needed!
2. **Input Format Support**
- CSV format: metric_name,labels,value,timestamp_ms
- JSON format: [{metric, labels, value, timestamp_ms}]
- Read from file (-file=path) or stdin
- Comments supported in CSV (#)
3. **Smart Routing Logic**
- 5-minute threshold determines ingestion method
- Handles mixed data (current + historic) in single import
- Clear logging shows which method is used for each sample
4. **Test Data Generation**
- generate-test-data.sh creates samples for all time ranges
- Demonstrates: current, 1h, 1d, 1w, 1m old data
- Actual timestamps generated dynamically
## Files Added
- auto-ingest.go: Core auto-detection logic
- AUTO-MODE.md: Complete documentation
- generate-test-data.sh: Test data generator
- test-data.csv: Example data template
- test-all-ages.csv: Generated test data (all ages)
## Example Usage
```bash
# Generate test data
./generate-test-data.sh
# Auto-import (detects ages automatically)
./prometheus-pusher-auto \
-mode=auto \
-file=test-all-ages.csv \
-pushgateway=http://localhost:9091 \
-prometheus=http://localhost:9090/api/v1/write
```
## Output Example
```
📊 Auto-ingest summary:
Total samples: 15
Realtime samples (< 5min old): 3
Historic samples (> 5min old): 12
🔄 Ingesting 3 REALTIME samples via Pushgateway...
⏰ Ingesting 12 HISTORIC samples via Remote Write...
[1/12] app_requests_total (age: 1.0 hours)
[2/12] app_temperature_celsius (age: 1.0 days)
...
🎉 Auto-ingest complete!
```
## Supported Time Ranges
✅ Current data (< 5min)
✅ 1 hour old data
✅ 1 day old data
✅ 1 week old data
✅ 1 month old data
All ages are automatically detected and routed correctly!
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Diffstat (limited to 'frontends/scripts')
0 files changed, 0 insertions, 0 deletions
