SLO template: batch job or data pipeline
For a batch job or data pipeline, what matters is whether the data is fresh when people need it.
What to measure
The preset uses a window-based freshness SLI on a gauge that reads 1 while the data is fresh enough and 0 otherwise (data_freshness_ok). Define "fresh enough" from what your consumers need, then expose it as the gauge.
A sensible starting SLO
Use a 28 to 30 day window. A shorter window makes a single missed run weigh heavily, and the wizard warns you below 14 days.
Pre-filled in the wizard
These are the exact values the wizard loads when you open this template. Rename them to match your own service and metric.
| Service name | data-pipeline |
|---|---|
| SLO name | pipeline_freshness |
| SLI kind | freshness |
| SLI type | window_based |
| Metric | data_freshness_ok |
Pitfalls
- Define freshness from your consumer's deadline, not from the job's schedule: a job that runs on time can still deliver stale data.
- A job that never starts produces no failure signal. Make sure the gauge drops to 0 when the data gets old, not only when a run fails.