How nines builds your alerts
The reasoning behind every generated rule, and the numbers it uses.
From HTTP status to error budget
Every SLO is built in the same order and the generator never skips a step: raw status codes become a success rate, the success rate is the SLI, the SLI is compared with an objective to form the SLO, and what the objective leaves over is the error budget.
Burn-rate alert policies
These four policies are the defaults from the Google SRE Workbook, unmodified. Each pairs a long window with a short one: the alert fires only when both exceed the burn rate, so the problem is both significant and still happening.
| Severity | Long window | Short window | Burn rate | Budget consumed |
|---|---|---|---|---|
| PAGE | 1h | 5min | 14.4× | 2% |
| PAGE | 6h | 30min | 6× | 5% |
| TICKET | 3d | 6h | 1× | 10% |
| TICKET | 30d | 6h | 1× | 100% |
Choosing the window
A window of 28 to 30 days is the sensible default. Below 14 days a single bad afternoon dominates the figures, and the wizard warns you about that volatility.
Business hours
PromQL's hour() and day_of_week() are evaluated when the query runs, not over history, so using them directly inside an SLI gives wrong results on past data. nines generates a dedicated recording rule instead, business_hours:active, which is 1 during working hours and 0 otherwise, and filters the SLI with it.
What is checked before generation
- A 100% objective is rejected: it leaves no error budget, so no operational decision is possible.
- An objective of 99.99% or more triggers a warning: at that level the error budget is under an hour a year.
- An objective under 95% triggers a warning that the SLI, or the criticality of the service, may need rethinking.
- Metric and label names are validated against the Prometheus naming rules, and the generated rules are checked with promtool before packaging.