Story · Engineering culture

The outage alarm you could not ignore

Alerting, physically · a Swedish video-streaming company

A siren, a spinning red light, and a two-button acknowledge. When production breaks, the whole office should know — once.

Dashboards are easy to ignore. Emails even easier. So when we wanted the office to actually notice a service outage, we skipped the notification settings and mounted a spinning red alarm light in the ceiling — with a siren.

Red alarm lights and wiring being installed in an office ceiling, with a laptop on a chair below — Installation in progress. Health and safety were notified by the siren, like everyone else.

When an outage was detected — during office hours only; we were enthusiastic, not cruel — the light would spin and the siren would sound until someone acknowledged it. Acknowledging meant pressing two buttons at the same time, missile-launch style: a deliberate act by someone who had actually seen the alarm, not a reflexive swat at a snooze button.

A spinning red alarm light mounted in the ceiling of an open-plan office — In position. Ignorable by no one.

It sounds like a joke, and it was fun to build — but it encoded a serious idea: an outage is an event in the room, not a line in a channel. Nobody asked "did you see the alert?" ever again.

Monitoring Alerting Hardware Engineering culture

← All stories

Have a system that needs this kind of attention?

Tell us where your current testing approach breaks down. We will help you define a practical path forward.

Talk to WhileOne