PuppetConf 2014

As the Yelp infrastructure and engineering team grew, so did the pain of managing Nagios. Problems like splitting alerting across multiple teams, providing high availability, and managing Nagios systems in multiple environments had become pressing. As we grew toward a service-oriented architecture and pushed some services out into the cloud, we rapidly needed more automated monitoring configuration.

An evolutionary solution wasn’t going to solve all of our problems—we needed to revolutionize our monitoring. Sensu is built from the ground up to solve many of our issues and to be easy to extend.

This talk will cover our puppet ‘monitoring_check’ API (which sets up monitoring for our services within puppet), how and why we deploy Sensu and our custom handlers and escalations, along with how we provide automatic ‘self service’ monitoring for dynamic services and deal with the challenges posed by the more ephemeral nature of cloud architectures.