Important but not Urgent: using Business Hours Schedules

In this guide we go over usage and best practices for Business Hours Schedules in HeyOnCall, an important tool for monitoring your applications without burning out your team.

Photo of Humberto Evans
Author
Humberto Evans

We have a saying at HeyOnCall: Our app should be the best app you never use.

Alerts that interrupt your life outside of work hours should only happen when something aboslutely critical breaks that must be dealt with immediately. So how do we deal with keeping track of and responding to issues that do need someone’s attention, just not immediately? Those issues that can “wait until Monday” are still important, they are just not urgent.

In HeyOnCall you can use Business Hours Schedules to delay notifications of non-critical Triggers of your choosing based on schedules defined in your Organization. Business Hours Schedules are a tool in your arsenal to avoid notification fatigue and prevent burnout, but still have confidence that all your systems are being monitored and nothing will slip through the cracks.

How to set up Business Hours Schedules

Business Hours Schedules can be created for your Organization:

Business Hours button

Specify a name, and the hours for each day of the week that are considered business hours. Most organizations will have only one Business Hours Schedule for the entire company, but you can define more than one if you have entire teams that operate in different time zones or schedules.

Business Hours create

Once you create one (or more) Busienss Hours Schedules you can then assign a Service to a Business Hours Schedule.

Assing a Business Hours Schedule to a Service

Now that our Service has a Business Hours Schedule you can edit any Trigger in the Service and select Defer Notifications Until Business Hours.

Turn on defer until business hours

Now when this Trigger transitions to Alerting when it is not business hours (say it’s a Saturday night), no notifications will be sent to the user that is on call, and no escalations will be fired. If the Trigger is still Alerting when it becomes business hours (on Monday at 9am Pacific), then the current user on call will receive a notification at that time.

If the Trigger becomes Resolved again before that time (such as an Outbound Probe Trigger monitoring a URL that was down but came back up, or an Inbound Liveness Trigger monitoring a cronjob that eventually checked in successfully), no notification will ever be delivered for that Incident, but the Incident can still be seen within the HeyOnCall dashboard.

What is good practice?

Generally the way we think about whether a Trigger should be imited to sending notifications during business hours or not is whether the issue will affect a customer imminently, or if you have time to fix it before it’s a real problem. Here are some examples:

  • The site is down and unreachable - do not defer until business hours
  • The SSL Certificate for the site expires in 7 days - defer until business hours
  • The SSL Certificate for the site expires in 1 day - do not defer until business hours
  • Database disk is 70% full - defer until business hours
  • Database disk is 99% full - do not defer until business hours
  • A cronjob that cleans up 30-day-old stale session table entries (check-in on success) - defer until business hours
  • A background worker process that handles sending email confirmations required of new users (periodic check-in from worker process) - do not defer until business hours

Just like in the examples above, we often have variants of the same trigger at different thresholds, with only the most critical version breaking through outside of business hours.

Remember: just because it is important, does not mean it is urgent.