Sysadmin Is there a downtime monitoring service that calls you?

I just had a bad morning... woke up to find that my server had crashed for five and a half hours while was sleeping (not sure why, Nginx process at 100% usage, not responding, no clues in the logs). I use New Relic for monitoring, and sure enough I got the text message alert and app push notifications from them, but that was not sufficient to wake me up.

---

I need a service which will actually do more to seriously alert me if my server is down for an extended period of time. The ideal solution would be something which keeps calling me every few minutes until I answer and acknowledge the alert.

Does anything like that exist? I know there are a million different monitoring services out there, but they all seem to have the standard email + text message alerts. That's ok, but it doesn't do you much good if you're the only sysadmin and you're asleep.

How do you guys deal with this issue?

 

I'm not sure what's going on, but this is the fourth time it's happened in the last month or so. I'm still running 1.11.5 though; I am going to upgrade to 1.11.8 tonight and hope that solves it. Partitions look good:

Code (Text):

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/md2        219G   80G  129G  39% /
tmpfs            16G     0   16G   0% /dev/shm
/dev/md0        477M  143M  309M  32% /boot
/dev/md4        902G  528G  328G  62% /drive2
tmpfs            16G  104K   16G   1% /tmp

Is there any other log I'm not aware of that I could check for clues?

You're right, that would be ideal! Unfortunately, though, I don't think it's that simple, at least not on iOS. You can't customize the sounds or the lengths of sounds played by an app's notification, unless the app developer specifically implements the ability to do so. All of the downtime alerts that I have seen just send a standard notification using the default sounds.

Thanks for these! UptimePal is exactly what I'm looking for.

I am also testing my own solution with New Relic and IFTTT. New Relic can send a webhook request to any URL you want when your server goes down, and IFTTT can call your phone. So, using the IFTTT Maker service, if a request comes in to my IFTTT Maker trigger URL, it calls my phone with a custom message that I set. I have set a custom ringtone to sound like a warning alarm for the IFTTT number. I wish it would keep calling me until I acknowledge the problem in some way, but this is a good alternative for now.

 

I found another service — this one is free! — that will send requests to a web hook URL: https://uptimerobot.com

Using this I should get at least two calls, one from New Relic and the other from Uptime Robot, if my server goes down.

 

Have you configured any special alerts to notify you of downtime while you are sleeping?

Wait a minute... You just never sleep, do you?