Drop #124 (2022-10-21): Weekend Project Edition ↠ Monitoring
HotHost; DomainMOD; Cachet
Weekend projects are fun and all, but if you fire up a bunch of internet-facing services, how do you keep track of them all? Use some of your side-project time this weekend to finally get a handle on the health of your ecosystem.
My weekend project is the Knowledge Drop I’ve been promising since last weekend.
If you use 300 tabs in your browser to monitor service availability, you may be Doing It Wrong™. There's a better (and, free!) way to regularly check if things you stood up have not fallen down.
HotHost is an open-source solution that seems to do everything one might need, including email/slack/telegram/et al. notifications for everything from uptime to resource utilization alarms. The agents and server can run on the same host, but you should really have some small host setup to be your central command.
The Docker images aren't set up to handle arm64 machines, but you can use emulation environments on arm64 Macs and Linux systems (or work on building arm64 compatible Docker images for the community).
The creators of HotHost have an excellent blog that provides enough information that you don't need me to keep typing in this section anymore.
When's the last time you took an inventory of DNS domains you've registered and use? Similarly, how often do you ensure your Let's Encrypt setup is working for all deployed services (the auto-renewals do have a failure rate). Invalid, dead, or pwnd DNS can be a headache, even on a side project TLS errors could get you demoted on Google, and, if nothing else, are just plain embarrassing.
DomainMOD (Docker) [GH] is an all-in-one solution for both DNS and TLS cert monitoring that is so stupid simple to use, and comes with extensive documentation that I, again, won't bore you with much expository beyond noting that the Docker setup supports arm64 environments 🎉, so you can use your M* Macs or fancy arm64 linux systems to run the application.
If you only really care aboot TLS certs, then Let's Monitor is a decent alternative.
Not that you're Monitoring All The Things™, it's time to be like the cool kids and set up a status page, just like the one at GitHub you visit 12 times a month because of how incompetent Microsoft is at running services.
list your service components
customize the UX
use markdown in messages 🎉
make changes via a spiffy JSON API
see metrics regarding the status page
support multiple human languages
have folks subscribe to status update notices
It even supports two-factor authentication!
Now that you have setup watchers for your services, who watches the watcher? If that's too meta for you, why not drop a link in the comments to the status page when you're done?