Monitoring

Monitoring architecture

Our monitoring architecture looks like this:

Monitoring alerts

Monitoring alerts of the Service monitoring & Infrastructure monitoring will be sent to the Monitoring channel in Mattermost.

Service monitoring

Uptime Kuma

We’re using Uptime Kuma to monitor all of our services.

Important

Please note we’re also running a public status site based on Uptime Kuma.

Note

The deployment, and all documentation for Uptime Kuma can be found in the GitLab Uptime project.

Infrastructure monitoring

Grafana

We’re using Grafana OSS as dashboard & alert manager to:

• Access Prometheus metrics via PromQL

• Access Loki logs via LogQL

• Visualise metrics & logs

• Use it as alert manager to send notifications to Mattermost

Note

The deployment, and all documentation for Grafana can be found in the GitLab Grafana project.

Prometheus

We’re using Prometheus as metrics collector to:

• Scrape OS, and 🐳 Docker metrics

• Scrape application metrics

• Persist those metrics

• Query, aggregate, and correlate metrics via Grafana

Note

The deployment, and all documentation for Prometheus can be found in the GitLab Prometheus project.

Loki

We’re using Grafana Loki as log receiver to:

• Retrieve OS, and 🐳 Docker logs

• Retrieve application logs

• Persist those logs

• Query, aggregate, and correlate logs via Grafana

Note

The deployment, and all documentation for Loki can be found in the GitLab Loki project.

Alloy

We’re using Grafana Alloy collectors to:

• Collect OS, and 🐳 Docker logs

• Retrieve 🐳 Docker metrics

• Forward them to Loki, and Prometheus

Note

The deployment, and all documentation for Alloy can be found in the GitLab Alloy project.

Node Exporter

We’re using Node Exporters to:

• Expose OS metrics to Prometheus

Note

The deployment, and all documentation for the Node Exporter can be found in the Ansible node_exporter role.