Scheduled jobs

We’ve several different scheduled jobs across our infrastructure:

  • Cronjobs on the Linux servers

  • systemd timers on the Linux servers

  • Scheduled GitLab CI pipelines

  • Application tasks (e.g. via Celery)

Schedules

Important

To ensure a stable environment, it’s important that all the jobs are properly synced in time. Whenever setting up a new job, ensure you’re following the schedules below.

Jobs for which downtime is expected will be communicated to customers via 🗓 Maintenance windows. Thus, don’t change the times without appropriate communication.

OS schedules

Time

Weekday

Action

Source

*:00 (+rand. 15min)

everyday

Run borg backups.

borg role

*:00 (+rand. 15min)

Sundays

Run borg checks.

borg role

01:00

everyday

Run short S.M.A.R.T. tests

smart role

02:00

Sundays

Run long S.M.A.R.T. tests

smart role

04:30 (+rand. 15min)

everyday

Download apt upgrades. After CI jobs, before upgrades.

apt_unattended_upgrades role

05:30 (+rand. 5min)

everyday

Install apt upgrades. After CI jobs & downloads, before morning.

apt_unattended_upgrades role

05:50 / 05:55

Mondays through Saturdays

Reboot systems when required. After upgrades, before morning. No interference with Sunday borg checks.

reboot role

CI & Docker schedules

Time

Weekday

Action

Source

*:00

everyday

Run Harbor (Docker registry) garbage collection. Doesn’t affect backup much.

Harbor garbage collection

/4:00

everyday

Run GitLab Guard. Doesn’t affect backup much.

GitLab Guard pipeline

/4:00

everyday

Run Renovate (auto-merges only between 08:00 & 18:00). Doesn’t affect backup much.

Renovate pipeline

03:30

everyday

Build Docker Base images (stage 1). After backup is done.

Docker image pipelines

03:45

everyday

Build Docker Base images (stage 2). After stage 1 is done.

Docker image pipelines

04:00

everyday

Run library tests. After new base images.

Python library pipelines

04:15

everyday

Run vulnerability scans. After new base images.

Harbor interrogation services

04:30

everyday

Run Docker system garbage collection. After backup, far after new base images.

docker-gc timer docker-gc service

Docker service schedules

Other schedules

Time

Weekday

Action

Source

05:50

Sundays

Synology NAS updates.

Control Panel → Update & Restore → Update Settings