Add maintenance mode: lock the platform down from the control panel

Closes every club subdomain with a 503 in that club's own colours, stands the
scheduled jobs down, and keeps open exactly what is needed to end it again.

The exemptions ARE the feature:

- /accounts/ stays open on the base domain. Close it too and you cannot sign in to
  turn maintenance off -- a lock-down with no key, fixable only from a shell.
- /healthz answers on every host. Close it and the load balancer decides the node
  is dead, stops routing to it, and takes the control panel down with everything
  else.
- migrate and collectstatic are NOT blocked. Maintenance is usually declared in
  order to run them; a blanket guard on BaseCommand would mean turning the mode off
  to do the work you turned it on for. Only the domain jobs (archive_overdue_clubs,
  extend_event_series, import_members_csv) refuse, and they exit non-zero so cron
  mails you -- a scheduled job that silently skips itself is how a month of billing
  goes missing.

The state is cached with a 10-second TTL, not for ever. Write-through makes the
flip instant for the shared Redis of a real deployment, and the TTL is the belt to
that braces: on a per-process cache -- a dev box with no Redis, or a misconfigured
deploy -- a lock-down that reached only one gunicorn worker would be worse than
useless. Live-verified: a club subdomain, its login page and the base domain all
503 while the control panel and the sign-in screens stay up.

Also adds the two deployment pieces asked for: compose.behind-proxy.yaml for a
dev/test box that already runs Caddy on :80 (app on the loopback, host Caddy proxies
to it -- and the host's Caddy still needs the DNS plugin, because the wildcard is
still a wildcard), and deploy/backup.sh + restore-check.sh with a cron schedule. The
backup writes to a .part file and only lands it once gzip -t says it is readable: a
truncated dump that looks like a backup is the failure you find on the day you need
it. The weekly restore rehearsal is the only line in that cron that proves the rest
work.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-14 10:11:15 +02:00
parent c0a44093d9
commit d30b163122
22 changed files with 796 additions and 12 deletions

View File

@@ -10,11 +10,15 @@ Everything waffle already offers (``everyone`` / ``percent`` / ``staff`` /
``superusers`` / per-user / per-group) keeps working untouched.
"""
from django.conf import settings
from django.db import models
from django.utils import timezone
from django.utils.translation import gettext_lazy as _
from waffle.models import CACHE_EMPTY, AbstractUserFlag
from waffle.utils import get_cache, keyfmt
from rosterchief.base import UUIDModel
#: Cache key template for a flag's club ids, mirroring waffle's own
#: FLAG_USERS_CACHE_KEY / FLAG_GROUPS_CACHE_KEY.
FLAG_CLUBS_CACHE_KEY = "flag:%s:clubs"
@@ -69,3 +73,74 @@ class Flag(AbstractUserFlag):
if self.everyone is not None:
return self.everyone
return club.pk in self._get_club_ids()
class Maintenance(UUIDModel):
"""Platform lock-down. One row, read on every request.
Cached rather than queried per request, and the cache is the same shared Redis the flags
use — so turning maintenance on in the control panel takes effect on every worker and
every server at once. A per-process cache would leave some workers still serving clubs.
"""
CACHE_KEY = "maintenance:current"
#: Cached, but not for ever. Write-through makes the flip instant for the process that
#: made it and — on the shared Redis of a real deployment — for every other one too. The
#: TTL is the belt to that braces: on a per-process cache (a dev box with no Redis, or a
#: misconfigured deploy) a lock-down that only reached one gunicorn worker would be worse
#: than useless, so the others notice within ten seconds regardless.
CACHE_SECONDS = 10
is_active = models.BooleanField(_("active"), default=False)
message = models.TextField(_("message"), blank=True, help_text=_("Shown to clubs while the platform is locked down."))
started_at = models.DateTimeField(_("started at"), null=True, blank=True)
started_by = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.SET_NULL, null=True, blank=True, related_name="maintenance_windows", verbose_name=_("started by"))
class Meta:
verbose_name = _("maintenance")
verbose_name_plural = _("maintenance")
def __str__(self):
return "Maintenance on" if self.is_active else "Maintenance off"
def save(self, *args, **kwargs):
super().save(*args, **kwargs)
get_cache().set(self.CACHE_KEY, self, self.CACHE_SECONDS)
@classmethod
def current(cls) -> Maintenance:
"""The one row, created on first read. Cached until it changes."""
cached = get_cache().get(cls.CACHE_KEY)
if cached is not None:
return cached
maintenance = cls.objects.first() or cls.objects.create()
get_cache().set(cls.CACHE_KEY, maintenance, cls.CACHE_SECONDS)
return maintenance
@classmethod
def is_on(cls) -> bool:
return cls.current().is_active
@classmethod
def start(cls, *, message: str = "", user=None) -> Maintenance:
maintenance = cls.current()
maintenance.is_active = True
maintenance.message = message
maintenance.started_at = timezone.now()
maintenance.started_by = user
maintenance.save()
return maintenance
@classmethod
def stop(cls) -> Maintenance:
maintenance = cls.current()
maintenance.is_active = False
maintenance.started_at = None
maintenance.started_by = None
maintenance.save()
return maintenance