Add maintenance mode: lock the platform down from the control panel
Closes every club subdomain with a 503 in that club's own colours, stands the scheduled jobs down, and keeps open exactly what is needed to end it again. The exemptions ARE the feature: - /accounts/ stays open on the base domain. Close it too and you cannot sign in to turn maintenance off -- a lock-down with no key, fixable only from a shell. - /healthz answers on every host. Close it and the load balancer decides the node is dead, stops routing to it, and takes the control panel down with everything else. - migrate and collectstatic are NOT blocked. Maintenance is usually declared in order to run them; a blanket guard on BaseCommand would mean turning the mode off to do the work you turned it on for. Only the domain jobs (archive_overdue_clubs, extend_event_series, import_members_csv) refuse, and they exit non-zero so cron mails you -- a scheduled job that silently skips itself is how a month of billing goes missing. The state is cached with a 10-second TTL, not for ever. Write-through makes the flip instant for the shared Redis of a real deployment, and the TTL is the belt to that braces: on a per-process cache -- a dev box with no Redis, or a misconfigured deploy -- a lock-down that reached only one gunicorn worker would be worse than useless. Live-verified: a club subdomain, its login page and the base domain all 503 while the control panel and the sign-in screens stay up. Also adds the two deployment pieces asked for: compose.behind-proxy.yaml for a dev/test box that already runs Caddy on :80 (app on the loopback, host Caddy proxies to it -- and the host's Caddy still needs the DNS plugin, because the wildcard is still a wildcard), and deploy/backup.sh + restore-check.sh with a cron schedule. The backup writes to a .part file and only lands it once gzip -t says it is readable: a truncated dump that looks like a backup is the failure you find on the day you need it. The weekly restore rehearsal is the only line in that cron that proves the rest work. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -10,11 +10,15 @@ Everything waffle already offers (``everyone`` / ``percent`` / ``staff`` /
|
||||
``superusers`` / per-user / per-group) keeps working untouched.
|
||||
"""
|
||||
|
||||
from django.conf import settings
|
||||
from django.db import models
|
||||
from django.utils import timezone
|
||||
from django.utils.translation import gettext_lazy as _
|
||||
from waffle.models import CACHE_EMPTY, AbstractUserFlag
|
||||
from waffle.utils import get_cache, keyfmt
|
||||
|
||||
from rosterchief.base import UUIDModel
|
||||
|
||||
#: Cache key template for a flag's club ids, mirroring waffle's own
|
||||
#: FLAG_USERS_CACHE_KEY / FLAG_GROUPS_CACHE_KEY.
|
||||
FLAG_CLUBS_CACHE_KEY = "flag:%s:clubs"
|
||||
@@ -69,3 +73,74 @@ class Flag(AbstractUserFlag):
|
||||
if self.everyone is not None:
|
||||
return self.everyone
|
||||
return club.pk in self._get_club_ids()
|
||||
|
||||
|
||||
class Maintenance(UUIDModel):
|
||||
"""Platform lock-down. One row, read on every request.
|
||||
|
||||
Cached rather than queried per request, and the cache is the same shared Redis the flags
|
||||
use — so turning maintenance on in the control panel takes effect on every worker and
|
||||
every server at once. A per-process cache would leave some workers still serving clubs.
|
||||
"""
|
||||
|
||||
CACHE_KEY = "maintenance:current"
|
||||
|
||||
#: Cached, but not for ever. Write-through makes the flip instant for the process that
|
||||
#: made it and — on the shared Redis of a real deployment — for every other one too. The
|
||||
#: TTL is the belt to that braces: on a per-process cache (a dev box with no Redis, or a
|
||||
#: misconfigured deploy) a lock-down that only reached one gunicorn worker would be worse
|
||||
#: than useless, so the others notice within ten seconds regardless.
|
||||
CACHE_SECONDS = 10
|
||||
|
||||
is_active = models.BooleanField(_("active"), default=False)
|
||||
message = models.TextField(_("message"), blank=True, help_text=_("Shown to clubs while the platform is locked down."))
|
||||
started_at = models.DateTimeField(_("started at"), null=True, blank=True)
|
||||
started_by = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.SET_NULL, null=True, blank=True, related_name="maintenance_windows", verbose_name=_("started by"))
|
||||
|
||||
class Meta:
|
||||
verbose_name = _("maintenance")
|
||||
verbose_name_plural = _("maintenance")
|
||||
|
||||
def __str__(self):
|
||||
return "Maintenance on" if self.is_active else "Maintenance off"
|
||||
|
||||
def save(self, *args, **kwargs):
|
||||
super().save(*args, **kwargs)
|
||||
get_cache().set(self.CACHE_KEY, self, self.CACHE_SECONDS)
|
||||
|
||||
@classmethod
|
||||
def current(cls) -> Maintenance:
|
||||
"""The one row, created on first read. Cached until it changes."""
|
||||
cached = get_cache().get(cls.CACHE_KEY)
|
||||
if cached is not None:
|
||||
return cached
|
||||
|
||||
maintenance = cls.objects.first() or cls.objects.create()
|
||||
get_cache().set(cls.CACHE_KEY, maintenance, cls.CACHE_SECONDS)
|
||||
|
||||
return maintenance
|
||||
|
||||
@classmethod
|
||||
def is_on(cls) -> bool:
|
||||
return cls.current().is_active
|
||||
|
||||
@classmethod
|
||||
def start(cls, *, message: str = "", user=None) -> Maintenance:
|
||||
maintenance = cls.current()
|
||||
maintenance.is_active = True
|
||||
maintenance.message = message
|
||||
maintenance.started_at = timezone.now()
|
||||
maintenance.started_by = user
|
||||
maintenance.save()
|
||||
|
||||
return maintenance
|
||||
|
||||
@classmethod
|
||||
def stop(cls) -> Maintenance:
|
||||
maintenance = cls.current()
|
||||
maintenance.is_active = False
|
||||
maintenance.started_at = None
|
||||
maintenance.started_by = None
|
||||
maintenance.save()
|
||||
|
||||
return maintenance
|
||||
|
||||
Reference in New Issue
Block a user