Tune gunicorn/Postgres/Redis for a memory-limited server

- gunicorn: 3 workers -> 2 (this workload isn't CPU-bound per
  DEPLOYMENT.md's own sizing), add --preload so workers share
  immutable memory via copy-on-write instead of each independently
  importing Django, add --max-requests so a worker that renders a
  WeasyPrint invoice doesn't carry that memory forever.
- Postgres: trim shared_buffers/max_connections from the image
  defaults (128MB/100), sized for a ~0.2GB dataset instead.
- Redis: cap with --maxmemory as a ceiling, not a saving.
This commit is contained in:
2026-08-06 22:29:20 +02:00
parent 1be9959481
commit fe19a6f08a
4 changed files with 31 additions and 5 deletions

View File

@@ -423,7 +423,8 @@ So do not size for the data. Size for the **processes**.
### What actually consumes the box ### What actually consumes the box
Measured, running this app under gunicorn with `DEBUG=False`: Measured, running this app under gunicorn with `DEBUG=False`, before the tuning below —
`--workers 3`, no `--preload`, Postgres and Redis on their image defaults:
| | memory | | | memory |
|---|---| |---|---|
@@ -434,6 +435,13 @@ Measured, running this app under gunicorn with `DEBUG=False`:
| OS + Docker daemon | ~400 MB | | OS + Docker daemon | ~400 MB |
| **steady state** | **~1.01.2 GB** | | **steady state** | **~1.01.2 GB** |
Since then, `Dockerfile`/`compose.yaml` were tuned for smaller boxes: `--workers 2 --preload`
(one fewer duplicated Django process, and `--preload` shares immutable memory across workers
via copy-on-write instead of each worker importing Django independently), plus trimmed Postgres
`shared_buffers`/`max_connections` and a Redis `--maxmemory` cap. Expect the gunicorn and
Postgres rows to come in lower than above — not yet re-measured, so treat the table as the
shape of where memory goes rather than exact numbers on the current config.
2 GB would run it. 4 GB is the recommendation for three reasons, all of which are the kind of 2 GB would run it. 4 GB is the recommendation for three reasons, all of which are the kind of
thing that bites at the worst moment: thing that bites at the worst moment:

View File

@@ -91,10 +91,19 @@ EXPOSE 8000
# Migrations are NOT run here. With more than one app container they would race, and a failed # Migrations are NOT run here. With more than one app container they would race, and a failed
# migration inside a starting web process is a bad place to find out — deploy runs them once, # migration inside a starting web process is a bad place to find out — deploy runs them once,
# explicitly (see DEPLOYMENT.md). # explicitly (see DEPLOYMENT.md).
# 2 workers, not 3: DEPLOYMENT.md's own sizing says this workload isn't CPU-bound, and each
# worker duplicates a full Django process — the single biggest lever on a memory-limited box.
# --preload imports the app once in the master and forks workers via copy-on-write instead of
# each re-importing Django independently (safe here: no app's ready() touches DB/Redis eagerly,
# checked club/features/news/events). --max-requests recycles a worker periodically so the one
# that happens to render a WeasyPrint invoice doesn't carry that +50-100MB forever.
CMD ["gunicorn", "rosterchief.wsgi:application", \ CMD ["gunicorn", "rosterchief.wsgi:application", \
"--bind", "0.0.0.0:8000", \ "--bind", "0.0.0.0:8000", \
"--workers", "3", \ "--workers", "2", \
"--threads", "4", \ "--threads", "4", \
"--preload", \
"--max-requests", "500", \
"--max-requests-jitter", "50", \
"--timeout", "60", \ "--timeout", "60", \
"--access-logfile", "-", \ "--access-logfile", "-", \
"--error-logfile", "-"] "--error-logfile", "-"]

View File

@@ -43,6 +43,8 @@ services:
POSTGRES_DB: ${POSTGRES_DB:-rosterchief} POSTGRES_DB: ${POSTGRES_DB:-rosterchief}
POSTGRES_USER: ${POSTGRES_USER:-rosterchief} POSTGRES_USER: ${POSTGRES_USER:-rosterchief}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set a database password} POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set a database password}
# See compose.yaml for why these are trimmed from the defaults.
command: ["postgres", "-c", "shared_buffers=64MB", "-c", "max_connections=20"]
volumes: volumes:
- pgdata:/var/lib/postgresql/data - pgdata:/var/lib/postgresql/data
healthcheck: healthcheck:
@@ -54,7 +56,7 @@ services:
redis: redis:
image: redis:7-alpine image: redis:7-alpine
restart: unless-stopped restart: unless-stopped
command: ["redis-server", "--save", "", "--appendonly", "no"] command: ["redis-server", "--save", "", "--appendonly", "no", "--maxmemory", "32mb", "--maxmemory-policy", "allkeys-lru"]
volumes: volumes:
pgdata: pgdata:

View File

@@ -59,6 +59,11 @@ services:
POSTGRES_DB: ${POSTGRES_DB:-rosterchief} POSTGRES_DB: ${POSTGRES_DB:-rosterchief}
POSTGRES_USER: ${POSTGRES_USER:-rosterchief} POSTGRES_USER: ${POSTGRES_USER:-rosterchief}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set a database password} POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set a database password}
# shared_buffers/max_connections default to 128MB / 100 — sized for a much bigger database
# than this app's (DEPLOYMENT.md: ~0.2GB after 5 years). 20 connections is comfortably above
# 2 gunicorn workers x 4 threads plus the odd `manage.py` one-off; trimmed both for the box,
# not for the data.
command: ["postgres", "-c", "shared_buffers=64MB", "-c", "max_connections=20"]
volumes: volumes:
- pgdata:/var/lib/postgresql/data - pgdata:/var/lib/postgresql/data
healthcheck: healthcheck:
@@ -70,9 +75,11 @@ services:
redis: redis:
image: redis:7-alpine image: redis:7-alpine
restart: unless-stopped restart: unless-stopped
command: ["redis-server", "--save", "", "--appendonly", "no"]
# Cache only, so nothing here needs to survive a restart. It is not optional though: it # Cache only, so nothing here needs to survive a restart. It is not optional though: it
# is what keeps every gunicorn worker agreeing about which feature flags are on. # is what keeps every gunicorn worker agreeing about which feature flags are on. maxmemory
# is a ceiling, not a saving — this is already the smallest process in the stack — but on a
# memory-limited box it should evict cache entries under pressure, not grow unbounded.
command: ["redis-server", "--save", "", "--appendonly", "no", "--maxmemory", "32mb", "--maxmemory-policy", "allkeys-lru"]
volumes: volumes:
pgdata: pgdata: