Tune gunicorn/Postgres/Redis for a memory-limited server
- gunicorn: 3 workers -> 2 (this workload isn't CPU-bound per DEPLOYMENT.md's own sizing), add --preload so workers share immutable memory via copy-on-write instead of each independently importing Django, add --max-requests so a worker that renders a WeasyPrint invoice doesn't carry that memory forever. - Postgres: trim shared_buffers/max_connections from the image defaults (128MB/100), sized for a ~0.2GB dataset instead. - Redis: cap with --maxmemory as a ceiling, not a saving.
This commit is contained in:
@@ -423,7 +423,8 @@ So do not size for the data. Size for the **processes**.
|
|||||||
|
|
||||||
### What actually consumes the box
|
### What actually consumes the box
|
||||||
|
|
||||||
Measured, running this app under gunicorn with `DEBUG=False`:
|
Measured, running this app under gunicorn with `DEBUG=False`, before the tuning below —
|
||||||
|
`--workers 3`, no `--preload`, Postgres and Redis on their image defaults:
|
||||||
|
|
||||||
| | memory |
|
| | memory |
|
||||||
|---|---|
|
|---|---|
|
||||||
@@ -434,6 +435,13 @@ Measured, running this app under gunicorn with `DEBUG=False`:
|
|||||||
| OS + Docker daemon | ~400 MB |
|
| OS + Docker daemon | ~400 MB |
|
||||||
| **steady state** | **~1.0–1.2 GB** |
|
| **steady state** | **~1.0–1.2 GB** |
|
||||||
|
|
||||||
|
Since then, `Dockerfile`/`compose.yaml` were tuned for smaller boxes: `--workers 2 --preload`
|
||||||
|
(one fewer duplicated Django process, and `--preload` shares immutable memory across workers
|
||||||
|
via copy-on-write instead of each worker importing Django independently), plus trimmed Postgres
|
||||||
|
`shared_buffers`/`max_connections` and a Redis `--maxmemory` cap. Expect the gunicorn and
|
||||||
|
Postgres rows to come in lower than above — not yet re-measured, so treat the table as the
|
||||||
|
shape of where memory goes rather than exact numbers on the current config.
|
||||||
|
|
||||||
2 GB would run it. 4 GB is the recommendation for three reasons, all of which are the kind of
|
2 GB would run it. 4 GB is the recommendation for three reasons, all of which are the kind of
|
||||||
thing that bites at the worst moment:
|
thing that bites at the worst moment:
|
||||||
|
|
||||||
|
|||||||
11
Dockerfile
11
Dockerfile
@@ -91,10 +91,19 @@ EXPOSE 8000
|
|||||||
# Migrations are NOT run here. With more than one app container they would race, and a failed
|
# Migrations are NOT run here. With more than one app container they would race, and a failed
|
||||||
# migration inside a starting web process is a bad place to find out — deploy runs them once,
|
# migration inside a starting web process is a bad place to find out — deploy runs them once,
|
||||||
# explicitly (see DEPLOYMENT.md).
|
# explicitly (see DEPLOYMENT.md).
|
||||||
|
# 2 workers, not 3: DEPLOYMENT.md's own sizing says this workload isn't CPU-bound, and each
|
||||||
|
# worker duplicates a full Django process — the single biggest lever on a memory-limited box.
|
||||||
|
# --preload imports the app once in the master and forks workers via copy-on-write instead of
|
||||||
|
# each re-importing Django independently (safe here: no app's ready() touches DB/Redis eagerly,
|
||||||
|
# checked club/features/news/events). --max-requests recycles a worker periodically so the one
|
||||||
|
# that happens to render a WeasyPrint invoice doesn't carry that +50-100MB forever.
|
||||||
CMD ["gunicorn", "rosterchief.wsgi:application", \
|
CMD ["gunicorn", "rosterchief.wsgi:application", \
|
||||||
"--bind", "0.0.0.0:8000", \
|
"--bind", "0.0.0.0:8000", \
|
||||||
"--workers", "3", \
|
"--workers", "2", \
|
||||||
"--threads", "4", \
|
"--threads", "4", \
|
||||||
|
"--preload", \
|
||||||
|
"--max-requests", "500", \
|
||||||
|
"--max-requests-jitter", "50", \
|
||||||
"--timeout", "60", \
|
"--timeout", "60", \
|
||||||
"--access-logfile", "-", \
|
"--access-logfile", "-", \
|
||||||
"--error-logfile", "-"]
|
"--error-logfile", "-"]
|
||||||
|
|||||||
@@ -43,6 +43,8 @@ services:
|
|||||||
POSTGRES_DB: ${POSTGRES_DB:-rosterchief}
|
POSTGRES_DB: ${POSTGRES_DB:-rosterchief}
|
||||||
POSTGRES_USER: ${POSTGRES_USER:-rosterchief}
|
POSTGRES_USER: ${POSTGRES_USER:-rosterchief}
|
||||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set a database password}
|
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set a database password}
|
||||||
|
# See compose.yaml for why these are trimmed from the defaults.
|
||||||
|
command: ["postgres", "-c", "shared_buffers=64MB", "-c", "max_connections=20"]
|
||||||
volumes:
|
volumes:
|
||||||
- pgdata:/var/lib/postgresql/data
|
- pgdata:/var/lib/postgresql/data
|
||||||
healthcheck:
|
healthcheck:
|
||||||
@@ -54,7 +56,7 @@ services:
|
|||||||
redis:
|
redis:
|
||||||
image: redis:7-alpine
|
image: redis:7-alpine
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
command: ["redis-server", "--save", "", "--appendonly", "no"]
|
command: ["redis-server", "--save", "", "--appendonly", "no", "--maxmemory", "32mb", "--maxmemory-policy", "allkeys-lru"]
|
||||||
|
|
||||||
volumes:
|
volumes:
|
||||||
pgdata:
|
pgdata:
|
||||||
|
|||||||
11
compose.yaml
11
compose.yaml
@@ -59,6 +59,11 @@ services:
|
|||||||
POSTGRES_DB: ${POSTGRES_DB:-rosterchief}
|
POSTGRES_DB: ${POSTGRES_DB:-rosterchief}
|
||||||
POSTGRES_USER: ${POSTGRES_USER:-rosterchief}
|
POSTGRES_USER: ${POSTGRES_USER:-rosterchief}
|
||||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set a database password}
|
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set a database password}
|
||||||
|
# shared_buffers/max_connections default to 128MB / 100 — sized for a much bigger database
|
||||||
|
# than this app's (DEPLOYMENT.md: ~0.2GB after 5 years). 20 connections is comfortably above
|
||||||
|
# 2 gunicorn workers x 4 threads plus the odd `manage.py` one-off; trimmed both for the box,
|
||||||
|
# not for the data.
|
||||||
|
command: ["postgres", "-c", "shared_buffers=64MB", "-c", "max_connections=20"]
|
||||||
volumes:
|
volumes:
|
||||||
- pgdata:/var/lib/postgresql/data
|
- pgdata:/var/lib/postgresql/data
|
||||||
healthcheck:
|
healthcheck:
|
||||||
@@ -70,9 +75,11 @@ services:
|
|||||||
redis:
|
redis:
|
||||||
image: redis:7-alpine
|
image: redis:7-alpine
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
command: ["redis-server", "--save", "", "--appendonly", "no"]
|
|
||||||
# Cache only, so nothing here needs to survive a restart. It is not optional though: it
|
# Cache only, so nothing here needs to survive a restart. It is not optional though: it
|
||||||
# is what keeps every gunicorn worker agreeing about which feature flags are on.
|
# is what keeps every gunicorn worker agreeing about which feature flags are on. maxmemory
|
||||||
|
# is a ceiling, not a saving — this is already the smallest process in the stack — but on a
|
||||||
|
# memory-limited box it should evict cache entries under pressure, not grow unbounded.
|
||||||
|
command: ["redis-server", "--save", "", "--appendonly", "no", "--maxmemory", "32mb", "--maxmemory-policy", "allkeys-lru"]
|
||||||
|
|
||||||
volumes:
|
volumes:
|
||||||
pgdata:
|
pgdata:
|
||||||
|
|||||||
Reference in New Issue
Block a user