Upgrade to Django 4.2 and support Postgres 17 (#402)

* Most minimal update to py3.9 and a better dockerfile

Signed-off-by: Irving Popovetsky <irving@honeycomb.io>

* Update django rest_framework by a few micro versions to fix a python 2 syntax warning

Signed-off-by: Irving Popovetsky <irving@honeycomb.io>

* A few more surgical dependency updates to rid us of the worst CVEs

Signed-off-by: Irving Popovetsky <irving@honeycomb.io>

* dramatically speed up launch and execution time

Signed-off-by: Irving Popovetsky <irving@honeycomb.io>

* downgrade psycopg2 because apparently we were picking up 2.9 which is too new

Signed-off-by: Irving Popovetsky <irving@honeycomb.io>

* add analysis of Postgres 14 upgrade

* Django and PG upgrade - phase 0

* phase 1 done

* functional test script to temporarily aid us

* phase 2 complete

* phase 3+4 done

* django 4.2 upgrade fully validated in staging

* let's go straight to Postgres 17

* more thoroughly update deps so that we're fully CVE-free and majorly cut down techdebt

* no circleci

* fix admin console

---------

Signed-off-by: Irving Popovetsky <irving@honeycomb.io>

Author: Irving Popovetsky

.circleci/config.yml

@@ -77,4 +77,8 @@ coverage.xml
 
 ## Installer logs
 pip-log.txt
-pip-delete-this-directory.txt
+pip-delete-this-directory.txt
+
+## Database backups and tools with credentials
+backups/
+scripts/db-tools.sh

.gitignore

@@ -0,0 +1,95 @@
+# syntax=docker/dockerfile:1
+
+# =============================================================================
+# Builder stage: compile dependencies
+# =============================================================================
+FROM python:3.12-alpine AS builder
+
+# Install build dependencies for compiling Python packages
+RUN apk add --no-cache \
+    gcc \
+    musl-dev \
+    libffi-dev \
+    postgresql-dev \
+    python3-dev \
+    zlib-dev \
+    jpeg-dev
+
+# Install poetry system-wide (not in venv, so it won't be copied to production)
+RUN pip install --no-cache-dir poetry
+
+# Create clean venv with upgraded pip (--upgrade-deps handles CVE-2025-8869)
+RUN python -m venv /venv --upgrade-deps
+
+WORKDIR /build
+COPY pyproject.toml poetry.lock ./
+
+# Tell poetry to use our venv instead of creating its own
+ENV VIRTUAL_ENV=/venv \
+    PATH="/venv/bin:$PATH"
+
+# Install production dependencies only
+RUN poetry install --only=main --no-interaction --no-cache --compile
+
+# =============================================================================
+# Test builder: add dev dependencies
+# =============================================================================
+FROM builder AS test-builder
+
+RUN poetry install --no-interaction --no-cache --compile
+
+# =============================================================================
+# Runtime base: minimal image shared by test and production
+# =============================================================================
+FROM python:3.12-alpine AS runtime-base
+
+# Install only runtime dependencies (no build tools, no poetry)
+# Upgrade system pip to fix CVE-2025-8869 (even though app uses venv pip)
+RUN apk upgrade --no-cache && \
+    apk add --no-cache libpq libjpeg-turbo && \
+    pip install --no-cache-dir --upgrade pip
+
+ENV PYTHONUNBUFFERED=1 \
+    PYTHONDONTWRITEBYTECODE=1 \
+    PATH="/venv/bin:$PATH"
+
+WORKDIR /app
+
+# =============================================================================
+# Test stage
+# =============================================================================
+FROM runtime-base AS test
+
+COPY --from=test-builder /venv /venv
+COPY src ./src
+COPY .dev ./src/.dev
+COPY pytest.ini ./
+
+WORKDIR /app/src
+
+ENV DJANGO_ENV=testing \
+    ENVIRONMENT=TEST
+
+CMD ["pytest", "-v"]
+
+# =============================================================================
+# Production stage
+# =============================================================================
+FROM runtime-base AS production
+
+COPY --from=builder /venv /venv
+COPY src ./src
+COPY .dev ./src/.dev
+
+# Pre-compile Python bytecode for faster cold starts
+RUN python -m compileall -q ./src/
+
+WORKDIR /app/src
+
+ENV DJANGO_ENV=production \
+    DB_ENGINE=django.db.backends.postgresql
+
+EXPOSE 8000
+
+# Run background task processor and gunicorn
+CMD ["sh", "-c", "python manage.py qcluster & gunicorn operationcode_backend.wsgi -c /app/src/gunicorn_config.py"]