Python Docker image size for legacy app: multistage, C extensions, secrets

[Daniel_G]

I'm a software engineer trying to containerize our team's legacy Python application using Docker to improve deployment consistency, but I'm running into issues with the image size blowing up due to dependencies and struggling to structure a multi-stage build effectively. The application has a mix of system libraries, Python packages, and some custom C extensions, which makes creating an efficient and secure Dockerfile more complex than the basic tutorials I've followed. For developers experienced with Docker in production, what are your best practices for optimizing image size and layer caching for a language like Python? How do you handle secrets and configuration management within containers, and what tools or patterns do you use for local development to mirror the containerized environment as closely as possible?

[MatthewW]

Reply 1
- Start with a two-stage build using a slim Python base (for example python:3.11-slim) as the runtime, and a separate builder stage that compiles any C extensions. In the final image, copy only what you need (Python runtime + your app) and drop build tools.
- Optimize your Dockerfile order to maximize layer caching: put requirements.txt first and install dependencies before copying app code, so changes to code don’t bust the whole layer. Use --no-cache-dir with pip to keep image size down. Consider enabling BuildKit and using cache mounts for pip and other heavy steps.
- Don’t forget to clean apt on Debian-based images and remove temporary files; run apt-get clean && rm -rf /var/lib/apt/lists/* as part of the RUN chain.
- Create a non-root user for the runtime and keep your application running under that user.