Why is environment variable mangled in docker compose environment file

Question

Context: docker-compose to start several containers, including Gunicorn that calls a Flask app. I'm using an environment file web/env.gunicorn to store my Gunicorn startup configuration. This file contains

GUNICORN_CMD_ARGS="--bind=127.0.0.1:8001 --workers=3"

The problem is that GUNICORN_CMD_ARGS is not processed correctly somewhere in the pipeline. The error that I get when running docker logs gunicorn is

Error: '8001 --workers=3' is not a valid port number.

The question is, where in my setup are my assumptions wrong, thereby causing the environment variable to be mangled? The environment variable is accepted when run manually in a terminal. The docker compose file looks as follows

version: "3"
services:
#  nginx:
#    image: nginx:latest
#    ports:
#      - "80:80"
#    volumes:
#      - ./nginx:/etc/nginx/conf.d
#    depends_on:
#      - web

  web:
    build: ./web
    container_name: gunicorn
    ports:
      - "8001:8001"
    environment:
      - APP_CONFIG_FILE=../config/development.py
    env_file:
      - 'web/env.gunicorn'
    networks:
      - backend
    command: gunicorn thymedata:app
    depends_on:
      - influxdb
      - grafana

  influxdb:
    image: influxdb:latest
    container_name: influxdb
    ports:
      - "8086:8086"
    env_file:
      - 'influxdb/env.influxdb'
      - 'influxdb/secrets.influxdb'
    networks:
      - backend
    volumes:
      - influxdb-data:/var/lib/influxdb

  grafana:
    image: grafana/grafana:latest
    container_name: grafana
    ports:
      - "3000:3000"
    env_file:
      - 'grafana/env.grafana'
      - 'grafana/secrets.grafana'
    networks:
      - backend
    volumes:
      - grafana-data:/var/lib/grafana
    depends_on:
      - influxdb

networks:
  backend:

volumes:
  influxdb-data:
  grafana-data:

Not using gunicorn, but reading the doc there's no = sign out of the note at about the env variable. I'd try with a space, but it sounds like it doesn't parse properly multiples arguments and as such is maybe a bug in gunicorn (inverting the arguments order to end by the bind argument may be a workaround or exhibit the split problem with the workers parameter) — Tensibai, Dec 29 '17 at 21:33
I tried changing the ordering of the parameters, but that did not help. I also tried starting Gunicorn from the command line with GUNICORN_CMD_ARGS set, and it accepted it properly (# of workers and address checked). That's why i suppose the problem being on Docker's side. — Moritz, Dec 30 '17 at 06:13
ERROR: build path /home/ben/devopsStackExchange/web either does not exist, is not accessible, or is not a valid URL.. Please create a steps to reproduce paragraph and add the required files so that other people could help you by at least allowing them to reproduce the issue. — 030, Dec 30 '17 at 16:11
I don't use a Gunicorn Dockerfile. I'll stick to the command: gunicorn thymedata:app --bind=127.0.0.1:8001 --workers=3 version, since the variables loaded via env_file are not accepted. They have to already be set in the calling terminal. — Moritz, Dec 31 '17 at 09:23

score 3 · Answer 1 · answered May 16 '18 at 23:44

I had a similar problem and this worked for me without it feeling like a hack...

docker-compose.yml:

version: '3'
services:
   web:
      build .
      environment:
      - GUNICORN_CMD_ARGS=--workers=0 --bind=0.0.0.0:8000 --timeout=10

Note: add env GUNICORN_CMD_ARGS without quotes

Dockerfile:

FROM python:3.6-slim
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
ENV GUNICORN_CMD_ARGS="--bind=0.0.0.0:8000 --workers=3"
CMD gunicorn app:app

Note: run CMD gunicorn app:app without passing options via array or quotes

score 2 · Accepted Answer · answered Dec 30 '17 at 16:26

2

After reading the commands chapter and finding this sample, it looks like that the custom GUNICORN_CMD_ARGS could be omitted by just passing the arguments directly:

command: gunicorn thymedata:app --bind=127.0.0.1:8001 --workers=3

If one really would like to use variables then it could be done as follows:

command: gunicorn thymedata:app --bind=${GUNICORN_BIND_ADDRESS} --workers=${GUNICORN_WORKERS}

.env

GUNICORN_BIND_ADDRESS=127.0.0.1:8001
GUNICORN_WORKERS=3

answered Dec 30 '17 at 16:26

030

13,235
16
74
173

${GUNICORN_BIND_ADDRESS} will only evaluate variables already in the terminal, none that are loaded via env_file. Since I do not set any secrets, it's an acceptable solution – Moritz Dec 31 '17 at 09:25

score 0 · Answer 3 · answered Sep 22 '20 at 20:43

In ran into this problem with my environment file with docker-compose, and I found the problem to be with the quotes.

If I put GUNICORN_CMD_ARGS=--bind=127.0.0.1:8001 --workers=3 on the .env file the parameters are accepted without any issue, but if I put quotes around them the variable gets mangled by docker-compose.

Why is environment variable mangled in docker compose environment file

3 Answers3