Expose port using DockerOperator

Question

I am using DockerOperator to run a container. But I do not see any related option to publish required port. I need to publish a webserver port when the task is triggered. Any help or guide will be helpful. Thank you!

Answer 1

First, don't forget docker_operator is deprecated, replaced (now) with providers.docker.operators.docker.

Second, I don't know of a command to expose a port in a live (running) Docker container.
As described in this article from Sidhartha Mani

Specifically, I needed access to the filled mysql database. .
I could think of a few ways to do this:

• Stop the container and start a new one with the added port exposure. docker run -p 3306:3306 -p 8080:8080 -d java/server.
• The second option is to start another container that links to this, and knows how to port forward.
• Setup iptables rules to forward a host port into the container.

So:

Following existing rules, I created my own rule to forward to the container

iptables -t nat -D DOCKER ! -i docker0 -p tcp --dport 3306-j DNAT \
         --to-destination 172.17.0.2:3306

This just says that whenever a packet is destined to port 3306 on the host, forward it to the container with ip 172.17.0.2, and its port 3306.

Once I did this, I could connect to the container using host port 3306.
I wanted to make it easier for others to expose ports on live containers.
So, I created a small repository and a corresponding docker image (named wlan0/redirect).

The same effect as exposing host port 3306 to container 172.17.0.2:3306 can be achieved using this command.
This command saves the trouble of learning how to use iptables.

docker run --privileged -v /proc:/host/proc \
           -e HOST_PORT=3306 -e DEST_IP=172.17.0.2 -e DEST_PORT=3306 \
           wlan0/redirect:latest

In other words, this kind of solution would not be implemented from a command run in the container, through an Airflow Operator.

---

As per my understanding DockerOperator will create a new container, then why is there no way of exposing ports while create a new container.

First, the EXPOSE part is, as I mentioned here, just a metadata added to the image. It is not mandatory.
The runtime (docker run) -p option is about publishing, not exposing: publishing a port and mapping it to a host port (see above) or another container port.
That might be not needed with an Airflow environment, where there is a default network, and even the possibility to setup a custom network or subnetwork.

Which means other (Airflow) containers attached to the same network should be able to access a ports of any container in said network, without needing any -p (publication) or EXPOSE directive.