How do I deal with the "max depth exceeded" error in docker?

Question

I have inherited a base image from a previous co-worker. I just have the image, no dockerfile corresponding to it. The image is name 'c1829df3bbea' below. When I run this command I see that it has 167 layers:

docker history c1829df3bbea | wc -l

Now some company configurations have changed which require me to add some new files to the image. I created the Dockerfile below, which uses the original image as the base layer:

FROM rbc/pmcs/dna_prod:latest

USER root

# Copy the run script to the docker container
COPY bin/run.sh $PMCS_HOME/run.sh

# Now copy in the most recent hadoop config files, as required by the migration to Range KMS on Saturday Sep 26 2020
COPY edlhp1-config/edlhp1/hadoop /app/hadoop_conf/strprod/hadoop

# Update the security certificates to the most recent versions
RUN cd /etc/pki/ca-trust/source/anchors && curl --insecure -O https://webapps4.fg.rbc.com/WLM0/Forms/Help/Production_RBC_G2_Root_CA.cer && curl --insecure -O https://webapps4.fg.rbc.com/WLM0/Forms/Help/Production_RBC_G2_Intermediate_CA.cer && update-ca-trust extract && ls /etc/pki/ca-trust/source/anchors && update-ca-trust force enable && update-ca-trust extract && chmod 755 $PMCS_HOME/run.sh

USER $USER

Then I rebuild using the command:

docker build . --tag rbc/pmcs/prod:1.2

It loads 5 of the 6 steps OK, but then fails on the last step, with the message "max depth exceeded".

I realize there are some things that can be done to reduce the number of layers, such as combining run statements, and making copy statements move multiple files at once. Please assume that this has been done as much as possible. I'd like to know if there is a way to reduce the layers in my base image when I don't have access to the Dockerfile that was used to build it.

Answer 1

When I run this command I see that it has 167 layers:

Wow. That's pretty big.

I'm not sure how you got here, but I'm betting that you (or rather, your co-worker) used a FROM command in your Dockerfile which referenced the output of the Dockerfile, then continued to stack layers on top of it when there was a configuration change.

Going forward, I think you could either squash the image or reverse engineer the Dockerfile. In the long run, reverse engineering the Dockerfile is probably better than squashing, though squashing will provide an immediate fix.

Squashing

Each of those layers adds and subtracts some files from the image. Basically, squashing takes all of the files which are added, and creates a single layer which represents all of the additions. This defeats Docker's strategy for intelligently diffing layers and only sending the layers that change in an image, so it's not widely used.

There are various ways of squashing an image.

1. You can do this with a tool called docker-squash.
2. You can run docker export <image hash> | docker import -. This will export the root filesystem of the image as a tarball, then import that image again. This will lose various configuration data, such as volumes and exposed ports. YMMV.

Reverse Engineering

You can run the following command:

docker history <image hash> --format "{{.CreatedBy}}" --no-trunc | tac

This will give you the commands used to create the image. In the case where you had files or directories copied into the image, it will just show you the hash and the destination path, though, so you'll have to do some work to figure out what that's doing.

You could also use dive to figure out what each layer is doing.