Building Python and OpenSSL from source, but ssl module fails

Question

I'm trying to build Python and OpenSSL from source in a container. Both seem to build correctly, but Python does not successfully create the _ssl module.

I've found a few guides online that say to un-comment and lines from Python-3.X.X/Modules/Setup and add the --openssldir=/usr/local/ssl flag to the ./configure step for OpenSSL. I do these in my dockerfile. This has had the effect that, during the ./configure output for Python, I see the following line.

checking for X509_VERIFY_PARAM_set1_host in libssl... yes

Yet I receive the following errors:

[91m*** WARNING: renaming "_ssl" since importing it failed: /usr/lib/x86_64-linux-gnu/libssl.so.1.1: version `OPENSSL_1_1_1' not found (required by build/lib.linux-x86_64-3.8/_ssl.cpython-38-x86_64-linux-gnu.so)
[0m[91m*** WARNING: renaming "_hashlib" since importing it failed: /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1: version `OPENSSL_1_1_1' not found (required by build/lib.linux-x86_64-3.8/_hashlib.cpython-38-x86_64-linux-gnu.so)
[0m
Python build finished successfully!

...

Following modules built successfully but were removed because they could not be imported:
_hashlib              _ssl                                     


Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381

If ./configure finds X509..., why am I still getting the hashlib and ssl errors?

The full Dockerfile, FWIW:

FROM jenkins/jenkins:lts
USER root
RUN apt-get update && apt-get install -y apt-utils gcc make zlib1g-dev \
    build-essential libffi-dev checkinstall libsqlite3-dev 
RUN wget https://www.openssl.org/source/openssl-1.1.1d.tar.gz && \
    tar xzf openssl-1.1.1d.tar.gz && \
    cd openssl-1.1.1d && \
    ./config -Wl,--enable-new-dtags,-rpath,'$(LIBRPATH)' --prefix=/usr/local/ssl --openssldir=/usr/local/ssl && \
    make && \
    make test && \
    make install
RUN wget -q https://www.python.org/ftp/python/3.8.2/Python-3.8.2.tgz && \
    tar -xzf Python-3.8.2.tgz && \
    cd Python-3.8.2 && \
    ./configure && \
    make && \
    make install
USER jenkins

Hi Jake, why can't you add that line in `Python/Modules/Setup`? You could do a `sed` with in place changes to the file. — ckaserer, Mar 05 '20 at 05:42
Trying... `sed -i 's|# -DUSE_SSL| -DUSE_SSL| w /dev/stdout' Python-3.8.2/Modules/Setup`. On my machine it works, on the dockerfile it doesn't. — Jake Stevens-Haas, Mar 05 '20 at 20:17
And even getting it to work on the Dockerfile gives: ```[91m./python: /usr/lib/x86_64-linux-gnu/libssl.so.1.1: version `OPENSSL_1_1_1' not found (required by ./python) [0mgenerate-posix-vars failed Makefile:592: recipe for target 'pybuilddir.txt' failed [91mmake: *** [pybuilddir.txt] Error 1 [0m``` — Jake Stevens-Haas, Mar 05 '20 at 20:36

Max-Florian Luchterhand · Answer 1 · 2020-03-05T06:32:59.263

1

I reckon that Jenkins Image comes with some openssl version installed that is not 1.1.1, hence you find X509... in libssl but cant build.

Regarding said config option, you can spin up the container with bash as CMD, copy the config from within the container to the machine where the Image lies, edit ist and bake your version of the config into the Image.

edited Mar 05 '20 at 06:32

answered Mar 05 '20 at 06:27

Max-Florian Luchterhand

408
3
9

Yes, but the point of a Dockerfile is to not need those steps so it can be distributed and reused... – Jake Stevens-Haas Mar 05 '20 at 20:23
After you have baked your config into the image, you have a working version of the image, which can be redistributed and reused – Max-Florian Luchterhand Mar 09 '20 at 18:13
1

Maybe I was unclear, I do not mean copy the config into the running container. I mean, copy the config OUT OF the running container, edit in on your local machine, and then create a new image which is based on the image in question (FROM image xyz), and use your config in that image (COPY myconfig /path/to/config/in/image) – Max-Florian Luchterhand Mar 09 '20 at 18:42
Ok, that makes sense. I can edit the Setup file directly in the container, but apparently that doesn't solve the issue. `make` still gives me an error: `./python: /usr/lib/x86_64-linux-gnu/libssl.so.1.1: version \`OPENSSL_1_1_1' not found (required by ./python)`. Any ideas? – Jake Stevens-Haas Mar 09 '20 at 18:43
I could not reproduce your issue, your Dockerfile works for me – Max-Florian Luchterhand Mar 10 '20 at 09:47

score 1 · Answer 2 · answered Apr 08 '20 at 04:41

Following modules built successfully but were removed because they could not be imported:
_hashlib              _ssl                                     

Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381

It seems like installation issue when building openssl from source. For build failure on _ssl module, try extra options like --with-openssl, CFLAGS and LDFLAGS when configuring Python using the script ./configure, e.g.

./configure  --with-openssl=/PATH/TO/YOUR/OPENSSL_INSTALL_FOLDER/ \
    --enable-optimizations \
    --with-ssl-default-suites=openssl \
    CFLAGS="-I/PATH/TO/YOUR/OPENSSL_INSTALL_FODLER/include" \
    LDFLAGS="-L/PATH/TO/YOUR/OPENSSL_INSTALL_FODLER/"

Also try this command openssl version, if it reports error like this :

/usr/lib/x86_64-linux-gnu/libssl.so.1.1: version `OPENSSL_1_1_1' not found

that means there is linking problem on your openssl library, I'm not sure if you're on Linux or other system, but for Linux system, you can manually modify the links to openssl library to fix the problem as described in my answer at here.

Reference

Building Python 3.7.1 - SSL module failed

Python 3.7.0 wont compile with SSL Support 1.1.0

Building Python and OpenSSL from source, but ssl module fails

2 Answers2