pip install with cert works but reference cert in global config file fails

Question

I'm behind a corporate network and can currently install Python packages from a Dockerfile in 2 ways.

(1) referencing the certificate

RUN pip3 install --cert=/usr/local/share/ca-certificates/zscaler-root.crt <SOME-PYTHON-PACKAGE>

(2) by adding repos/URLs to trusted hosts

RUN touch /etc/pip.conf

RUN echo "[global] \n trusted-host = pypi.python.org pypi.org files.pythonhosted.org" >> /etc/pip.conf

RUN pip3 install <SOME-PYTHON-PACKAGE>

Instead of option #2, I would prefer to set the cert globally.

RUN touch /etc/pip.conf

RUN echo "[global] \n cert=/usr/local/share/ca-certificates/zscaler-root.crt" >> /etc/pip.conf

But this does not work.

Results in a SSLError(SSLCertVerificationError.

WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)'))': /simple/flask/

How come option #1 can reference the cert but adding it to global config file fails ?

EDIT:

As per @phd's suggestion tried the following.

RUN pip3 config --global set global.cert /usr/local/share/ca-certificates/zscaler-root.crt

RUN pip3 config set global.cert /usr/local/share/ca-certificates/zscaler-root.crt

But no joy.

Also tried setting SSL_CERT_DIR per https://stackoverflow.com/a/24353642/6265370.

ENV SSL_CERT_DIR="/usr/local/share/ca-certificates/"

Again no cigar.

Here the some more info

pip 21.1.2 from /usr/local/lib/python3.7/site-packages/pip (python 3.7)

        PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
        NAME="Debian GNU/Linux"
        VERSION_ID="9"
        VERSION="9 (stretch)"
        VERSION_CODENAME=stretch
        ID=debian
        HOME_URL="https://www.debian.org/"
        SUPPORT_URL="https://www.debian.org/support"
        BUG_REPORT_URL="https://bugs.debian.org/"
    
        pip3 config list -v
        For variant 'global', will try loading '/etc/xdg/pip/pip.conf'
        For variant 'global', will try loading '/etc/pip.conf'
        For variant 'user', will try loading '/root/.pip/pip.conf'
        For variant 'user', will try loading '/root/.config/pip/pip.conf'
        For variant 'site', will try loading '/usr/local/pip.conf'

Use [`pip config`](https://pip.pypa.io/en/stable/cli/pip_config/) instead of `echo`. — phd, Jun 29 '21 at 14:23
You may check this thread https://stackoverflow.com/questions/25981703/pip-install-fails-with-connection-error-ssl-certificate-verify-failed-certi?rq=1 — nityanarayan44, Jun 30 '21 at 10:42
Thanks @nityanarayan44 but I tried many of proposed solutions on that SO question. — gary, Jun 30 '21 at 11:52

score 1 · Accepted Answer · answered Jul 02 '21 at 12:01

Managed to figure this out with a little help from posted suggestions/other SO questions and some good old fashioned trail and error.

The solution is to set the global cert to ca-certificates.crt.

Here the full code.

RUN wget --no-check-certificate https://<URL>/zscaler-crts.zip && \
    unzip zscaler-crts.zip && cp zscaler-root.crt /usr/local/share/ca-certificates/ && \
    update-ca-certificates

RUN pip3 config set global.cert /usr/lib/ssl/certs/ca-certificates.crt

(And with pip3 config there is no need to create the config file and write to it.)

My zscaler-root.crt is picked up when update-ca-certificates is executed. See man page description.

update-ca-certificates  is a program that updates the directory /etc/ssl/certs to hold SSL
certificates  and  generates  ca-certificates.crt,  a  concatenated  single-file  list  of
certificates.

It  reads  the  file  /etc/ca-certificates.conf.  Each  line  gives  a  pathname  of  a CA
certificate under /usr/share/ca-certificates that should be  trusted.   Lines  that  begin
with  "#"  are  comment lines and thus ignored.  Lines that begin with "!" are deselected,
causing the deactivation of the CA certificate in question. Certificates must have a  .crt
extension in order to be included by update-ca-certificates.

Furthermore  all  certificates  with  a  .crt  extension  found below /usr/local/share/ca-
certificates are also included as implicitly trusted.

score 0 · Answer 2 · answered Jun 30 '21 at 13:23

It is best to install the ZScaler certs system wide instead limiting the to the use of pip. Linx distros relay on openssl. First check the openssl version and certs directory:

openssl version -d

You will get something like:

OPENSSLDIR: "/usr/lib/ssl"

And then copy the certificate into the certs directory within it.

In this case it would be:

/usr/lib/ssl/certs

Alternatively, you could do it through the ca trust authority. In Ubuntu:

sudo mkdir /usr/local/share/ca-certificates/zscaler
sudo cp <certfile.crt> /usr/local/share/ca-certificates/zscaler/
sudo chmod 755 /usr/local/share/ca-certificates/zscaler/
sudo chmod 644 /usr/local/share/ca-certificates/zscaler/<certfile.crt>
sudo update-ca-certificates

Thanks @Malvido but both suggestions didn't work. – gary Jun 30 '21 at 14:40 — gary, Jun 30 '21 at 14:40

pip install with cert works but reference cert in global config file fails

2 Answers2