#StackBounty: #linux #virtualbox #ssl #openssl #trusted-root-certificates SSL handshake keeps failing even after adding certificates to…

Bounty: 50

When executing

wget https://docs.conda.io/projects/conda/en/4.6.0/_downloads/52a95608c49671267e40c689e0bc00ca/conda-cheatsheet.pdf

I have this error:

--2020-06-03 20:55:06--  https://docs.conda.io/projects/conda/en/4.6.0/_downloads/52a95608c49671267e40c689e0bc00ca/conda-cheatsheet.pdf
Resolving docs.conda.io (docs.conda.io)... 104.31.71.166, 104.31.70.166, 172.67.149.185, ...
Connecting to docs.conda.io (docs.conda.io)|104.31.71.166|:443... connected.
ERROR: cannot verify docs.conda.io's certificate, issued by ‘CN=SSL-SG1-GFRPA2,OU=Operations,O=Cloud Services,C=US’:
  Unable to locally verify the issuer's authority.
To connect to docs.conda.io insecurely, use `--no-check-certificate'.

The certificates chain in the URL above contains 4 certificates.

What I have tried to solve this problem:

0) Extract the 4 certificates in the chain, from chrome when opening the url

1) Just to ensure not missing certificates, I put all the 4 certificates (namely conda1.crt, conda2.crt, conda3.crt, conda4.crt) in /usr/share/ca-certificates/mozilla/ by doing sudo cp conda*.crt /usr/share/ca-certificates/mozilla/

2) sudo vi /etc/ca-certificates.conf and append mozilla/conda1.crt, mozilla/conda2.crt, mozilla/conda3.crt, mozilla/conda4.crt at the end

3) run sudo update-ca-certificates -f

4) I can see symbolic link created under /etc/ssl/certs which looks like: conda1.pem -> /usr/share/ca-certificates/mozilla/conda1.crt, conda2.pem -> /usr/share/ca-certificates/mozilla/conda2.crt, etc.

Verification:

openssl verify -no-CAfile -no-CApath -partial_chain -CAfile conda1.pem conda2.pem
conda2.pem: OK

openssl verify -no-CAfile -no-CApath -partial_chain -CAfile conda2.pem conda3.pem
conda3.pem: OK

openssl verify -no-CAfile -no-CApath -partial_chain -CAfile conda3.pem conda4.pem
conda4.pem: OK

Result: still fail with wget

P.S.
I am facing this ssl problem in many aspects and many urls since a month ago (no problem before):

  1. I cannot do conda search a_package
  2. I cannot do requests.get(url) in python code
  3. I cannot open it in a browser within my ubuntu system (can only access in windows)
  4. I cannot do fromUrl in scala

It seems the problem is not only due to one or two certificates, instead, it’s a systematic problem in my ubuntu system. Looks like it’s missing a list of certificates in my truststore.

uname => Linux user 5.3.0-53-generic #47~18.04.1-Ubuntu SMP Thu May 7 13:10:50 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

I’m using Oracle VirtualBox.

UPDATE1

For conda1.crt:

openssl x509 -noout -text < conda1.crt
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            1b:b7:86:d3:b6:ad:8f:65:b9:7a:79:3e:c7:48:84:27
        Signature Algorithm: sha1WithRSAEncryption
        Issuer: C = US, O = "BlueCoat Systems, Inc.", CN = Cloud Services Root CA
        Validity
            Not Before: Sep  6 00:00:00 2011 GMT
            Not After : Sep  5 23:59:59 2021 GMT
        Subject: C = US, O = "BlueCoat Systems, Inc.", CN = Cloud Services Root CA
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    00:c4:00:7b:f6:a2:29:37:43:40:a5:44:b4:6d:ed:
                    0d:15:80:ea:9d:8d:e0:f6:32:6c:61:9e:87:55:1b:
                    1b:c3:67:89:9c:ed:81:29:88:68:04:e5:b9:7e:65:
                    1c:f4:56:93:d1:56:e1:22:89:07:15:18:f8:c3:77:
                    36:91:e5:95:81:39:45:1d:ba:7a:11:96:9a:2b:51:
                    fc:c9:cc:d3:7f:9e:d6:95:72:0b:b8:2a:c9:f5:e1:
                    98:b1:61:36:76:82:5e:3e:71:69:4f:54:1e:8c:34:
                    50:60:c2:93:8c:07:d0:03:4b:70:08:14:b1:c6:66:
                    79:4f:31:09:ff:10:2e:e1:c6:13:73:70:a7:32:b8:
                    00:de:7f:bf:b5:c1:fb:62:7e:4f:0c:d1:80:8b:06:
                    4c:59:fe:4e:3d:b9:2d:1f:7d:db:da:be:f2:7b:1f:
                    9b:81:75:e2:bd:8d:4c:c3:a9:3c:d9:16:0b:4c:b4:
                    6c:6b:c0:28:96:e0:43:4e:99:6a:31:b1:e8:d5:01:
                    3b:02:eb:de:78:59:0b:2f:91:97:5f:ff:14:c5:aa:
                    34:98:1b:ee:77:63:49:08:74:d9:f4:47:32:1e:7e:
                    7f:63:68:27:a8:95:b8:b6:66:cc:35:7a:eb:84:01:
                    3e:e5:8d:5d:58:c0:14:f1:01:52:17:46:ac:cd:04:
                    04:db
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Certificate Sign, CRL Sign
            X509v3 Basic Constraints: critical
                CA:TRUE
            X509v3 Subject Alternative Name:
                DirName:/CN=MPKI-2048-1-99
            X509v3 Subject Key Identifier:
                A6:4A:17:D1:BC:58:B5:77:25:16:92:2B:D2:4C:95:23:CF:28:14:36
    Signature Algorithm: sha1WithRSAEncryption
         8c:f8:95:4c:29:f3:4d:4c:a0:32:dc:68:0e:9e:83:03:26:a6:
         a6:66:07:1d:bc:ef:0f:89:d7:60:df:77:ce:7b:a0:1d:e8:76:
         ac:e6:02:86:4d:cc:4a:d1:ff:73:64:68:cb:15:f7:84:f4:fc:
         df:5c:d0:eb:9c:ca:f9:06:76:97:b9:1c:da:33:a0:38:b6:2c:
         78:89:d0:12:35:19:cc:4c:1e:78:03:4d:f8:31:dd:33:8b:69:
         a8:69:52:c7:34:2f:20:33:2d:53:c2:f4:ff:5f:c2:98:19:fb:
         ca:19:1f:7a:4c:84:c6:9c:7d:18:03:59:8f:a1:9a:bc:dd:64:
         fe:cc:7e:16:7b:59:73:e6:64:a0:60:cf:38:64:f7:4f:33:fd:
         9d:86:8e:5f:78:cd:09:ba:31:a1:06:24:d3:af:cb:fd:df:ba:
         c6:ac:84:37:b1:61:2a:32:02:48:59:66:4b:27:f1:9e:bf:1f:
         9a:45:a4:0d:48:42:42:d7:13:f8:55:7a:33:2c:a7:6c:5e:ba:
         b6:27:8f:5f:72:0a:45:aa:24:bc:a1:d5:f6:68:30:c4:9f:01:
         5d:c3:a5:c0:4c:0e:93:0f:f1:4d:e2:cb:41:e0:76:97:6e:f8:
         ac:f9:1d:9b:06:8f:e6:a9:c7:dd:df:73:57:37:c6:f8:8d:bc:
         07:01:ff:ad

For conda4.crt:

openssl x509 -noout -text < conda4.crt
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            f1:e0:c2:3f:00:00:00:00
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = US, O = Cloud Services, OU = Operations, CN = SSL-SG1-GFRPA2
        Validity
            Not Before: Jan 31 00:00:00 2020 GMT
            Not After : Oct  9 12:00:00 2020 GMT
        Subject: C = US, ST = CA, L = San Francisco, O = "Cloudflare, Inc.", CN = sni.cloudflaressl.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    00:d8:31:07:5c:6d:c6:b3:4b:79:60:2f:87:14:39:
                    97:ca:0b:d1:ea:a2:a9:89:7a:2c:a6:11:16:aa:38:
                    0f:ac:11:11:96:da:ae:ab:27:7c:7f:6c:ff:bd:35:
                    67:29:a2:26:fa:85:96:1d:97:ff:b1:3e:ca:81:eb:
                    13:50:cd:55:f2:47:c2:ea:a4:c9:9c:5c:0e:3f:46:
                    9e:65:4a:a3:fb:58:3d:7b:de:1c:2e:a1:d2:82:66:
                    a4:6d:79:d6:23:8d:0e:cb:1c:80:4e:f9:99:8c:dc:
                    c1:84:e3:15:c5:0f:b2:e0:83:a4:78:a6:d3:76:b6:
                    07:85:ff:6f:ee:69:71:80:41:54:75:ee:2d:c6:68:
                    de:e3:87:87:13:88:1b:1e:bd:d0:14:b0:49:7e:90:
                    b6:b4:5f:c2:ff:ff:0b:fe:fe:a4:70:01:da:1f:8f:
                    5b:50:80:be:16:c6:8e:1a:b5:9e:e5:c2:9a:01:09:
                    10:6b:c2:2d:16:15:c3:cf:0d:a7:0c:e1:56:17:9e:
                    ca:bf:f6:db:dd:51:30:02:d9:b9:11:ca:6f:ac:ec:
                    ab:c0:a4:17:2b:8c:ad:60:4d:67:e4:a5:97:4d:b2:
                    e7:cc:06:59:89:2b:bf:77:9e:d2:44:5d:79:d6:38:
                    03:9f:fe:55:cb:fa:7b:0e:75:d4:5d:6c:e9:1e:f2:
                    b2:a7
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints: 
                CA:FALSE
            X509v3 Key Usage: 
                Digital Signature, Non Repudiation, Key Encipherment
            X509v3 Authority Key Identifier: 
                keyid:80:69:47:45:27:B6:26:29:03:06:1E:01:BC:42:A1:9C:DE:C1:94:A6

            X509v3 Subject Alternative Name: 
                DNS:conda.io, DNS:*.conda.io, DNS:sni.cloudflaressl.com
            Netscape Comment: 
                090560AE68F2769F04BBD27072BD6E3EJan 31 00:00:00 2020 GMTOct  9 12:00:00 2020 GMT
            X509v3 Certificate Policies: 
                Policy: X509v3 Any Policy
                  User Notice:
                    Explicit Text: 090560AE68F2769F04BBD27072BD6E3EJan 31 00:00:00 2020 GMTOct  9 12:00:00 2020 GMT

    Signature Algorithm: sha256WithRSAEncryption
         13:92:fe:3e:d2:d5:35:5b:6e:5a:d3:97:24:ea:f3:92:fe:84:
         cb:da:0f:b0:77:e9:fc:29:75:3e:03:72:ad:5f:6d:49:98:c8:
         6d:15:90:19:13:31:5a:bc:98:01:0c:cb:33:cf:2f:b4:52:a7:
         73:e9:70:cc:5d:e4:12:0a:af:e0:71:15:20:cf:1c:fa:1a:3e:
         68:dc:7d:90:95:b6:b8:b9:54:51:e2:49:4a:80:43:3c:e2:b8:
         e6:98:db:28:57:72:28:e7:b3:cc:a3:25:80:00:11:1f:d7:8a:
         90:a3:97:a4:7a:67:95:91:9f:1d:22:18:ce:42:56:1b:80:e2:
         e1:75:34:8c:6f:02:b9:ff:04:13:86:ad:b0:31:bd:15:6f:1e:
         2d:11:21:82:45:57:0e:df:6e:9e:e0:98:af:b8:54:a4:7f:49:
         20:5a:b2:72:57:a8:55:00:8d:be:e4:3e:b3:90:6b:3c:d1:fc:
         a7:1b:2f:5a:b0:f6:c6:b8:f3:da:d9:05:9e:d4:4d:c3:be:05:
         36:c6:78:cc:d5:b8:e3:28:40:2f:02:0a:e4:d2:1b:be:69:9a:
         e3:f1:33:34:21:ce:39:3e:42:d7:f0:7d:5b:5c:5e:8b:aa:49:
         e7:80:07:dd:e1:80:2f:57:3b:c6:d4:22:55:6f:ad:10:e3:51:
         90:e6:c4:4b

UPDATE2

For /etc/ssl/certs/ca-certificates.crt:

openssl x509 -noout -text < /etc/ssl/certs/ca-certificates.crt
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 6828503384748696800 (0x5ec3b7a6437fa4e0)
        Signature Algorithm: sha1WithRSAEncryption
        Issuer: CN = ACCVRAIZ1, OU = PKIACCV, O = ACCV, C = ES
        Validity
            Not Before: May  5 09:37:37 2011 GMT
            Not After : Dec 31 09:37:37 2030 GMT
        Subject: CN = ACCVRAIZ1, OU = PKIACCV, O = ACCV, C = ES
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (4096 bit)
                Modulus:
                    00:9b:a9:ab:bf:61:4a:97:af:2f:97:66:9a:74:5f:
                    d0:d9:96:fd:cf:e2:e4:66:ef:1f:1f:47:33:c2:44:
                    a3:df:9a:de:1f:b5:54:dd:15:7c:69:35:11:6f:bb:
                    c8:0c:8e:6a:18:1e:d8:8f:d9:16:bc:10:48:36:5c:
                    f0:63:b3:90:5a:5c:24:37:d7:a3:d6:cb:09:71:b9:
                    f1:01:72:84:b0:7d:db:4d:80:cd:fc:d3:6f:c9:f8:
                    da:b6:0e:82:d2:45:85:a8:1b:68:a8:3d:e8:f4:44:
                    6c:bd:a1:c2:cb:03:be:8c:3e:13:00:84:df:4a:48:
                    c0:e3:22:0a:e8:e9:37:a7:18:4c:b1:09:0d:23:56:
                    7f:04:4d:d9:17:84:18:a5:c8:da:40:94:73:eb:ce:
                    0e:57:3c:03:81:3a:9d:0a:a1:57:43:69:ac:57:6d:
                    79:90:78:e5:b5:b4:3b:d8:bc:4c:8d:28:a1:a7:a3:
                    a7:ba:02:4e:25:d1:2a:ae:ed:ae:03:22:b8:6b:20:
                    0f:30:28:54:95:7f:e0:ee:ce:0a:66:9d:d1:40:2d:
                    6e:22:af:9d:1a:c1:05:19:d2:6f:c0:f2:9f:f8:7b:
                    b3:02:42:fb:50:a9:1d:2d:93:0f:23:ab:c6:c1:0f:
                    92:ff:d0:a2:15:f5:53:09:71:1c:ff:45:13:84:e6:
                    26:5e:f8:e0:88:1c:0a:fc:16:b6:a8:73:06:b8:f0:
                    63:84:02:a0:c6:5a:ec:e7:74:df:70:ae:a3:83:25:
                    ea:d6:c7:97:87:93:a7:c6:8a:8a:33:97:60:37:10:
                    3e:97:3e:6e:29:15:d6:a1:0f:d1:88:2c:12:9f:6f:
                    aa:a4:c6:42:eb:41:a2:e3:95:43:d3:01:85:6d:8e:
                    bb:3b:f3:23:36:c7:fe:3b:e0:a1:25:07:48:ab:c9:
                    89:74:ff:08:8f:80:bf:c0:96:65:f3:ee:ec:4b:68:
                    bd:9d:88:c3:31:b3:40:f1:e8:cf:f6:38:bb:9c:e4:
                    d1:7f:d4:e5:58:9b:7c:fa:d4:f3:0e:9b:75:91:e4:
                    ba:52:2e:19:7e:d1:f5:cd:5a:19:fc:ba:06:f6:fb:
                    52:a8:4b:99:04:dd:f8:f9:b4:8b:50:a3:4e:62:89:
                    f0:87:24:fa:83:42:c1:87:fa:d5:2d:29:2a:5a:71:
                    7a:64:6a:d7:27:60:63:0d:db:ce:49:f5:8d:1f:90:
                    89:32:17:f8:73:43:b8:d2:5a:93:86:61:d6:e1:75:
                    0a:ea:79:66:76:88:4f:71:eb:04:25:d6:0a:5a:7a:
                    93:e5:b9:4b:17:40:0f:b1:b6:b9:f5:de:4f:dc:e0:
                    b3:ac:3b:11:70:60:84:4a:43:6e:99:20:c0:29:71:
                    0a:c0:65
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            Authority Information Access: 
                CA Issuers - URI:http://www.accv.es/fileadmin/Archivos/certificados/raizaccv1.crt
                OCSP - URI:http://ocsp.accv.es

            X509v3 Subject Key Identifier: 
                D2:87:B4:E3:DF:37:27:93:55:F6:56:EA:81:E5:36:CC:8C:1E:3F:BD
            X509v3 Basic Constraints: critical
                CA:TRUE
            X509v3 Authority Key Identifier: 
                keyid:D2:87:B4:E3:DF:37:27:93:55:F6:56:EA:81:E5:36:CC:8C:1E:3F:BD

            X509v3 Certificate Policies: 
                Policy: X509v3 Any Policy
                  User Notice:
                    Explicit Text: 
                  CPS: http://www.accv.es/legislacion_c.htm

            X509v3 CRL Distribution Points: 

                Full Name:
                  URI:http://www.accv.es/fileadmin/Archivos/certificados/raizaccv1_der.crl

            X509v3 Key Usage: critical
                Certificate Sign, CRL Sign
            X509v3 Subject Alternative Name: 
                email:accv@accv.es
    Signature Algorithm: sha1WithRSAEncryption
         97:31:02:9f:e7:fd:43:67:48:44:14:e4:29:87:ed:4c:28:66:
         d0:8f:35:da:4d:61:b7:4a:97:4d:b5:db:90:e0:05:2e:0e:c6:
         79:d0:f2:97:69:0f:bd:04:47:d9:be:db:b5:29:da:9b:d9:ae:
         a9:99:d5:d3:3c:30:93:f5:8d:a1:a8:fc:06:8d:44:f4:ca:16:
         95:7c:33:dc:62:8b:a8:37:f8:27:d8:09:2d:1b:ef:c8:14:27:
         20:a9:64:44:ff:2e:d6:75:aa:6c:4d:60:40:19:49:43:54:63:
         da:e2:cc:ba:66:e5:4f:44:7a:5b:d9:6a:81:2b:40:d5:7f:f9:
         01:27:58:2c:c8:ed:48:91:7c:3f:a6:00:cf:c4:29:73:11:36:
         de:86:19:3e:9d:ee:19:8a:1b:d5:b0:ed:8e:3d:9c:2a:c0:0d:
         d8:3d:66:e3:3c:0d:bd:d5:94:5c:e2:e2:a7:35:1b:04:00:f6:
         3f:5a:8d:ea:43:bd:5f:89:1d:a9:c1:b0:cc:99:e2:4d:00:0a:
         da:c9:27:5b:e7:13:90:5c:e4:f5:33:a2:55:6d:dc:e0:09:4d:
         2f:b1:26:5b:27:75:00:09:c4:62:77:29:08:5f:9e:59:ac:b6:
         7e:ad:9f:54:30:22:03:c1:1e:71:64:fe:f9:38:0a:96:18:dd:
         02:14:ac:23:cb:06:1c:1e:a4:7d:8d:0d:de:27:41:e8:ad:da:
         15:b7:b0:23:dd:2b:a8:d3:da:25:87:ed:e8:55:44:4d:88:f4:
         36:7e:84:9a:78:ac:f7:0e:56:49:0e:d6:33:25:d6:84:50:42:
         6c:20:12:1d:2a:d5:be:bc:f2:70:81:a4:70:60:be:05:b5:9b:
         9e:04:44:be:61:23:ac:e9:a5:24:8c:11:80:94:5a:a2:a2:b9:
         49:d2:c1:dc:d1:a7:ed:31:11:2c:9e:19:a6:ee:e1:55:e1:c0:
         ea:cf:0d:84:e4:17:b7:a2:7c:a5:de:55:25:06:ee:cc:c0:87:
         5c:40:da:cc:95:3f:55:e0:35:c7:b8:84:be:b4:5d:cd:7a:83:
         01:72:ee:87:e6:5f:1d:ae:b5:85:c6:26:df:e6:c1:9a:e9:1e:
         02:47:9f:2a:a8:6d:a9:5b:cf:ec:45:77:7f:98:27:9a:32:5d:
         2a:e3:84:ee:c5:98:66:2f:96:20:1d:dd:d8:c3:27:d7:b0:f9:
         fe:d9:7d:cd:d0:9f:8f:0b:14:58:51:9f:2f:8b:c3:38:2d:de:
         e8:8f:d6:8d:87:a4:f5:56:43:16:99:2c:f4:a4:56:b4:34:b8:
         61:37:c9:c2:58:80:1b:a0:97:a1:fc:59:8d:e9:11:f6:d1:0f:
         4b:55:34:46:2a:8b:86:3b

Both of these works:

wget --ca-certificates=/etc/ssl/certs/ca-certificates.crt https://docs.conda.io/projects/conda/en/4.6.0/_downloads/52a95608c49671267e40c689e0bc00ca/conda-cheatsheet.pdf

wget --ca-certificates=conda1.crt https://docs.conda.io/projects/conda/en/4.6.0/_downloads/52a95608c49671267e40c689e0bc00ca/conda-cheatsheet.pdf

UPDATE3
Regarding VM network setting:
enter image description here

Part of the cause is found

Bluecoat service which intercepts the network is the root cause (it has problem to VM Ubuntu only though, the host machine windows works fine with ssl).

However, I have not figured out how to solve this Bluecoat problem. Any help is really appreciated!


Get this bounty!!!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.