#StackBounty: #python-3.x #pip How to determine where pip3 installs binaries/scripts for non-user installs?

Bounty: 150

I can determine where pip installs package binaries/scripts with pip3 install --user mypackage using:

$(python3 -m site --user-base)/bin

Is there a mechanism for determining where non-user binaries/scripts get installed (e.g. pip3 install mypackage) get installed?

Note that I’m only interested in where the binaries/scripts get installed and NOT where the library code gets installed.

Update: the objective is to ensure that that the PATH environment variable contains the folder where the binaries/scripts get installed as this has caused some issues in the past for me.


Get this bounty!!!

#StackBounty: #python #docker #pip #ssl-certificate Pip upgrade cannot install packages

Bounty: 150

I have jumped around for some time now to solve this, and I cannot seem to get it working. I have a docker container where I set up an nvidia image for machine learning. I install all python dependencies. I then start with the pip package installations. I get the first error:

requests.exceptions.SSLError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Max retries exceeded with url: /packages/5e/c4/6c4fe722df5343c33226f0b4e0bb042e4dc13483228b4718baf286f86d87/certifi-2020.6.20-py2.py3-none-any.whl (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),))

Simple enough I have a certificate to deal with Cisco umbrella. I can then install all packages nice and easy. However to be able to install newest packages I need to upgrade pip, and upgrading works fine. After pip is upgraded to 20.2.3 I suddenly get an error again:

Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)) - skipping

I have then googled around and tried the suggestions I stumbled upon:

Timing

I found that the system time was wrong – it worked for the initial pip version, which was weird. However changing the time did not help the issue.

conf

I added a pip.conf file with global tags for trusted hosts and for certifications. Still the same error persists.

pip install

I have tried with different trusted host flags and also the cert flag, which should already be specified from the conf file – if I understand it correctly. Nevertheless, neither method worked.

What to do

I am kind of at a loss right now, installing the certificate in the container allows me to install packages with pip 9.0.1 (default in the system) after upgrading to pip 20.2.3. I cannot get it to work with any package. I have tried multiple pip versions – but as soon as I upgrade I lose the certificate trying to reinstall it with

ADD Cisco_Umbrella_Root_CA.cer /usr/local/share/ca-certificates/Cisco_Umbrella_Root_CA.crt
RUN chmod 644 /usr/local/share/ca-certificates/Cisco_Umbrella_Root_CA.crt
RUN update-ca-certificates --fresh

Anybody has an idea how this can happen?

UPDATE

Curl

 RUN curl -v -k -H"Host; files.pythonhosted.org" https://files.pythonhosted.org/packages/8a/fd/bbbc569f98f47813c50a116b539d97b3b17a86ac7a309f83b2022d26caf2/Pillow-6.2.2-cp36-cp36m-manylinux1_x86_64.whl
  ---> Running in ac095828b9ec
   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                  Dload  Upload   Total   Spent    Left  Speed
   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying ::ffff:146.112.56.166...
 * TCP_NODELAY set
 * Connected to files.pythonhosted.org (::ffff:146.112.56.166) port 443 (#0)
 * ALPN, offering h2
 * ALPN, offering http/1.1
 * successfully set certificate verify locations:
 *   CAfile: /etc/ssl/certs/ca-certificates.crt
   CApath: /etc/ssl/certs
 } [5 bytes data]
 * TLSv1.3 (OUT), TLS handshake, Client hello (1):
 } [512 bytes data]
 * TLSv1.3 (IN), TLS handshake, Server hello (2):
 { [85 bytes data]
 * TLSv1.2 (IN), TLS handshake, Certificate (11):
 { [3177 bytes data]
 * TLSv1.2 (IN), TLS handshake, Server finished (14):
 { [4 bytes data]
 * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
 } [262 bytes data]
 * TLSv1.2 (OUT), TLS change cipher, Client hello (1):
 } [1 bytes data]
 * TLSv1.2 (OUT), TLS handshake, Finished (20):
 } [16 bytes data]
 * TLSv1.2 (IN), TLS handshake, Finished (20):
 { [16 bytes data]
 * SSL connection using TLSv1.2 / AES256-GCM-SHA384
 * ALPN, server did not agree to a protocol

From the last line it can be seen that they do not agree on protocol and the communication fails


Get this bounty!!!

#StackBounty: #python #amazon-ec2 #pip #amazon #setuptools Why does `pip` throw `ModuleNotFoundError` even though `setuptools` appears …

Bounty: 50

On an EC2 instance, I am attempting to install a Python package using pip, but receive an exception that states ModuleNotFoundError: No module named 'setuptools'.

For context, the command I run is

pip install --user mlflow

I attempt to resolve the issue my installing setuptools with the following command: sudo yum install python36-setuptools. Yet, the message returned indicates that setuptools is already installed:

Package python36-setuptools-36.2.7-1.33.amzn1.noarch already installed and latest version
Nothing to do

Please advise.

UPDATE: After switching the EC2 instance from Amazon Linux AMI 2018.03.0 to Amazon Linux 2 AMI I was able to successfully install mlflow using pip.

The question still remains – why was I able to successfully install using Amazon Linux 2 AMI but not Amazon Linux AMI 2018.03.0?


Get this bounty!!!

#StackBounty: #upgrade #python #sudo #pip Are two instances of pip & youtube-dl installed after using sudo to update?

Bounty: 50

I always used pip3 install --upgrade pip && pip3 install --upgrade youtube-dl to update pip and the relevant program youtube-dl. However, I started getting the warning:
Defaulting to user installation because normal site-packages is not writeable
Requirement already up-to-date:...

First, I tried only typing pip instead of instances of pip3 and subsequently only pip, but those attempts resulted in ‘real’ errors.
Then I went for sudo pip3 install --upgrade pip && sudo pip3 install --upgrade youtube-dl.
Both programmes were downloaded and installed, but the version of youtube-dl that was replaced was so old that I’m sure it’s not the only which I regularly updated using the command mentioned in the first line.
Now, I suspect I have installed two instances of each of those programmes, one root and one not. Is that true? If so, how do I get rid of one – preferably the root one since sudo privileges are not necessary, I believe.

And off topic: What causes the original warning and how do I fix it instead?


Get this bounty!!!

#StackBounty: #pip #console-application #setup.py #pkg-resources fragile console script: pkg_resources.DistributionNotFound

Bounty: 50

If I create console_scripts via entry_point in setup.py, then this file gets created:

#!/home/myproject_cok_d/bin/python
# EASY-INSTALL-ENTRY-SCRIPT: 'mylib','console_scripts','do-magic'
__requires__ = 'mylib'
import re
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script.pyw?|.exe)?$', '', sys.argv[0])
    sys.exit(
        load_entry_point('mylib', 'console_scripts', 'do-magic')()
    )

Unfortunately this script is fragile.

Sometimes I get exceptions like this:

Traceback (most recent call last):
  File ".../bin/do-magic", line 6, in <module>
    from pkg_resources import load_entry_point
  File ".../local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2999, in <module>
    @_call_aside
  File ".../local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2985, in _call_aside
    f(*args, **kwargs)
  File ".../local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3012, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File ".../local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 662, in _build_master
    ws.require(__requires__)
  File ".../local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 970, in require
    needed = self.resolve(parse_requirements(requirements))
  File ".../local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 856, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'PyPDF2==1.26' distribution was not found and is required by mylib

Is there a way to make my script (do-magic) more mature?

In this case: It does not need PyPDF2.


Get this bounty!!!