#StackBounty: #upgrade #amazon-ec2 #aws AWS EC2 Ubuntu 16.04.7 LTS Upgrade problems

Bounty: 50

I have an AWS EC2 instance that I am running a website on and I am running into issues while upgrading it.

Here are the steps I have tried:

$ sudo apt update
$ sudo apt upgrade

enter image description here

After the first part everything looked good and than I proceeded:

sudo do-release-upgrade

And then we error out:

Traceback (most recent call last):
  File "/tmp/ubuntu-release-upgrader-_29ga8ky/bionic", line 8, in <module>
    sys.exit(main())
  File "/tmp/ubuntu-release-upgrader-_29ga8ky/DistUpgrade/DistUpgradeMain.py", line 238, in main
    if app.run():
  File "/tmp/ubuntu-release-upgrader-_29ga8ky/DistUpgrade/DistUpgradeController.py", line 2072, in run
    return self.fullUpgrade()
  File "/tmp/ubuntu-release-upgrader-_29ga8ky/DistUpgrade/DistUpgradeController.py", line 1981, in fullUpgrade
    if not self.askDistUpgrade():
  File "/tmp/ubuntu-release-upgrader-_29ga8ky/DistUpgrade/DistUpgradeController.py", line 1153, in askDistUpgrade
    changes = self.calcDistUpgrade()
  File "/tmp/ubuntu-release-upgrader-_29ga8ky/DistUpgrade/DistUpgradeController.py", line 1120, in calcDistUpgrade
    if not self.cache.installTasks(self.tasks):
  File "/tmp/ubuntu-release-upgrader-_29ga8ky/DistUpgrade/DistUpgradeCache.py", line 856, in installTasks
    pkg.mark_install()
  File "/usr/lib/python3/dist-packages/apt/package.py", line 1356, in mark_install
    fixer.resolve(True)
SystemError: E:Unable to correct problems, you have held broken packages.
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/problem_report.py", line 497, in add_to_existing
    self.write(f)
  File "/usr/lib/python3/dist-packages/problem_report.py", line 450, in write
    block = f.read(1048576)
  File "/usr/lib/python3.5/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

Original exception was:
Traceback (most recent call last):
  File "/tmp/ubuntu-release-upgrader-_29ga8ky/bionic", line 8, in <module>
    sys.exit(main())
  File "/tmp/ubuntu-release-upgrader-_29ga8ky/DistUpgrade/DistUpgradeMain.py", line 238, in main
    if app.run():
  File "/tmp/ubuntu-release-upgrader-_29ga8ky/DistUpgrade/DistUpgradeController.py", line 2072, in run
    return self.fullUpgrade()
  File "/tmp/ubuntu-release-upgrader-_29ga8ky/DistUpgrade/DistUpgradeController.py", line 1981, in fullUpgrade
    if not self.askDistUpgrade():
  File "/tmp/ubuntu-release-upgrader-_29ga8ky/DistUpgrade/DistUpgradeController.py", line 1153, in askDistUpgrade
    changes = self.calcDistUpgrade()
  File "/tmp/ubuntu-release-upgrader-_29ga8ky/DistUpgrade/DistUpgradeController.py", line 1120, in calcDistUpgrade
    if not self.cache.installTasks(self.tasks):
  File "/tmp/ubuntu-release-upgrader-_29ga8ky/DistUpgrade/DistUpgradeCache.py", line 856, in installTasks
    pkg.mark_install()
  File "/usr/lib/python3/dist-packages/apt/package.py", line 1356, in mark_install
    fixer.resolve(True)
SystemError: E:Unable to correct problems, you have held broken packages.
=== Command terminated with exit status 1 (Thu Oct 22 03:03:11 2020) ===


Get this bounty!!!

#StackBounty: #amazon-web-services #amazon-ec2 #load-balancing #nat #lets-encrypt How to work Stun/Turn Server (COturn) Under Aws Netwo…

Bounty: 50

I have just setup a coturn server, it works perfectly fine when using the ip or teh domain without loadbalancer, it was tested using this online tool :

https://webrtc.github.io/samples/src/content/peerconnection/trickle-ice/

The problem is when i use a network loadbalancer, rerouting tcp_udp works on port 80, but when trying to use tls for port 443, it doesn’t work.

I configured the Network load balancer to route tls traffic for port 443 to the target group under port 443 also.
I’m using letsencrypt certificate for domain.com and *.domain.com from letsencrypt in my network load balancer.
Same certificates are added in the config file the turnserver.conf.

And this is my config :

external-ip=1.2.3.4
listening-port=80
min-port=10000
max-port=20000
log-file=/var/log/turnserver.log
verbose
tls-listening-port=443
lt-cred-mech
server-name=domain.com realm=domain.com
user=tester:12345678
total-quota=100
stale-nonce=600
cert=/opt/coturn/fullchain.pem
pkey=/opt/coturn/privkey.pem
cipher-list="ECDH+AESGCM:ECDH+CHACHA20:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS"

log-file=/var/log/coturn.log


# Specify the process user and group
proc-user=turnserver
proc-group=turnserver

And this is what i get from the log :

3170: IPv4. tcp or tls connected to: 9.8.7.6:34274
3170: session 001000000000003730: TCP socket closed remotely 9.8.7.6:34274
3170: session 001000000000003730: closed (2nd stage), user <> realm <domain.com> origin <>, local 0.0.0.0:443, remote 9.8.7.6:34274, reason: TCP connection closed by client (callback)

And btw, I always get 701 error from the online tool.

Thank you,


Get this bounty!!!

#StackBounty: #python #amazon-ec2 #pip #amazon #setuptools Why does `pip` throw `ModuleNotFoundError` even though `setuptools` appears …

Bounty: 50

On an EC2 instance, I am attempting to install a Python package using pip, but receive an exception that states ModuleNotFoundError: No module named 'setuptools'.

For context, the command I run is

pip install --user mlflow

I attempt to resolve the issue my installing setuptools with the following command: sudo yum install python36-setuptools. Yet, the message returned indicates that setuptools is already installed:

Package python36-setuptools-36.2.7-1.33.amzn1.noarch already installed and latest version
Nothing to do

Please advise.

UPDATE: After switching the EC2 instance from Amazon Linux AMI 2018.03.0 to Amazon Linux 2 AMI I was able to successfully install mlflow using pip.

The question still remains – why was I able to successfully install using Amazon Linux 2 AMI but not Amazon Linux AMI 2018.03.0?


Get this bounty!!!

#StackBounty: #amazon-ec2 #environment-variables #amazon-cloudwatch #aws-regions Adding custom variable in cloudwatch agent config

Bounty: 50

I have the following config that I generate as part of user data for my EC2 instance

{
        "agent": {
                        "run_as_user": "root"
        },
        "logs": {
                "logs_collected": {
                        "files": {
                        "collect_list": [
                                {
                                "file_path": "/var/log/hapee-2.0/lb-access*",
                                "log_group_name" : <insert-region-here>
                                "log_stream_name": "haproxy-{instance_id}"
                                }
                        ]
                }
                }
        },
       "metrics": {
                "metrics_collected": {
                    "net": {
                    "measurement": [
                        "net_packets_recv",
                    "net_packets_sent"
                    ],
                    "metrics_collection_interval": 60
                },
                "mem": {
                        "measurement": [
                                "used_percent"
                        ]
                }
        }

        }
} 

As the log_group_name I want to be able to append the region that the instance is running in. I understand that the region is part of the instance meta data but can I use it in the cloudwatch config?


Get this bounty!!!

#StackBounty: #ubuntu #amazon-ec2 #datascience-anaconda Ubuntu Cronjob can't execute conda environment

Bounty: 50

Whenever I login to my ec2 Ubuntu instance my terminal is like this

(base) ubuntu@ip-xx-xx-xx

When I run my Python script which is converted from a Jupyter notebook with the default conda environment the script runs perfect. The problem starts when I try to make a cronjob.

When the instance starts(I will try to explain as best I can), the ipython that runs the script can’t find the packages. Because it’s the default conda environment. I thought all I have to do is somehow activate the source bash, but everything failed. What I’ve tried so far

  • I’ve tried to exec bash
  • I’ve tried to add to SHELL the bash
  • I’ve tried to source bash
  • I’ve tried to run ipython directly from conda environment

At this point I don’t know what I am missing.
All I want, is when the instance start to “activate” the conda environment and with ipython run the script.


Get this bounty!!!

#StackBounty: #ubuntu #amazon-ec2 #datascience-anaconda Ubuntu Cronjob can't execute conda environment

Bounty: 50

Whenever I login to my ec2 Ubuntu instance my terminal is like this

(base) ubuntu@ip-xx-xx-xx

When I run my Python script which is converted from a Jupyter notebook with the default conda environment the script runs perfect. The problem starts when I try to make a cronjob.

When the instance starts(I will try to explain as best I can), the ipython that runs the script can’t find the packages. Because it’s the default conda environment. I thought all I have to do is somehow activate the source bash, but everything failed. What I’ve tried so far

  • I’ve tried to exec bash
  • I’ve tried to add to SHELL the bash
  • I’ve tried to source bash
  • I’ve tried to run ipython directly from conda environment

At this point I don’t know what I am missing.
All I want, is when the instance start to “activate” the conda environment and with ipython run the script.


Get this bounty!!!

#StackBounty: #ubuntu #amazon-ec2 #datascience-anaconda Ubuntu Cronjob can't execute conda environment

Bounty: 50

Whenever I login to my ec2 Ubuntu instance my terminal is like this

(base) ubuntu@ip-xx-xx-xx

When I run my Python script which is converted from a Jupyter notebook with the default conda environment the script runs perfect. The problem starts when I try to make a cronjob.

When the instance starts(I will try to explain as best I can), the ipython that runs the script can’t find the packages. Because it’s the default conda environment. I thought all I have to do is somehow activate the source bash, but everything failed. What I’ve tried so far

  • I’ve tried to exec bash
  • I’ve tried to add to SHELL the bash
  • I’ve tried to source bash
  • I’ve tried to run ipython directly from conda environment

At this point I don’t know what I am missing.
All I want, is when the instance start to “activate” the conda environment and with ipython run the script.


Get this bounty!!!

#StackBounty: #ubuntu #amazon-ec2 #datascience-anaconda Ubuntu Cronjob can't execute conda environment

Bounty: 50

Whenever I login to my ec2 Ubuntu instance my terminal is like this

(base) ubuntu@ip-xx-xx-xx

When I run my Python script which is converted from a Jupyter notebook with the default conda environment the script runs perfect. The problem starts when I try to make a cronjob.

When the instance starts(I will try to explain as best I can), the ipython that runs the script can’t find the packages. Because it’s the default conda environment. I thought all I have to do is somehow activate the source bash, but everything failed. What I’ve tried so far

  • I’ve tried to exec bash
  • I’ve tried to add to SHELL the bash
  • I’ve tried to source bash
  • I’ve tried to run ipython directly from conda environment

At this point I don’t know what I am missing.
All I want, is when the instance start to “activate” the conda environment and with ipython run the script.


Get this bounty!!!

#StackBounty: #ubuntu #amazon-ec2 #datascience-anaconda Ubuntu Cronjob can't execute conda environment

Bounty: 50

Whenever I login to my ec2 Ubuntu instance my terminal is like this

(base) ubuntu@ip-xx-xx-xx

When I run my Python script which is converted from a Jupyter notebook with the default conda environment the script runs perfect. The problem starts when I try to make a cronjob.

When the instance starts(I will try to explain as best I can), the ipython that runs the script can’t find the packages. Because it’s the default conda environment. I thought all I have to do is somehow activate the source bash, but everything failed. What I’ve tried so far

  • I’ve tried to exec bash
  • I’ve tried to add to SHELL the bash
  • I’ve tried to source bash
  • I’ve tried to run ipython directly from conda environment

At this point I don’t know what I am missing.
All I want, is when the instance start to “activate” the conda environment and with ipython run the script.


Get this bounty!!!

#StackBounty: #ubuntu #amazon-ec2 #datascience-anaconda Ubuntu Cronjob can't execute conda environment

Bounty: 50

Whenever I login to my ec2 Ubuntu instance my terminal is like this

(base) ubuntu@ip-xx-xx-xx

When I run my Python script which is converted from a Jupyter notebook with the default conda environment the script runs perfect. The problem starts when I try to make a cronjob.

When the instance starts(I will try to explain as best I can), the ipython that runs the script can’t find the packages. Because it’s the default conda environment. I thought all I have to do is somehow activate the source bash, but everything failed. What I’ve tried so far

  • I’ve tried to exec bash
  • I’ve tried to add to SHELL the bash
  • I’ve tried to source bash
  • I’ve tried to run ipython directly from conda environment

At this point I don’t know what I am missing.
All I want, is when the instance start to “activate” the conda environment and with ipython run the script.


Get this bounty!!!