#StackBounty: #python-3.x #module #package How to install python module into dist-packages directory directly?

Bounty: 100

I build a package in /tmp.

tree myanalyse
myanalyse
├── myanalyse
│   ├── ana.py
│   ├── __init__.py
├── README.md
└── setup.py

Install it locally.

cd  /tmp/myanalyse
sudo python3 setup.py build
sudo python3 setup.py install

Now check it.

import  myanalyse
print(myanalyse.__path__)
['/usr/local/lib/python3.5/dist-packages/myanalyse-0.1-py3.5.egg/myanalyse']

Normally,when you install a module such as pandas in your python ,there must be a directory pandas in /usr/local/lib/python3.5/dist-packages.

import pandas
print(pandas.__path__)
['/usr/local/lib/python3.5/dist-packages/pandas']

It is /usr/local/lib/python3.5/dist-packages/myanalyse-0.1-py3.5.egg/myanalyse ,not /usr/local/lib/python3.5/dist-packages/myanalyse in my python,how can i install this package myanalyse in the format /usr/local/lib/python3.5/dist-packages/myanalyse ,instead of /usr/local/lib/python3.5/dist-packages/myanalyse-0.1-py3.5.egg/myanalyse ?


Get this bounty!!!

#StackBounty: #javascript #django #python-3.x #opencv #pythonanywhere How to access webcam in OpenCV on PythonAnywhere through Javascri…

Bounty: 50

I have developed a WebApplication in Django that has a view method which contains the OpevCV code that when triggered opens the User Webcam to detect its face. This app works fine in my localserver but when I have hosted it on PythonAnywhere it says camera not found as my PA hosting doesnt serve a camera.
So someone suggested me to open the webcam through javascript as it deals with the client machine and then pass its feed to server machine which is my hosting.
But as i am a rookie in Python i am not able to figure how to perform the above task.
I found this piece of js code but i dont know how and where to add this in my Django App.

Code for getting the feed with Javascript

var video = document.querySelector("#videoElement");

if (navigator.mediaDevices.getUserMedia) {
    navigator.mediaDevices.getUserMedia({video: true}).then(function(stream) {
      video.srcObject = stream;
  }).catch(function(err0r) {
      console.log("Something went wrong!");
  });
}

My Python code for opening the camera and detecting faces is as follows (it works in localserver)

import cv2

cascade = cv2.CascadeClassifier('./haarcascade_frontalface_default.xml')

cam = cv2.VideoCapture(0)


while True:
    ret, frame = cam.read()
    frame = cv2.flip(frame, 1)

    if ret:
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

        faces = cascade.detectMultiScale(gray, scaleFactor=1.3, minNeighbors=3)

        for (x, y, w, h) in faces:
            cropped = cv2.resize(frame[y:y+h, x:x+w], (198,198))
            cv2.rectangle(frame, (x, y), (x+w, y+h), (255, 0, 0), 2)

        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
            cv2.destroyAllWindows()

        cv2.imshow('Stream', frame)

Any help is appreciated. Thank you in advance


Get this bounty!!!

#StackBounty: #python #django #python-3.x #mocking #python-unittest Why does this assert_has_calls with the usage of ANY fail in python…

Bounty: 50

It looks like the version of the mock is 3.0.5 in both Python 2.7.15 and 3.7.5 I am testing in.

with patch('a.models.BManager.methodC',autospec=True) as method_mock:
    do something

    expected_calls = [
        call(ANY, param_a=some int value, param_c=some datetime value, etc..),
        call(ANY, param_a=some int value, param_c=some datetime value, etc..)
    ]

    method_mock.assert_has_calls(expected_calls, any_order=True)

This succeeds in Python 2.

However, in Python 3, I get (a few new lines inserted for better formatting)

AssertionError: 'methodC' does not contain all of 

(('', <BoundArguments (self=<ANY>, param_a=1, param_b=2, param_c=datetime.datetime(2020, 1, 29, 11, 47, 11))>), ('', <BoundArguments (self=<ANY>, param_a=1, param_b=3, param_c=datetime.datetime(2020, 1, 30, 3, 47, 11))>))

in its call list, found

[('', <BoundArguments (self=<a.models.BManager object at 0x7fc95ff66fd0>, param_a=1, param_b=2, param_c=datetime.datetime(2020, 1, 29, 11, 47, 11))>), ('', <BoundArguments (self=<a.models.BManager object at 0x7fc95ff66fd0>, param_a=1, param_b=3, param_c=datetime.datetime(2020, 1, 30, 3, 47, 11))>)]

instead

The code that’s being run is:

ModelClassOfBManager.objects.methodC(param_a=1, etc)

Can anyone help me why this, which succeeds in Python 2, fail in Python 3? The expected calls and actual calls are exactly the same except the value for ‘self’ which I used ANY.

Additional Info: The test passes also in python 3 if I remove “autospec=True” from mocking and remove “ANY” from the call(). So I have a hunch that it is related to how ANY and ‘self’ work in Python 3.


Get this bounty!!!

#StackBounty: #python-3.x #pandas #dataframe #datetime #resampling How to resample data inside multiindex dataframe

Bounty: 100

I have the following dataframe:enter image description here

I need to resample the data to calculate the weekly pct_change(). How can i get the weekly change ?

Something like data['pct_week'] = data['Adj Close'].resample('W').ffill().pct_change() but the data need to groupby data.groupby(['month', 'week'])

This way every month would yield 4 values for weekly change.Which i can graph then

What i did was df['pct_week'] = data['Adj Close'].groupby(['week', 'day']).pct_change() but i got this error TypeError: 'type' object does not support item assignment


Get this bounty!!!

#StackBounty: #python-3.x #virtualbox How to lock virtualbox to get a screenshot through SOAP API

Bounty: 50

I’m trying to use the SOAP interface of Virtualbox 6.1 from Python to get a screenshot of a machine. I can start the machine but get locking errors whenever I try to retrieve the screen layout.

This is the code:

import zeep

# helper to show the session lock status
def show_lock_state(session_id):
    session_state = service.ISession_getState(session_id)
    print('current session state:', session_state)

# connect
client = zeep.Client('http://127.0.0.1:18083?wsdl')
service = client.create_service("{http://www.virtualbox.org/}vboxBinding", 'http://127.0.0.1:18083?wsdl')
manager_id = service.IWebsessionManager_logon('fakeuser', 'fakepassword')
session_id = service.IWebsessionManager_getSessionObject(manager_id)

# get the machine id and start it
machine_id = service.IVirtualBox_findMachine(manager_id, 'Debian')
progress_id = service.IMachine_launchVMProcess(machine_id, session_id, 'gui')
service.IProgress_waitForCompletion(progress_id, -1)

print('Machine has been started!')
show_lock_state(session_id)

# unlock and then lock to be sure, doesn't have any effect apparently

service.ISession_unlockMachine(session_id)
service.IMachine_lockMachine(machine_id, session_id, 'Shared')


show_lock_state(session_id)

console_id = service.ISession_getConsole(session_id)
display_id = service.IConsole_getDisplay(console_id)
print(service.IDisplay_getGuestScreenLayout(display_id))

The machine is started properly but the last line gives the error VirtualBox error: rc=0x80004001 which from what I read around means locked session.

I tried to release and acquire the lock again, but even though it succeeds the error remains. I went through the documentation but cannot find other types of locks that I’m supposed to use.


Get this bounty!!!

#StackBounty: #python-3.x #pyinstaller Pyinstaller NotImplementedError can't perform this operation

Bounty: 50

I am trying to turn my pthon code into an excutable in window using pyinstaller.
But unforunately it keeps giving me the following error:

NotImplementedError can’t perform this operation

So I did a research to find out how to solve this issue. In here I could find out that one of my package that has dependancy of jinja2 was causing the issue (“folium in this case”). And one of the user named bjones1 even said how to do simple workaround.

However, a simple workaround is to exclude the package containing Jinja2 templates from PyInstaller, then use datas = collect_data_files(‘template_package’, None, True) in a hook file (or manually copy the package). I’ve attached my working example (but used the manual copy process): jinja2.zip.

Now, the problem is I’m fairly new to the Pyinstaller so I didn’t really understand what he was talking about. So I did the more research and figured out how to include hook file + exclude packages. But I’m still not sure how to resolve this issue. So my question is this, how should I apply this into my situation so that when I try to import folium it doesn’t give me any error.

FYI:
I’m using python 3.6 window7 and
here is the list of packages I imported in case anyone wonder.

import folium
from PyQt5.QtWebEngineWidgets import *
from PyQt5.QtWidgets import *
from PyQt5.QtCore import *
from PyQt5.QtGui import *
import sys
import os
import engine                           # this is just self-made module that helps to run my code
import mplleaflet
import matplotlib.pyplot as plt
import math
import io 
import requests
from bs4 import BeautifulSoup
import time
import datetime
import math


Get this bounty!!!

#StackBounty: #python-3.x #tensorflow #tensorflow-datasets Scaling tf.io.gfile.GFile over 100MB/s throughput

Bounty: 50

To keep a GPU fully utilized during training I need to be able to feed about 250 MB/s of raw data to the GPU (the data is uncompressible). I am accessing the data over a fast network which can feed well over 2GB/sec without a problem. Python’s GIL makes it rather hard to get those speeds into the same process that runs Tensorflow without negatively impacting the training loop. Python 3.8’s shared memory may alleviate this, but that’s not supported by Tensorflow just yet.

So I’m using tf.io.gfile.GFile to read data over the network (data is stored on a high bandwidth S3 compliant interface). The value of GFile is that it doesn’t engage the GIL, and thus plays nicely with the training loop. In order to achieve high throughput there needs to be significant parallelization of the network IO.

I only seem to be able to get about 75-100 MB/sec out this approach though.

I’ve timed two approaches:

  • Create a tf.data.Dataset and use tf.data.Dataset.map(mymapfunc, num_parallel_calls=50) (I’ve tried many values of num_parallel_calls including AUTOTUNE).
  • Create a function that reads data using tf.io.gfile.GFile and simply run it using multiple threads in a concurrent.futures.ThreadPoolExecutor, attempting thread counts up to about 100 (there’s no improvement above about 20, and eventually more threads slow it down).

In both cases I’m topping out at 75-100 MB/sec.

Questions:

I’m wondering if there’s a reason for GFile to hit an upper limit that is perhaps more
obvious to someone else.

I’m also making an assumption I should validate: tf.io.gfile.GFile
runs in numpy land, in both cases above I’m running GFile operations
from python land (in the case of tf.data.Dataset I’m using
tf.py_function). If GFile is meant to run as part of the graph
operations more efficiently I’m unaware of this and need to be
corrected.


Get this bounty!!!

#StackBounty: #python-3.x #amazon-s3 #aws-lambda Extract specific column from csv file, and convert to string in Lambda using Python

Bounty: 50

I’m trying to get all email addresses in a comma separated format from a specific column. This is coming from a csv temp file in Lambda. My goal is to save that file in s3 with only one column containing the email addresses.

This is what the source data looks like:

enter image description here

Here is my code:

#open file and extract email address
with open('/tmp/maillist.csv', 'w') as mail_file:
    wm = csv.writer(mail_file)
    mail_list = csv.reader(open('/tmp/filtered.csv', "r"))
    for rows in mail_list:
        ','.join(rows)
        wm.writerow(rows[3])
bucket.upload_file('/tmp/maillist.csv', key)

I was hoping to get a result like this:

enter image description here

But instead, I’m getting a result like this:

enter image description here

I also tried this code:

#open file and extract email address
mail_list = csv.reader(open('/tmp/filtered.csv', "r"))
with open('/tmp/maillist.csv', 'w') as mail_file:
    wm = csv.writer(mail_file)
    wm.writerow(mail_list[3])
bucket.upload_file('/tmp/maillist.csv', key)

But I get this error instead:

Response:
{
  "errorMessage": "'_csv.reader' object is not subscriptable",
  "errorType": "TypeError",
  "stackTrace": [
    "  File "/var/task/lambda_function.py", line 68, in lambda_handlern     wm.writerow(mail_list[3])n"

Any help is appreciated.


Get this bounty!!!

#StackBounty: #python #arrays #python-3.x How to get cordinates of selected numbers which are from array

Bounty: 50

You are given to paint a floor area of size A. There will be 12 paint buckets from 4 primary
colors with each having 3 shades (i.e. total 12 buckets). The bucket size, or more specifically,
the amount of area you can paint from each shade is given in the following arrays.
The different shades of the same primary color are shown in the same row.

  • [12, 23, 14]
  • [10, 30, 15]
  • [16, 22, 35]
  • [14, 24, 20]

Problem & Constraints

You need to select 4 shades to paint the floor area such that;

  1. Entire floor area should be painted; also no overlaps of shades are allowed
  2. “One and only one” shade from each primary color has been selected for the final painting
  3. Amount of wastage is minimized (i.e. assume once we open and use a bucket, any
    remainings will be discarded)

Implement a python program to answer the following problems;

Q1. The color shades (or buckets) satisfying the above constraints (if A = 100)
Q2. The amount of wastage in the above scenario
Q3. What will be the solution for Q1 and Q2 if A = 90?

Note: You may use the below notation to reference each shade in the above map.

R – row index
C – column index
(r,c) – shade in (r+1)th row and (c+1)th column
e.g. (0,0) -> 12, (0,1) -> 23, (1,2) -> 15, etc.

With this, the answer for Q1 can be given in the format [(0,1), (1,2), (2,0), (3,2)]


If user enter 100 system should display color code areas with their coordinates which sum are most close to 100

Example

Enter your Area = 100
100 ~ 101(sum of areas 12+30+35+24, There for 101 is the closet number to 100)

Shades          = [12 30 35 24]
Cordinates of shades = (0,0)(1,1)(2,2)(3,1)

This is my answering code

import numpy as np
import itertools

colors = np.array([[12, 23, 14], [10, 30, 15], [16, 22, 35], [14, 24, 20]])

max_tot = 0

#get total of all integers which are in array
for i in range(len(colors)):
    max_tot = max_tot + max(colors[i])

#Enter Area
area = int(input("Enter your Area = "))

if(area > max_tot):
    print("Area is too long. Don't have enough paints for painting")
elif(area <= 0):
    print("Wrong area")
else:
    #get shades which are given lowest minimum wastage
    for element in itertools.product(*colors):
        if(sum(element) >= area):
            x = sum(element)
            if(x <= max_tot):
                max_tot = x
                el = np.array(element)

    print()
    print("sum of shades   =", max_tot)
    print("minimum wastage =" , max_tot - area)
    print("Shades          =", el)
    print("Cordinates of shades = " ,end ='')

    #get coordinates using manual method
    for i in range(4):
        g,h = np.where(colors == el[i])
        print("(" ,g[0],",", h[0], ")", sep=' ', end='', flush=True)

Output -:

Enter your Area = 100

sum of shades   = 101
minimum wastage = 1
Shades          = [12 30 35 24]
Cordinates of shades = ( 0 , 0 )( 1 , 1 )( 2 , 2 )( 3 , 1 )

You can see I have got coordinates of 12 30 35 24 is manual type. That is not a good programming method.

How can I get that coordinates directly(using better way)?

Note that I have done all question. I want a better answer for Q3


Get this bounty!!!