#StackBounty: #python #python-2.7 #debugging #gdb #deadlock Debugging running python app using gdb in python mode does not work

Bounty: 50

I have a multithreaded python application running using virtualenv and I can not properly attach to it using gdb. I want to debug the deadlock and I think this is the only way I can do this with because the app is too huge and I can not reproduce the issue in develop environment.
I have installed python2.7-dbg and then trying to run debugging with gdb /home/PATHTOVENV/venv/bin/python 30183. Here is the output:

GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/PATHTOVENV/venv/bin/python...(no debugging symbols found)...done.
Attaching to program: /home/PATHTOVENV/venv/bin/python, process 30183
Reading symbols from /lib/x86_64-linux-gnu/libpthread.so.0...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libpthread-2.19.so...done.
done.
[New LWP 30326]
[New LWP 30325]
[New LWP 30324]
[New LWP 30323]
[New LWP 30322]
[New LWP 30321]
[New LWP 30320]
[New LWP 30319]
[New LWP 30318]
[New LWP 30317]
[New LWP 30316]
[New LWP 30315]
[New LWP 30314]
[New LWP 30313]
[New LWP 30312]
[New LWP 30311]
[New LWP 30310]
[New LWP 30309]
[New LWP 30308]
[New LWP 30307]
[New LWP 30306]
[New LWP 30305]
[New LWP 30304]
[New LWP 30302]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Loaded symbols for /lib/x86_64-linux-gnu/libpthread.so.0
Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libc-2.19.so...done.
done.
Loaded symbols for /lib/x86_64-linux-gnu/libc.so.6
Reading symbols from /lib/x86_64-linux-gnu/libdl.so.2...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libdl-2.19.so...done.
done.
Loaded symbols for /lib/x86_64-linux-gnu/libdl.so.2
Reading symbols from /lib/x86_64-linux-gnu/libutil.so.1...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libutil-2.19.so...done.
done.
Loaded symbols for /lib/x86_64-linux-gnu/libutil.so.1
Reading symbols from /lib/x86_64-linux-gnu/libz.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/x86_64-linux-gnu/libz.so.1
Reading symbols from /lib/x86_64-linux-gnu/libm.so.6...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libm-2.19.so...done.
done.
Loaded symbols for /lib/x86_64-linux-gnu/libm.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/ld-2.19.so...done.
done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/lxml/etree.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/lxml/etree.so
Reading symbols from /usr/lib/x86_64-linux-gnu/libxslt.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/x86_64-linux-gnu/libxslt.so.1
Reading symbols from /usr/lib/x86_64-linux-gnu/libexslt.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/x86_64-linux-gnu/libexslt.so.0
Reading symbols from /usr/lib/x86_64-linux-gnu/libxml2.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/x86_64-linux-gnu/libxml2.so.2
Reading symbols from /lib/x86_64-linux-gnu/libgcrypt.so.11...(no debugging symbols found)...done.
Loaded symbols for /lib/x86_64-linux-gnu/libgcrypt.so.11
Reading symbols from /lib/x86_64-linux-gnu/liblzma.so.5...(no debugging symbols found)...done.
Loaded symbols for /lib/x86_64-linux-gnu/liblzma.so.5
Reading symbols from /lib/x86_64-linux-gnu/libgpg-error.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib/x86_64-linux-gnu/libgpg-error.so.0
Reading symbols from /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/_hashlib.x86_64-linux-gnu.so...Reading symbols from /usr/lib/debug/usr/lib/python2.7/lib-dynload/_hashlib.x86_64-linux-gnu.so...done.
done.
Loaded symbols for /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/_hashlib.x86_64-linux-gnu.so
Reading symbols from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0...(no debugging symbols found)...done.
Loaded symbols for /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
Reading symbols from /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/datetime.x86_64-linux-gnu.so...Reading symbols from /usr/lib/debug/usr/lib/python2.7/lib-dynload/datetime.x86_64-linux-gnu.so...done.
done.
Loaded symbols for /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/datetime.x86_64-linux-gnu.so
Reading symbols from /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so...Reading symbols from /usr/lib/debug/usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so...done.
done.
Loaded symbols for /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so
Reading symbols from /usr/lib/x86_64-linux-gnu/libffi.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/x86_64-linux-gnu/libffi.so.6
Reading symbols from /lib/x86_64-linux-gnu/libuuid.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/x86_64-linux-gnu/libuuid.so.1
Reading symbols from /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/_ssl.x86_64-linux-gnu.so...Reading symbols from /usr/lib/debug/usr/lib/python2.7/lib-dynload/_ssl.x86_64-linux-gnu.so...done.
done.
Loaded symbols for /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/_ssl.x86_64-linux-gnu.so
Reading symbols from /lib/x86_64-linux-gnu/libssl.so.1.0.0...(no debugging symbols found)...done.
Loaded symbols for /lib/x86_64-linux-gnu/libssl.so.1.0.0
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/bson/_cbson.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/bson/_cbson.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/pymongo/_cmessage.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/pymongo/_cmessage.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/greenlet.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/greenlet.so
Reading symbols from /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/_csv.x86_64-linux-gnu.so...Reading symbols from /usr/lib/debug/usr/lib/python2.7/lib-dynload/_csv.x86_64-linux-gnu.so...done.
done.
Loaded symbols for /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/_csv.x86_64-linux-gnu.so
Reading symbols from /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/_json.x86_64-linux-gnu.so...Reading symbols from /usr/lib/debug/usr/lib/python2.7/lib-dynload/_json.x86_64-linux-gnu.so...done.
done.
Loaded symbols for /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/_json.x86_64-linux-gnu.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/cchardet/_cchardet.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/cchardet/_cchardet.so
Reading symbols from /usr/lib/x86_64-linux-gnu/libstdc++.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/x86_64-linux-gnu/libstdc++.so.6
Reading symbols from /lib/x86_64-linux-gnu/libgcc_s.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/x86_64-linux-gnu/libgcc_s.so.1
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/_yaml.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/_yaml.so
Reading symbols from /usr/lib/x86_64-linux-gnu/libyaml-0.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/x86_64-linux-gnu/libyaml-0.so.2
Reading symbols from /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/_elementtree.x86_64-linux-gnu.so...Reading symbols from /usr/lib/debug/usr/lib/python2.7/lib-dynload/_elementtree.x86_64-linux-gnu.so...done.
done.
Loaded symbols for /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/_elementtree.x86_64-linux-gnu.so
Reading symbols from /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/pyexpat.x86_64-linux-gnu.so...Reading symbols from /usr/lib/debug/usr/lib/python2.7/lib-dynload/pyexpat.x86_64-linux-gnu.so...done.
done.
Loaded symbols for /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/pyexpat.x86_64-linux-gnu.so
Reading symbols from /lib/x86_64-linux-gnu/libexpat.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/x86_64-linux-gnu/libexpat.so.1
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/datrie.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/datrie.so
Reading symbols from /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/parser.x86_64-linux-gnu.so...Reading symbols from /usr/lib/debug/usr/lib/python2.7/lib-dynload/parser.x86_64-linux-gnu.so...done.
done.
Loaded symbols for /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/parser.x86_64-linux-gnu.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/simplejson/_speedups.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/simplejson/_speedups.so
Reading symbols from /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/_multiprocessing.x86_64-linux-gnu.so...Reading symbols from /usr/lib/debug/usr/lib/python2.7/lib-dynload/_multiprocessing.x86_64-linux-gnu.so...done.
done.
Loaded symbols for /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/_multiprocessing.x86_64-linux-gnu.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/thrift/protocol/fastbinary.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/thrift/protocol/fastbinary.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/hiredis/hiredis.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/hiredis/hiredis.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/tornado/speedups.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/tornado/speedups.so
Reading symbols from /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/_curses.x86_64-linux-gnu.so...Reading symbols from /usr/lib/debug/usr/lib/python2.7/lib-dynload/_curses.x86_64-linux-gnu.so...done.
done.
Loaded symbols for /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/_curses.x86_64-linux-gnu.so
Reading symbols from /lib/x86_64-linux-gnu/libncursesw.so.5...(no debugging symbols found)...done.
Loaded symbols for /lib/x86_64-linux-gnu/libncursesw.so.5
Reading symbols from /lib/x86_64-linux-gnu/libtinfo.so.5...(no debugging symbols found)...done.
Loaded symbols for /lib/x86_64-linux-gnu/libtinfo.so.5
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/zope/interface/_zope_interface_coptimizations.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/zope/interface/_zope_interface_coptimizations.so
Reading symbols from /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/termios.x86_64-linux-gnu.so...Reading symbols from /usr/lib/debug/usr/lib/python2.7/lib-dynload/termios.x86_64-linux-gnu.so...done.
done.
Loaded symbols for /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/termios.x86_64-linux-gnu.so
Reading symbols from /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/resource.x86_64-linux-gnu.so...Reading symbols from /usr/lib/debug/usr/lib/python2.7/lib-dynload/resource.x86_64-linux-gnu.so...done.
done.
Loaded symbols for /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/resource.x86_64-linux-gnu.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/_psutil_linux.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/_psutil_linux.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/_psutil_posix.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/_psutil_posix.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/_pydevd_bundle/pydevd_cython.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/_pydevd_bundle/pydevd_cython.so
Reading symbols from /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/_sqlite3.x86_64-linux-gnu.so...Reading symbols from /usr/lib/debug/usr/lib/python2.7/lib-dynload/_sqlite3.x86_64-linux-gnu.so...done.
done.
Loaded symbols for /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/_sqlite3.x86_64-linux-gnu.so
Reading symbols from /usr/lib/x86_64-linux-gnu/libsqlite3.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/x86_64-linux-gnu/libsqlite3.so.0
Reading symbols from /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/_lsprof.x86_64-linux-gnu.so...Reading symbols from /usr/lib/debug/usr/lib/python2.7/lib-dynload/_lsprof.x86_64-linux-gnu.so...done.
done.
Loaded symbols for /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/_lsprof.x86_64-linux-gnu.so
Reading symbols from /lib/x86_64-linux-gnu/libnss_files.so.2...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libnss_files-2.19.so...done.
done.
Loaded symbols for /lib/x86_64-linux-gnu/libnss_files.so.2
Reading symbols from /lib/x86_64-linux-gnu/libnss_dns.so.2...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libnss_dns-2.19.so...done.
done.
Loaded symbols for /lib/x86_64-linux-gnu/libnss_dns.so.2
Reading symbols from /lib/x86_64-linux-gnu/libresolv.so.2...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libresolv-2.19.so...done.
done.
Loaded symbols for /lib/x86_64-linux-gnu/libresolv.so.2
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/dawg.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/dawg.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/PIL/_imaging.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/PIL/_imaging.so
Reading symbols from /usr/lib/x86_64-linux-gnu/libjpeg.so.8...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/x86_64-linux-gnu/libjpeg.so.8
Reading symbols from /usr/lib/x86_64-linux-gnu/libtiff.so.5...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/x86_64-linux-gnu/libtiff.so.5
Reading symbols from /usr/lib/x86_64-linux-gnu/libjbig.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/x86_64-linux-gnu/libjbig.so.0
Reading symbols from /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/mmap.x86_64-linux-gnu.so...Reading symbols from /usr/lib/debug/usr/lib/python2.7/lib-dynload/mmap.x86_64-linux-gnu.so...done.
done.
Loaded symbols for /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/mmap.x86_64-linux-gnu.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/maxminddb/extension.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/maxminddb/extension.so
Reading symbols from /usr/lib/x86_64-linux-gnu/libmaxminddb.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/x86_64-linux-gnu/libmaxminddb.so.0
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/numpy/core/multiarray.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/numpy/core/multiarray.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/numpy/core/umath.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/numpy/core/umath.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/numpy/core/_dotblas.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/numpy/core/_dotblas.so
Reading symbols from /usr/lib/libcblas.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libcblas.so.3
Reading symbols from /usr/lib/libatlas.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libatlas.so.3
Reading symbols from /usr/lib/x86_64-linux-gnu/libgfortran.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/x86_64-linux-gnu/libgfortran.so.3
Reading symbols from /usr/lib/x86_64-linux-gnu/libquadmath.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/x86_64-linux-gnu/libquadmath.so.0
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/numpy/core/scalarmath.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/numpy/core/scalarmath.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/numpy/lib/_compiled_base.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/numpy/lib/_compiled_base.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/numpy/linalg/lapack_lite.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/numpy/linalg/lapack_lite.so
Reading symbols from /usr/lib/liblapack.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/liblapack.so.3
Reading symbols from /usr/lib/libf77blas.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libf77blas.so.3
Reading symbols from /usr/lib/libblas.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libblas.so.3
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/numpy/linalg/_umath_linalg.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/numpy/linalg/_umath_linalg.so
Reading symbols from /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/future_builtins.x86_64-linux-gnu.so...Reading symbols from /usr/lib/debug/usr/lib/python2.7/lib-dynload/future_builtins.x86_64-linux-gnu.so...done.
done.
Loaded symbols for /home/PATHTOVENV/venv/lib/python2.7/lib-dynload/future_builtins.x86_64-linux-gnu.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/numpy/fft/fftpack_lite.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/numpy/fft/fftpack_lite.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/numpy/random/mtrand.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/numpy/random/mtrand.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/matplotlib/_path.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/matplotlib/_path.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/matplotlib/ft2font.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/matplotlib/ft2font.so
Reading symbols from /usr/lib/x86_64-linux-gnu/libfreetype.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/x86_64-linux-gnu/libfreetype.so.6
Reading symbols from /lib/x86_64-linux-gnu/libpng12.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib/x86_64-linux-gnu/libpng12.so.0
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/matplotlib/_png.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/matplotlib/_png.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/matplotlib/_cntr.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/matplotlib/_cntr.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/matplotlib/_image.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/matplotlib/_image.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/matplotlib/_tri.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/matplotlib/_tri.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/matplotlib/_qhull.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/matplotlib/_qhull.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/matplotlib/backends/_backend_agg.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/matplotlib/backends/_backend_agg.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/PIL/_imagingft.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/PIL/_imagingft.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/Crypto/Util/_counter.so...(no debugging symbols found)...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/Crypto/Util/_counter.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/Crypto/Cipher/_AES.so...(no debugging symbols found)...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/Crypto/Cipher/_AES.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/Crypto/Util/strxor.so...(no debugging symbols found)...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/Crypto/Util/strxor.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/_geohash.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/_geohash.so
Reading symbols from /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/bx/intervals/intersection.so...done.
Loaded symbols for /home/PATHTOVENV/venv/local/lib/python2.7/site-packages/bx/intervals/intersection.so
Reading symbols from /usr/lib/x86_64-linux-gnu/gconv/CP1251.so...Reading symbols from /usr/lib/debug//usr/lib/x86_64-linux-gnu/gconv/CP1251.so...done.
done.
Loaded symbols for /usr/lib/x86_64-linux-gnu/gconv/CP1251.so
sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
85  ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S: No such file or directory.
(gdb) info thread
  Id   Target Id         Frame 
  25   Thread 0x7f08551db700 (LWP 30302) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  24   Thread 0x7f08541d9700 (LWP 30304) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  23   Thread 0x7f08539d8700 (LWP 30305) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  22   Thread 0x7f08531d7700 (LWP 30306) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  21   Thread 0x7f08529d6700 (LWP 30307) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  20   Thread 0x7f08521d5700 (LWP 30308) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  19   Thread 0x7f08519d4700 (LWP 30309) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  18   Thread 0x7f08511d3700 (LWP 30310) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  17   Thread 0x7f08509d2700 (LWP 30311) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  16   Thread 0x7f08501d1700 (LWP 30312) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  15   Thread 0x7f084f9d0700 (LWP 30313) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  14   Thread 0x7f084f1cf700 (LWP 30314) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  13   Thread 0x7f084e9ce700 (LWP 30315) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  12   Thread 0x7f084e1cd700 (LWP 30316) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  11   Thread 0x7f084d9cc700 (LWP 30317) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  10   Thread 0x7f084d1cb700 (LWP 30318) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  9    Thread 0x7f084c9ca700 (LWP 30319) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  8    Thread 0x7f084c1c9700 (LWP 30320) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  7    Thread 0x7f084b9c8700 (LWP 30321) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  6    Thread 0x7f084b1c7700 (LWP 30322) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  5    Thread 0x7f084a9c6700 (LWP 30323) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  4    Thread 0x7f084a1c5700 (LWP 30324) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  3    Thread 0x7f08499c4700 (LWP 30325) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
  2    Thread 0x7f08491c3700 (LWP 30326) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
* 1    Thread 0x7f0903832740 (LWP 30183) "python" sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85

The gdb sees all the threads and their sem_wait state but I can not use python helpers for GDB:

(gdb) py-list
Undefined command: "py-list".  Try "help".

OS Ubuntu 14.04, python 2.7.
I have tried different hacks I had googled (like linking python-dbg.py into virtualenv bin directory) but with no profit, may be I have do it wrong way…


Get this bounty!!!

#StackBounty: #python #regression #convnet #keras #audio-recognition Optimizing CNN network

Bounty: 50

I am currently trying to recreate the result of this paper, in which they do feature extraction from a “spectogram” of log-melfilter energies..

enter image description here

Since the paper doesn’t state what kind of feature I am seeking, i am currently trying to extract features, and match them to MFCC features. The paper states a technique called LWS (Limited weight sharing) in which the spectogram frequency axis will be divided into section, and each section don’t share their weight with others.

So i’ve divided the my input image into 13 section to receive 1 output features from a (6,3,3) input image. 6 for the number of rows, 3 as each column represent [static delta delta_delta] data of the given log melfilter energi, and the last 3 is the color channels.

If i’d used 13 filterbanks, and made the plot, will the result of this be that each (1,3,3) matrix would result in one feature, but that seemed a bit too good to be true, so i decided to use 78 filterbanks and divide it into 13 section which should result in one feature can be extracted from a matrix of size (6,3,3)

I am training the network with this model structure:

def create_model(init_mode='normal',activation_mode='softsign',optimizer_mode="Adamax", activation_mode_conv = 'softsign'):
    model = Sequential()


    model.add(ZeroPadding2D((6,4),input_shape=(6,3,3)))
    model.add(Convolution2D(32,3,3 , activation=activation_mode_conv))
    print model.output_shape
    model.add(Convolution2D(32, 3,3, activation=activation_mode_conv))
    print model.output_shape
    model.add(MaxPooling2D(pool_size=(2,2),strides=(2,1)))
    print model.output_shape
    model.add(Convolution2D(64, 3,3 , activation=activation_mode_conv))
    print model.output_shape
    model.add(Convolution2D(64, 3,3 , activation=activation_mode_conv))
    print model.output_shape
    model.add(MaxPooling2D(pool_size=(2,2),strides=(2,1)))
    model.add(Flatten())
    print model.output_shape
    model.add(Dense(output_dim=32, input_dim=64, init=init_mode,activation=activation_mode))
    model.add(Dense(output_dim=13, input_dim=50, init=init_mode,activation=activation_mode))
    model.add(Dense(output_dim=1, input_dim=13, init=init_mode,activation=activation_mode))
    model.add(Dense(output_dim=1,  init=init_mode, activation=activation_mode))
    #print model.summary()
    model.compile(loss='mean_squared_error',optimizer=optimizer_mode)

    return model

This model keeps for some reason providing me very bad results..
I seem to keep getting an loss of 216, which is nearly 3 times the data range…

I did a grid seach to find out which parameter (activation function, init_mode, epochs and batch_size would be best, which are those chosen in the function above (eventhough there wasn’t much change in the outcome..)

What can i do to get better results?
Is the CNN network poorly designed?


Get this bounty!!!

#StackBounty: #python #python-2.7 #socket #tcp #network-file-transfer Protocol implementation, TCP, sending images through sockets (soc…

Bounty: 50

I would like to ask about your opinion about my code. The idea is simple: I designed my own protocol, where client asks server about the image, and the server sends the image, following the below steps (this is the actual protocol I wanted to implement):

CLIENT                                      SERVER
                      GETrn   
       ----------------------------------->
                      OKrn
       <----------------------------------- 
                   GET_SIZErn
       ----------------------------------->
                    SIZE 1024rn
       <-----------------------------------
                   GET_IMGrn
       ----------------------------------->
    IMG_DATA an when image sending is over, EOIMGrn 
       <-----------------------------------
                   DONErn
       ----------------------------------->
            server disconnects the client   

I improved the code as suggested by @TobErnack, but still, would like to know your opinion of my improvements. Here’s the link to the original question and code: Implementing my own network protocol using TCP, sending images through sockets (sockets, python)

Everything works (it seems so) but I would like to ask, what else I could possibly improve here.

  1. Does the implementation reflects in 100% what I wanted to achieve in
    my protocol flow?
  2. Are the steps implemented as I wanted them to be in my protocol?

client.py

#!/usr/bin/env python

import socket
import sys

HOST = '127.0.0.1'
PORT = 6666

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = (HOST, PORT)
sock.connect(server_address)
fname = 'fromserver.png'


def recvall(sock, msgLen):
    msg = ""
    bytesRcvd = 0

    while bytesRcvd < msgLen:

        chunk = sock.recv(msgLen - bytesRcvd)

        if chunk == "": break

        bytesRcvd += len(chunk)
        msg += chunk

        if "rn" in msg: break
    return msg


try:

    sock.sendall("GETrn")
    data = recvall(sock, 4096)

    if data:
        txt = data.strip()
        print '--%s--' % txt

        if txt == 'OK':

            sock.sendall("GET_SIZErn")
            data = recvall(sock, 4096)

            if data:
                txt = data.strip()
                print '--%s--' % txt

                if txt.startswith('SIZE'):

                    tmp = txt.split()
                    size = int(tmp[1])

                    print '--%s--' % size

                    sock.sendall("GET_IMGrn")

                    myfile = open(fname, 'wb')

                    amount_received = 0
                    while amount_received < size:
                        data = recvall(sock, 4096)
                        if not data:
                            break
                        amount_received += len(data)
                        print amount_received

                        txt = data.strip('rn')

                        if 'EOIMG' in str(txt):
                            print 'Image received successfully'
                            myfile.write(data)
                            myfile.close()
                            sock.sendall("DONErn")
                        else:
                            myfile.write(data)
finally:
    sock.close()

server.py

# !/usr/bin/env python

import random
import socket, select
from time import gmtime, strftime

image = 'tux.png'

HOST = '127.0.0.1'
PORT = 6666

connected_clients_sockets = []

server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server_socket.bind((HOST, PORT))
server_socket.listen(10)

connected_clients_sockets.append(server_socket)


def recvall(sock, msgLen):
    msg = ""
    bytesRcvd = 0

    while bytesRcvd < msgLen:

        chunk = sock.recv(msgLen - bytesRcvd)

        if chunk == "": break

        bytesRcvd += len(chunk)
        msg += chunk

        if "rn" in msg: break
    return msg


while True:

    read_sockets, write_sockets, error_sockets = select.select(connected_clients_sockets, [], [])

    for sock in read_sockets:

        if sock == server_socket:

            sockfd, client_address = server_socket.accept()
            connected_clients_sockets.append(sockfd)

        else:
            try:
                data = recvall(sock, 4096)

                if data:

                    txt = data.strip()
                    print '--%s--' % txt

                    if txt == 'GET':
                        sock.sendall('OKrn')

                    elif txt == 'GET_SIZE':

                        with open('tux.png', 'rb') as f1:
                            file_size = len(f1.read())
                            f1.seek(0)

                        print '--%s--' % file_size

                        file_size = '%s' % file_size
                        sock.sendall('SIZE %srn' % file_size)

                    elif txt == 'GET_IMG':
                        with open(image, 'rb') as fp:
                            image_data = fp.read()

                        msg = '%sEOIMGrn' % image_data
                        sock.sendall(msg)

                    elif txt == 'DONE':
                        sock.close()
                        connected_clients_sockets.remove(sock)

            except:
                sock.close()
                connected_clients_sockets.remove(sock)
                continue
server_socket.close()


Get this bounty!!!

#StackBounty: #python #algorithm #python-2.7 #sudoku Sudoku solver recursive solution

Bounty: 50

Here is my code in Python 2.7 for a Sudoku resolver. Any advice on performance improvement, code bugs or general code style advice is appreciated.

My major idea is:

  1. Using method generate some random data, then using check_invalid to see if it is a valid Sudoku
  2. If from 1, it is a valid Sudoku, then I will call resolve_sudoku to fill valid values

I find my code sometimes run a long time without completion. Are there any code bugs you can find?

import random
from collections import defaultdict
found = False
def check_invalid(matrix):
    # check for each row
    for row in range(len(matrix)):
        cur_row = set()
        for col in range(len(matrix[0])):
            if matrix[row][col] == 0:
                continue
            elif 1 <= matrix[row][col] <= 9:
                if matrix[row][col] in cur_row:
                    return False
                else:
                    cur_row.add(matrix[row][col])
            else:
                return False # invalid number
    # check each col
    for col in range(len(matrix[0])):
        cur_col = set()
        for row in range(len(matrix)):
            if matrix[row][col] == 0:
                continue
            elif 1 <= matrix[row][col] <= 9:
                if matrix[row][col] in cur_col:
                    return False
                else:
                    cur_col.add(matrix[row][col])
            else:
                return False # invalid number
    # check each 3*3 square
    for start_row in [0,3,6]:
        for start_col in [0,3,6]:
            cur_square = set()
            for row in range(start_row, start_row+3):
                for col in range(start_col, start_col + 3):
                    if matrix[row][col] == 0:
                        continue
                    elif 1 <= matrix[row][col] <= 9:
                        if matrix[row][col] not in cur_square:
                            cur_square.add(matrix[row][col])
                        else:
                            return False
                    else:
                        return False # invalid value
    return True

def resolve_sudoku(matrix, row_map, col_map, square_map, cur_row, cur_col):
    global found
    if found:
        return
    if cur_row == len(matrix):
        found = True
        for r in matrix:
            print r
    elif cur_col == len(matrix[0]):
        resolve_sudoku(matrix, row_map, col_map, square_map, cur_row+1, 0)
    elif matrix[cur_row][cur_col] != 0:
        resolve_sudoku(matrix, row_map, col_map, square_map, cur_row, cur_col+1)
    else:
        for val in range(1,10):
            square_x = cur_row / 3
            square_y = cur_col / 3
            if val in row_map[cur_row] or val in col_map[cur_col] or val in square_map[(square_x, square_y)]:
                continue
            else:
                row_map[cur_row].add(val)
                col_map[cur_col].add(val)
                square_map[(square_x, square_y)].add(val)
                matrix[cur_row][cur_col] = val
                resolve_sudoku(matrix, row_map, col_map, square_map, cur_row, cur_col+1)
                row_map[cur_row].remove(val)
                col_map[cur_col].remove(val)
                square_map[(square_x, square_y)].remove(val)
                matrix[cur_row][cur_col] = 0
                if found:
                    return
if __name__ == "__main__":
    matrix = []
    for row in range(9):
        cur_row = []
        for col in range(9):
            if random.random() < 0.1:
                cur_row.append(random.randint(1,9))
            else:
                cur_row.append(0)
        matrix.append(cur_row)
    for r in matrix:
        print r
    re = check_invalid(matrix)
    print re
    if re:
        # init for row map and col map
        row_map = defaultdict(set)
        col_map = defaultdict(set)
        square_map = defaultdict(set)
        for row in range(len(matrix)):
            for col in range(len(matrix[0])):
                square_x = row / 3
                square_y = row / 3
                if matrix[row][col] != 0:
                    row_map[row].add(matrix[row][col])
                    col_map[col].add(matrix[row][col])
                    square_map[(row, col)].add(matrix[row][col])
        resolve_sudoku(matrix, row_map, col_map, square_map, 0, 0)


Get this bounty!!!

#StackBounty: #python #design-patterns #activerecord #flask-sqlalchemy #datamapper Flask SQLAlchemy Data Mapper vs Active Record Pattern

Bounty: 50

I have recently started working on Flask and Flask-SQLAlchemy. Coming from Django background I found Flask-SQLAlchmey to be quite complex. I have read that SQLAlchemy implements Data Mapper pattern while Django ORM is based on Active Record Pattern.

Here is a sample code written that implements repository pattern to access the database.

Here is another link of a comment by S.Lott (271k reputation) who says that ORM is the data access layer and it is separate from model.

My questions are these:

  1. Can you provide a practical use case in the above example or an example of your own where Data mapper pattern is useful? Everywhere I have read is that data mapper pattern is useful in complex situation, but not seen examples.
  2. Is using repositories pattern as in above case same as using a data mapper pattern?
  3. Does data mapper advocates write select queries in a different class than the model as done in the example?
  4. Why is Question.query.filter_by(text = text).all() not better to use than
    db.session.query(Question).filter(Question.text == text).all()?

This is not a duplicate of DataMapper vs ActiveRecord pattern because this just tells the definition, I am more interested in the practical examples.


Get this bounty!!!

#StackBounty: Efficient extraction of patch features over an image

Bounty: 50

Input:

  • data – a matrix of size n*m*3*3 (complex values)
  • indices – a list of coordinates (x,y), where x < n and y < m
  • fp – a feature parameter which is a tuple of ((fp11, fp12), (fp21, fp22)), id)
  • reference – a list of 3*3 matrices
  • swrd – a function which computes a similarity value between two complex valued 3*3 matrices

Output:

  • feature_values – a list of features – one feature for each index in (indices)

Functionality:

Given an image (data) were each pixel is a 3*3 matrix. And there is a list of target pixels (indices). For each target pixel, I want to extract features of the patch surrounding it.

A patch feature is either:
a) the swrd of a pixel in the patch with a reference matrix or
b) the swrd of two pixels in the patch

Thus a feature can be described by the relative coordinates fp11, fp12 (x and y offset of pixel of interest 1) and fp21, fp22 (x and y offset of pixel of interest 2).
If fp11 == fp21 and fp12== fp22, then i want to compute a), else i want to compute b).
The reference matrix of interest is defined by the feature parameter called id.

Note that the indices of interest are already filtered so that the sum x+fp__ < n and y+fp__< m for all possible fp__.

Code

Computing the symetric revised wishart distance with regularization in case a matrix A or B is not invertible

def srwd(A, B):
    """This function computes the symetric revised wishart distance as from the paper
    SPECTRAL CLUSTERING OF POLARIMETRIC SAR DATA WITH WISHART-DERIVED DISTANCE MEASURES"""
    try:
        dist = 0.5 * np.trace(np.dot(A, inv(B)) + np.dot(B, inv(A))) - len(A)      
    except:
        A, B = A.reshape(3, 3) + np.eye(3) * 0.000001, B.reshape(3, 3) + np.eye(3) * 0.000001
        dist = 0.5 * np.trace(np.dot(A, inv(B)) + np.dot(B, inv(A))) - len(A)      
    return abs(dist)

Getting the features with the input as given above:

def feature(data, indices, fp, reference):
    # fp is a tuple of 2 coordinates in a patch ((x1,x2),(y1,y2),ref),
    # where ref is an index of a random reference matrix in reference only relevant in case x1=y1 and x2=y2
    res = []
    if fp[0] != fp[1]:
        for i in indices:
            x, y = i
            res.append(srwd(data[x + fp[0][0]][y + fp[0][1]], data[x + fp[1][0]][y + fp[1][1]]))
    else:
        for i in indices:
            x, y = i
            res.append(srwd(data[x + fp[0][0]][y + fp[0][1]], reference[fp[2]]))
    return res

Finally there is another loop such as:

for fp in feature_params:
    feature_values = feature(data, indices, fp, reference)
    #here work on feature_values

The current implementation is rather inefficent and a bottleneck of the whole process. How could I improve it?

Is there a chance to efficiently compute a feature matrix efficiently and operate on it afterwards?

An executable toy example including the whole code is given here (allowing copy-paste)

import numpy as np
from numpy.linalg import inv

#toy example
data = np.random.rand(1000, 1000, 3, 3) #an image of 1000*1000 pixels, each pixel a 3*3 matrix
indices = np.random.randint(3,96, size = (10000,2)) # a list of 10000 target pixels (lets assume they are unique)
reference = [np.random.rand(3,3)] # a single reference matrix in a list (in actual application there are multiple reference matrices)
feature_params = [((0,0),(-1,-1), 0), ((0,0), (0,0), 0), ((0,1), (0,0), 0), ((1,0), (0,0), 0), ((1,1), (0,0), 0)] 



def srwd(A, B):
    """This function computes the symetric revised wishart distance as from the paper
    SPECTRAL CLUSTERING OF POLARIMETRIC SAR DATA WITH WISHART-DERIVED DISTANCE MEASURES"""
    try:
        dist = 0.5 * np.trace(np.dot(A, inv(B)) + np.dot(B, inv(A))) - len(A)      
    except:
        A, B = A.reshape(3, 3) + np.eye(3) * 0.000001, B.reshape(3, 3) + np.eye(3) * 0.000001
        dist = 0.5 * np.trace(np.dot(A, inv(B)) + np.dot(B, inv(A))) - len(A)      
    return abs(dist)


def feature(data, indices, fp, reference):
    # fp is a tuple of 2 coordinates in a patch ((x1,x2),(y1,y2),ref),
    # where ref is an index of a random reference matrix in reference only relevant in case x1=y1 and x2=y2
    res = []
    if fp[0] != fp[1]:
        for i in indices:
            x, y = i
            res.append(srwd(data[x + fp[0][0]][y + fp[0][1]], data[x + fp[1][0]][y + fp[1][1]]))
    else:
        for i in indices:
            x, y = i
            res.append(srwd(data[x + fp[0][0]][y + fp[0][1]], reference[fp[2]]))
    return res

for fp in feature_params:
    feature_values = feature(data, indices, fp, reference)        
    #here work on feature_values

A final note about the dimensions of the actual problem:

  • Image of size 6000*1700,
  • around 500 features in feature_params
  • indices is list of around 8.000.000 target indices


Get this bounty!!!

Launch HTML code in browser from Python

Lets say you have generated some html content for a web page dynamically and have it in memory variable in python.

In order to view and test this content, you would need a Python program that prints out the HTML code, which then would have to be copied and pasted to a HTML file, then from there, you can test it in a browser.

In Python, there is a way to launch such HTML code in a web browser so that you don’t have to go through the copy and paste method every time

Using webbrowser.open:

Source

Sort a list of tuples by Nth item in Python

Suppose you have a list of tuples that looks something like this:

[('abc', 121),('abc', 231),('abc', 148), ('abc',221)]

And you want to sort this list in ascending order by the integer value inside the tuples.

We can achieve this using the key keyword with sorted().

sorted([('abc', 121),('abc', 231),('abc', 148), ('abc',221)], key=lambda x: x[1])

key should be a function that identifies how to retrieve the comparable element from your data structure. For example, the second element of the tuple, so we access [1].

Source: StackOverflow.com

The Fuck

Magnificent app which corrects your previous console command, inspired by a @liamosaur tweet.

Few more examples:

➜ apt-get install vim
E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission denied)
E: Unable to lock the administration directory (/var/lib/dpkg/), are you root?

➜ fuck
sudo apt-get install vim [enter/↑/↓/ctrl+c]
[sudo] password for nvbn:
Reading package lists... Done
...
➜ git push
fatal: The current branch master has no upstream branch.
To push the current branch and set the remote as upstream, use

    git push --set-upstream origin master


➜ fuck
git push --set-upstream origin master [enter/↑/↓/ctrl+c]
Counting objects: 9, done.
...
➜ puthon
No command 'puthon' found, did you mean:
 Command 'python' from package 'python-minimal' (main)
 Command 'python' from package 'python3' (main)
zsh: command not found: puthon

➜ fuck
python [enter/↑/↓/ctrl+c]
Python 3.4.2 (default, Oct  8 2014, 13:08:17)
...
➜ git brnch
git: 'brnch' is not a git command. See 'git --help'.

Did you mean this?
    branch

➜ fuck
git branch [enter/↑/↓/ctrl+c]
* master
➜ lein rpl
'rpl' is not a task. See 'lein help'.

Did you mean this?
         repl

➜ fuck
lein repl [enter/↑/↓/ctrl+c]
nREPL server started on port 54848 on host 127.0.0.1 - nrepl://127.0.0.1:54848
REPL-y 0.3.1
...

Source