I have a very simple Python script. All it does is open two data files from a given directory, read the data, make a series of plots and save as PDF. It works, but it is very slow. It takes almost 20 seconds for data files that have 50-100 lines and <30 variables.
import numpy as np import matplotlib.pyplot as plt from matplotlib.backends.backend_pdf import PdfPages with open('file1.out') as f: var1 = f.readline().split() with open('file2.out') as f: var1 = f.readline().split() df1 = np.loadtxt('file1.out', skiprows=1, unpack=True) df2 = np.loadtxt('file2.out', skiprows=1, unpack=True) nc1 = df1.shape nc2 = df2.shape with PdfPages('file_output.pdf') as pdf: ## file1.out fig = plt.figure(figsize=(11,7)) j = 1 for i in range(1,nc1): ax = fig.add_subplot(3,2,j) ax.plot(df1, df1[i], linestyle='-', color='black') ax.set(title=var1[i], xlabel='seconds', ylabel='') if j == 6: pdf.savefig(fig) fig = plt.figure(figsize=(11,7)) j = 1 else: j = j + 1 pdf.savefig(fig) ## file2.out fig = plt.figure(figsize=(11,7)) j = 1 for i in range(1,nc2): ... # and it continues like the block of code above
My questions are:
- Do I need all those imports and are they slowing down the execution?
- Is there a better way to read the data files then opening them twice (once to get the file header and once to get data)?
- Am I using the matplotlib commands correctly/efficiently (I am not very familiar with matplotlib, and this is basically my first attempt to use it)?
Please keep in mind that ideally this script should have as few dependencies as possible, because it is meant to be used on different systems by different users.
The data files have the following format:
t X1 X2 X3 X4 X5 X6 X7 X8 X11 X12 X13 X14 X15 X16 6.000000E+001 4.309764E-007 2.059219E-004 9.055840E-007 2.257223E-003 1.148868E-002 7.605114E-002 4.517820E-004 3.228596E-008 2.678874E-006 7.095441E-006 1.581115E-007 1.010346E-006 1.617892E-006 9.706194E-007 1.200000E+002 4.309764E-007 2.059219E-004 9.055840E-007 2.257223E-003 1.148868E-002 7.605114E-002 4.517820E-004 3.228596E-008 2.678874E-006 7.095441E-006 1.581115E-007 1.010346E-006 1.617892E-006 9.706194E-007 1.800000E+002 3.936234E-007 2.027775E-004 8.644279E-007 2.180931E-003 1.131226E-002 7.476778E-002 4.353550E-004 3.037527E-008 2.534515E-006 6.778434E-006 1.470889E-007 9.488175E-007 1.531702E-006 9.189112E-007