IndexError: too many indices for array

Question

I know there is a ton of these threads but all of them are for very simple cases like 3x3 matrices and things of that sort and the solutions do not even begin to apply to my situation. So I'm trying to graph G versus l1 (that's not an eleven, but an L1). The data is in the file that I loaded from an excel file. The excel file is 14x250 so there are 14 arguments, each with 250 data points. I had another user (shout out to Hugh Bothwell!) help me with an error in my code, but now another error has surfaced.

So here is the code in question:

# format for CSV file:
header = ['l1', 'l2', 'l3', 'l4', 'l5', 'EI',
      'S', 'P_right', 'P1_0', 'P3_0',
      'w_left', 'w_right', 'G_left', 'G_right']

def loadfile(filename, skip=None, *args):
    skip = set(skip or [])
    with open(filename, *args) as f:
        cr = csv.reader(f, quoting=csv.QUOTE_NONNUMERIC)
        return np.array(row for i,row in enumerate(cr) if i not in skip)
#plot data
outputs_l1 = [loadfile('C:\\Users\\Chris\\Desktop\\Work\\Python Stuff\\BPCROOM - Shingles analysis\\ERR analysis\\l_1 analysis//BS(1) ERR analysis - l_1 - P_3 = {}.csv'.format(p)) for p in p3_arr]

col = {name:i for i,name in enumerate(header)}

fig = plt.figure()
for data,color in zip(outputs_l1, colors):
    xs  = data[:, col["l1"     ]]
    gl = data[:, col["G_left" ]] * 1000.0    # column 12
    gr = data[:, col["G_right"]] * 1000.0    # column 13
    plt.plot(xs, gl, color + "-", gr, color + "--")
for output, col in zip(outputs_l1, colors):
    plt.plot(output[:,0], output[:,11]*1E3, col+'--')
plt.ticklabel_format(axis='both', style='plain', scilimits=(-1,1))
plt.xlabel('$l1 (m)$')
plt.ylabel('G $(J / m^2) * 10^{-3}$')
plt.xlim(xmin=.2)
plt.ylim(ymax=2, ymin=0)

plt.subplots_adjust(top=0.8, bottom=0.15, right=0.7)

After running the entire program, I recieve the error message:

Traceback (most recent call last):
  File "C:/Users/Chris/Desktop/Work/Python Stuff/New Stuff from Brenday 8 26 2014/CD_ssa_plot(2).py", line 115, in <module>
    xs  = data[:, col["l1"     ]]
IndexError: too many indices for array

and before I ran into that problem, I had another involving the line a few below the one the above error message refers to:

Traceback (most recent call last): File "FILE", line 119, in <module> 
gl = data[:, col["G_left" ]] * 1000.0 # column 12 
IndexError: index 12 is out of bounds for axis 1 with size 12

I understand the first error, but am just having problems fixing it. The second error is confusing for me though. My boss is really breathing down my neck so any help would be GREATLY appreciated!

Answer 1

I think the problem is given in the error message, although it is not very easy to spot:

IndexError: too many indices for array
xs  = data[:, col["l1"     ]]

'Too many indices' means you've given too many index values. You've given 2 values as you're expecting data to be a 2D array. Numpy is complaining because data is not 2D (it's either 1D or None).

This is a bit of a guess - I wonder if one of the filenames you pass to loadfile() points to an empty file, or a badly formatted one? If so, you might get an array returned that is either 1D, or even empty (np.array(None) does not throw an Error, so you would never know...). If you want to guard against this failure, you can insert some error checking into your loadfile function.

I highly recommend in your for loop inserting:

print(data)

This will work in Python 2.x or 3.x and might reveal the source of the issue. You might well find it is only one value of your outputs_l1 list (i.e. one file) that is giving the issue.