I'm trying to train a LSTM network on some data, unfortunately I keep running into following error:
InvalidArgumentError: indices[] = is not in [0, 4704)
Train on 180596 samples, validate on 45149 samples
Epoch 1/1
---------------------------------------------------------------------------
InvalidArgumentError Traceback (most recent call last)
<ipython-input-52-a434c3619685> in <module>()
14 epochs=1,
15 batch_size=128,
---> 16 validation_split=0.2)
c:\program files\python3x64\lib\site-packages\keras\engine\training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, **kwargs)
1040 initial_epoch=initial_epoch,
1041 steps_per_epoch=steps_per_epoch,
-> 1042 validation_steps=validation_steps)
1043
1044 def evaluate(self, x=None, y=None,
c:\program files\python3x64\lib\site-packages\keras\engine\training_arrays.py in fit_loop(model, f, ins, out_labels, batch_size, epochs, verbose, callbacks, val_f, val_ins, shuffle, callback_metrics, initial_epoch, steps_per_epoch, validation_steps)
197 ins_batch[i] = ins_batch[i].toarray()
198
--> 199 outs = f(ins_batch)
200 if not isinstance(outs, list):
201 outs = [outs]
c:\program files\python3x64\lib\site-packages\keras\backend\tensorflow_backend.py in __call__(self, inputs)
2659 return self._legacy_call(inputs)
2660
-> 2661 return self._call(inputs)
2662 else:
2663 if py_any(is_tensor(x) for x in inputs):
c:\program files\python3x64\lib\site-packages\keras\backend\tensorflow_backend.py in _call(self, inputs)
2629 symbol_vals,
2630 session)
-> 2631 fetched = self._callable_fn(*array_vals)
2632 return fetched[:len(self.outputs)]
2633
c:\program files\python3x64\lib\site-packages\tensorflow\python\client\session.py in __call__(self, *args)
1452 else:
1453 return tf_session.TF_DeprecatedSessionRunCallable(
-> 1454 self._session._session, self._handle, args, status, None)
1455
1456 def __del__(self):
c:\program files\python3x64\lib\site-packages\tensorflow\python\framework\errors_impl.py in __exit__(self, type_arg, value_arg, traceback_arg)
517 None, None,
518 compat.as_text(c_api.TF_Message(self.status.status)),
--> 519 c_api.TF_GetCode(self.status.status))
520 # Delete the underlying status object from memory otherwise it stays alive
521 # as there is a reference to status from this from the traceback due to
InvalidArgumentError: indices[62,0] = 15757 is not in [0, 4704)
[[Node: embedding_15/embedding_lookup = GatherV2[Taxis=DT_INT32, Tindices=DT_INT32, Tparams=DT_FLOAT, _class=["loc:@training_14/RMSprop/Assign_1"], _device="/job:localhost/replica:0/task:0/device:CPU:0"](embedding_15/embeddings/read, embedding_15/Cast, training_14/RMSprop/gradients/embedding_15/embedding_lookup_grad/concat/axis)]]
I've tried to find a solution to this, but to no avail.
The data I am using are One Hot Encoded and normalized netflows.
At least in terms of NaN and Infinity-values I should at least be good to go.
Here's a glance at the input data:
In [14]:
print(flows_nd.shape)
print(type(flows_nd))
print(type(flows_nd[0]))
(225745, 4704)
<class 'numpy.ndarray'>
<class 'numpy.ndarray'>
This is the network itself:
from keras.models import Sequential
from keras.layers import Dense, Embedding
from keras.layers import LSTM
model = Sequential()
model.add(Embedding(flows_nd.shape[-1], 32))
model.add(LSTM(32))
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='rmsprop',
loss='binary_crossentropy',
metrics=['acc'])
history = model.fit(flows_nd, enc_labels,
epochs=1,
batch_size=128,
validation_split=0.2)
From what I've read there could be something wrong with the vocab size in tensorflow, but I am not certain that I can change that out of my Keras layer.
I've assembled a Jupyter Notebook with my current code, if anyone wants a closer look.
Any help is greatly appreciated. Thanks in advance!
Solution:
As Mitiku pointed out, I was passing the number of features into the embedding layer, not the vocabulary size (Keras Embedding Layer Documentation). This is how I updated my code to a working example:
#find the maximum vocabulary size
voc_size = (flows_scaled.max()+1).astype('int64')
print(voc_size)
# build the model
from keras.models import Sequential
from keras.layers import Dense, Embedding
from keras.layers import LSTM
model = Sequential()
model.add(Embedding(voc_size, 32))
model.add(LSTM(32))
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='rmsprop',
loss='binary_crossentropy',
metrics=['acc'])
history = model.fit(flows_scaled, enc_labels,
epochs=1,
batch_size=128,
validation_split=0.2)
Embedding layer expects the first argument to be size of vocabulary, i.e it is the maximum integer index + 1
But you are passing the number of features as vocabulary size. What you can do is, find the maximum number in flows_nd and pass the maximum number plus one to the Embedding layer.
voc_size = flows_nd.max()+1
model = Sequential()
model.add(Embedding(voc_size, 32))