I'm trying to build a model as ilustrated in below diagram. The idea is to take more than one categorical features (one-hot vectors) and embed them separately, then combine those embedded vectors with a 3D tensor for a LSTM.
With following code in Keras2.0.2, when creating the Model()
object with multiple inputs, it raises an AttributeError: 'NoneType' object has no attribute 'inbound_nodes'
similar to this question. Can anyone help me to figure out what's the problem?
Model:
Code:
from keras.layers import Dense, LSTM, Input
from keras.layers.merge import concatenate
from keras import backend as K
from keras.models import Model
cat_feats_dims = [315, 14] # Dimensions of the cat_feats
emd_inputs = [Input(shape=(in_size,)) for in_size in cat_feats_dims]
emd_out = concatenate([Dense(20, use_bias=False)(inp) for inp in emd_inputs])
emd_out_3d = K.repeat(emd_out, 10)
lstm_input = Input(shape=(10,5))
merged = concatenate([emd_out_3d,lstm_input])
lstm_output = LSTM(32)(merged)
dense_output = Dense(1, activation='linear')(lstm_output)
model = Model(inputs=emd_inputs+[lstm_input], outputs=[dense_output])
#ERROR MESSAGE
Traceback (most recent call last):
File "C:\Program Files\Anaconda2\envs\mle-env\lib\site-packages\IPython\core\interactiveshell.py", line 2881, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-2-a9da7f276aa7>", line 14, in <module>
model = Model(inputs=emd_inputs+[lstm_input], outputs=[dense_output])
File "C:\Program Files\Anaconda2\envs\mle-env\lib\site-packages\keras\legacy\interfaces.py", line 88, in wrapper
return func(*args, **kwargs)
File "C:\Program Files\Anaconda2\envs\mle-env\lib\site-packages\keras\engine\topology.py", line 1648, in __init__
build_map_of_graph(x, seen_nodes, depth=0)
File "C:\Program Files\Anaconda2\envs\mle-env\lib\site-packages\keras\engine\topology.py", line 1644, in build_map_of_graph
layer, node_index, tensor_index)
File "C:\Program Files\Anaconda2\envs\mle-env\lib\site-packages\keras\engine\topology.py", line 1644, in build_map_of_graph
layer, node_index, tensor_index)
File "C:\Program Files\Anaconda2\envs\mle-env\lib\site-packages\keras\engine\topology.py", line 1639, in build_map_of_graph
next_node = layer.inbound_nodes[node_index]
AttributeError: 'NoneType' object has no attribute 'inbound_nodes'
keras.backend.repeat is a function, not a layer. Try using keras.layers.core.RepeatVector instead. It has the same functionality as the function.
emd_out_3d = RepeatVector(10)(emd_out)