How are neural networks used when the number of inputs could be variable?

Question

All the examples I have seen of neural networks are for a fixed set of inputs which works well for images and fixed length data. How do you deal with variable length data such sentences, queries or source code? Is there a way to encode variable length data into fixed length inputs and still get the generalization properties of neural networks?

Answer 1

I have been there, and I faced this problem. The ANN was made for fixed feature vector length, and so are many other classifiers such as KNN, SVM, Bayesian, etc. i.e. the input layer should be well defined and not varied, this is a design problem. However, some researchers opt for adding zeros to fill the missing gap, I personally think that this is not a good solution because those zeros (unreal values) will affect the weights that the net will converge to. in addition there might be a real signal ending with zeros.

ANN is not the only classifier, there are more and even better such as the random forest. this classifier is considered the best among researchers, it uses a small number of random features, creating hundreds of decision trees using bootstrapping an bagging, this might work well, the number of the chosen features normally the sqrt of the feature vector size. those features are random. each decision tree converges to a solution, using majority rules the most likely class will chosen then.

Another solution is to use the dynamic time warping DTW, or even better to use Hidden Markov models HMM.

Another solution is the interpolation, interpolate (compensate for missing values along the small signal) all the small signals to be with the same size as the max signal, interpolation methods include and not limited to averaging, B-spline, cubic.....

Another solution is to use feature extraction method to use the best features (the most distinctive), this time make them fixed size, those method include PCA, LDA, etc.

another solution is to use feature selection (normally after feature extraction) an easy way to select the best features that give the best accuracy.

that's all for now, if non of those worked for you, please contact me.