How to train large Dataset for classification

Question

I have a training dataset of 1600000 tweets. How can I train this type of huge data.

I have tried something using nltk.NaiveBayesClassifier. It will take more than 5 days to train if I run.

def extract_features(tweet):

    tweet_words = set(tweet)
    features = {}
    for word in featureList:
        features['contains(%s)' % word] = (word in tweet_words)

    return features


training_set = nltk.classify.util.apply_features(extract_features, tweets)

NBClassifier = nltk.NaiveBayesClassifier.train(training_set)  # This takes lots of time  

What should I do?

I need to classify my Dataset using SVM and naive bayes.

Dataset I want to use : Link

Sample(training Dataset):

Label     Tweet
0         url aww bummer you shoulda got david carr third day
4         thankyou for your reply are you coming england again anytime soon

Sample(testing Dataset):

Label     Tweet
4         love lebron url
0         lebron beast but still cheering the til the end
^
I have to predict Label 0/4 only

How can I train this huge dataset efficiently?

Answer 1

Before speeding up the training I'd personally make sure that you actually need to. While not a direct answer to your question, I'll try to provide a different angle which you might or might not be missing (hard to tell from your initial post).

Take e.g. superbly's implementation as a baseline. 1.6Mio training and 500 test samples with 3 features yields 0.35 accuracy.

Using the exact same setup, you can go as low as 50k training samples without losing accuracy, in fact the accuracy will slightly go up - probably because you are overfitting with that many examples (you can check this running his code with a smaller sample size). I'm pretty sure that using a neural network at this stage would give horrible accuracy with this setup (the SVM can be kinda tuned to overcome overfitting though that's not my point).

You wrote in your initial post that you have 55k features (which you deleted for some reason?). This number should correlate with your training set size. Since you didn't specify your list of features it's not really possible to give you a proper working model or test my assumption.

However, I highly suggest that you reduce your training data as a first step and see a) how well you perform and b) at which point possible overfitting occurs. I would also adjust the test size to be of a higher size. 500-1.6Mio is kind of a weird split of the sets. Try 80/20% for train/test. As a third step check your feature list size. Is it representative of what you need? If there's unnecessary/duplicate features in that list, you should consider pruning.

As a final thought, if you come back to longer training sizes (e.g. because you decide that you do in fact need much more data than provided now), consider if slow learning really is an issue (besides testing your model). Many state-of-the-art classifiers are trained for days/weeks using GPU computing. Training time doesn't matter in that case because they're only trained once and possibly only updated with small batches of data when they "go online".