How do I create test and train samples from one dataframe with pandas?

python python-2.7 pandas dataframe
tooty44 picture tooty44 · Jun 10, 2014 · Viewed 435.9k times · Source

I have a fairly large dataset in the form of a dataframe and I was wondering how I would be able to split the dataframe into two random samples (80% and 20%) for training and testing.

Thanks!

Answer

gobrewers14 picture gobrewers14 · Jun 11, 2014

scikit learn's train_test_split is a good one.

from sklearn.model_selection import train_test_split

train, test = train_test_split(df, test_size=0.2)

Related questions

Why isn't my Pandas 'apply' function referencing multiple columns working?

I have some problems with the Pandas apply function, when using multiple columns with the following dataframe df = DataFrame ({'a' : np.random.randn(6), 'b' : ['foo', 'bar'] * 3, 'c' : np.random.randn(6)}) and the following function def my_test(a, b): return …

Read More
How to display pandas DataFrame of floats using a format string for columns?

I would like to display a pandas dataframe with a given format using print() and the IPython display(). For example: df = pd.DataFrame([123.4567, 234.5678, 345.6789, 456.7890], index=['foo','bar','baz','quux'], columns=['cost']) print df cost foo 123.4567 bar 234.5678 baz 345.6789 quux 456.7890 I would like …

Read More
Pandas OHLC aggregation on OHLC data

I understand that OHLC re-sampling of time series data in Pandas, using one column of data, will work perfectly, for example on the following dataframe: >>df ctime openbid 1443654000 1.11700 1443654060 1.11700 ... df['ctime'] = pd.to_datetime(df['ctime'], unit='s') df = …

Read More