Simultaneously melt multiple columns in Python Pandas

Martin Petrov picture Martin Petrov · Jul 25, 2018 · Viewed 18.9k times · Source

wondering if pd.melt supports melting multiple columns. I have the below examples trying to have the value_vars as list of lists but i am getting an error:

ValueError: Location based indexing can only have [labels (MUST BE IN THE INDEX), slices of labels (BOTH endpoints included! Can be slices of integers if the index is integers), listlike of labels, boolean] types

Using pandas 0.23.1.

df = pd.DataFrame({'City': ['Houston', 'Austin', 'Hoover'],
                   'State': ['Texas', 'Texas', 'Alabama'],
                   'Name':['Aria', 'Penelope', 'Niko'],
                   'Mango':[4, 10, 90],
                   'Orange': [10, 8, 14], 
                   'Watermelon':[40, 99, 43],
                   'Gin':[16, 200, 34],
                   'Vodka':[20, 33, 18]},
                 columns=['City', 'State', 'Name', 'Mango', 'Orange', 'Watermelon', 'Gin', 'Vodka'])

Desired output:

      City    State       Fruit  Pounds  Drink  Ounces
0  Houston    Texas       Mango       4    Gin    16.0
1   Austin    Texas       Mango      10    Gin   200.0
2   Hoover  Alabama       Mango      90    Gin    34.0
3  Houston    Texas      Orange      10  Vodka    20.0
4   Austin    Texas      Orange       8  Vodka    33.0
5   Hoover  Alabama      Orange      14  Vodka    18.0
6  Houston    Texas  Watermelon      40    nan     NaN
7   Austin    Texas  Watermelon      99    nan     NaN
8   Hoover  Alabama  Watermelon      43    nan     NaN

I tried that and i get the aforementioned error:

df.melt(id_vars=['City', 'State'], 
        value_vars=[['Mango', 'Orange', 'Watermelon'], ['Gin', 'Vodka']],var_name=['Fruit', 'Drink'], 
        value_name=['Pounds', 'Ounces'])

Answer

jezrael picture jezrael · Jul 25, 2018

Use double melt for each catogories and then concat, but because duplicted values add cumcount for unique triples in MultiIndex:

df1 = df.melt(id_vars=['City', 'State'], 
              value_vars=['Mango', 'Orange', 'Watermelon'],
              var_name='Fruit', value_name='Pounds')
df2 = df.melt(id_vars=['City', 'State'], 
              value_vars=['Gin', 'Vodka'], 
              var_name='Drink', value_name='Ounces')

df1 = df1.set_index(['City', 'State', df1.groupby(['City', 'State']).cumcount()])
df2 = df2.set_index(['City', 'State', df2.groupby(['City', 'State']).cumcount()])


df3 = (pd.concat([df1, df2],axis=1)
         .sort_index(level=2)
         .reset_index(level=2, drop=True)
         .reset_index())
print (df3)
      City    State       Fruit  Pounds  Drink  Ounces
0   Austin    Texas       Mango      10    Gin   200.0
1   Hoover  Alabama       Mango      90    Gin    34.0
2  Houston    Texas       Mango       4    Gin    16.0
3   Austin    Texas      Orange       8  Vodka    33.0
4   Hoover  Alabama      Orange      14  Vodka    18.0
5  Houston    Texas      Orange      10  Vodka    20.0
6   Austin    Texas  Watermelon      99    NaN     NaN
7   Hoover  Alabama  Watermelon      43    NaN     NaN
8  Houston    Texas  Watermelon      40    NaN     NaN