18

How to rename columns with multiple levels after pandas pivot operation?

Here's some code to generate test data:

import pandas as pd
df = pd.DataFrame({
    'c0': ['A','A','B','C'],
    'c01': ['A','A1','B','C'],
    'c02': ['b','b','d','c'],
    'v1': [1, 3,4,5],
    'v2': [1, 3,4,5]})

print(df)

gives a test dataframe:

   c0 c01 c02  v1  v2
0  A   A   b   1   1
1  A  A1   b   3   3
2  B   B   d   4   4
3  C   C   c   5   5

applying pivot

df2 = pd.pivot_table(df, index=["c0"], columns=["c01","c02"], values=["v1","v2"])
df2 = df2.reset_index()

gives

output1

how to rename the columns by joining levels? with format <c01 value>_<c02 value>_<v1>

for example first column should look like "A_b_v1"

The order of joining levels isn't really important to me.

muon
  • 10,786
  • 7
  • 60
  • 76

1 Answers1

29

If you want to coalesce the multi-index into a single string index without caring about the index level order, you can simply map a join function over the columns, and assign the result list back:

df2.columns = list(map("_".join, df2.columns))

And for your question, you can loop through the columns where each element is a tuple, unpack the tuple and join them back in the order you want:

df2 = pd.pivot_table(df, index=["c0"], columns=["c01","c02"], values=["v1","v2"])

# Use the list comprehension to make a list of new column names and assign it back
# to the DataFrame columns attribute.
df2.columns = ["_".join((j,k,i)) for i,j,k in df2.columns]
df2.reset_index()

enter image description here

Psidom
  • 195,464
  • 25
  • 298
  • 322
  • 12
    thanks ```['_'.join(str(s).strip() for s in col if s) for col in df2.columns]``` worked as a general solution, independent of number of levels – muon Feb 07 '17 at 20:20
  • @muon I like your general solution! It's perfect. – vagabond Mar 13 '18 at 21:36