9

I have a Pandas Data Frame where I would like to filter out all columns which only contain zeros. For example, in the Data Frame below, I'd like to remove column 2:

        0      1      2      3      4
0   0.381  0.794  0.000  0.964  0.304
1   0.538  0.029  0.000  0.327  0.928
2   0.041  0.312  0.000  0.208  0.284
3   0.406  0.786  0.000  0.334  0.118
4   0.511  0.166  0.000  0.181  0.980

How can I do this? I've been trying something like this:

df.filter(lambda x: x == 0)
turtle
  • 7,533
  • 18
  • 68
  • 97
  • 1
    I believe this was answered [here](http://stackoverflow.com/questions/21164910/delete-column-in-pandas-based-on-condition). Hope this helps! – WaveRider Aug 30 '16 at 17:45

1 Answers1

12

The following works for me. It gives a series where column names are now the index, and the value for an index is True/False depending on whether all items in the column are 0.

import pandas, numpy as np
# Create DataFrame "df" like yours...

df.apply(lambda x: np.all(x == 0))

And if you want to actually filter out the 0 values:

df[df.columns[(df != 0).any()]]
ely
  • 74,674
  • 34
  • 147
  • 228
  • 7
    I'd write `df[df.columns[(df != 0).any()]]` instead [had to fix a typo]. It looks like it works in 0.8.1, anyway. – DSM Sep 13 '12 at 17:37
  • The inner syntax `(df!=0).any()` doesn't work. A DataFrame object doesn't have the `any` function, at least not in 0.7.3. You'd have to map that to the columns using `map` or `apply` or something. – ely Sep 13 '12 at 17:41
  • I can't even find 0.7.3 on the website to check, but I'm sure you're right. `pandas` is one of the packages I like to keep up to date because of the rapid development. – DSM Sep 13 '12 at 17:49
  • Thanks for the help. I'm using 0.8.1, so @DSM's solution works as well. – turtle Sep 13 '12 at 18:02
  • Glad to hear that it works in 0.8.1. I think my group will be upgrading is soon (but upgrading is hard for large groups, who are a main consumer of the Pandas code). Are there any roadmap docs about where Pandas sees specific functionality heading in the future? It's reasonably well-documented for an open source library, but I fear if they keep changing things without acknowledging that they used to be bugs in older versions, people are not going to keep up. I have a lot of hacky work-around code that I think 0.8.1 makes obsolete. That's unpleasant. – ely Sep 13 '12 at 18:21