1

I have a df which looks like the following:

Group. Score.
red 34
blue 42
green 1000
green 34
blue 34
red 42

I would like to add a column onto this which specifies if the value is an outlier. If there were no groups then I would use something like:

df['outliers'] = df[df[col] > df[col].mean() + 3 * df[col].std()]

But how would I do this so it is within the groups?

Barney Cooper
  • 43
  • 6
  • look into the where clause for pandas. https://www.geeksforgeeks.org/python-pandas-dataframe-where/ – Justin Oberle Apr 26 '21 at 13:54
  • Does this answer your question? [Checking a Pandas Dataframe for Outliers](https://stackoverflow.com/questions/48087534/checking-a-pandas-dataframe-for-outliers) – Irfan Bilir Apr 26 '21 at 13:59
  • Almost but not quite. Because I have different groups I need to compare each value against the mean of that group, not against the mean of the whole column. – Barney Cooper Apr 26 '21 at 14:05

1 Answers1

3

You can use GroupBy.transform:

df["is_outlier"] = df.groupby("Group.").transform(lambda x: (x - x.mean()).abs() > 3*x.std())

In each group, we take the distance of elements from the group mean and see if its absolute value exceeds 3 times std of the group.

Mustafa Aydın
  • 17,645
  • 4
  • 15
  • 38