How to remove duplicates from Excel file using pandas

Question

I am working on excel sheet:

animal name    age
dog    puppy   1
dog    doggy   2
dog    snooppy 3
cat    pussy   1
pig    piggy   1
pig    cutty   2
rabit  robby   1
rabbit bunny   2

Here I should check if animal age is 1 should delete that row and print next row and remove duplicates if there are no duplicates, should print that row and this output should print in other excel sheet.

Can any help me out of this?

import 
imatplotlib.pyplot as plt
import pandas as pd

data = pd.read_excel(r"C:\Users\c_ssaiva\Desktop\sampladata.xlsx")
for index,row in data.iterrows():
print(index,row['animal'],row['name'],row['age'])
for j,row in data.iterrows():

 if a[i] == a[j]:
 if a[i] == 1
 print a[j]
 else:
 print (a[i])
 df = data
 df.to_excel(r"C:\Users\c_ssaiva\Desktop\selcol.xlsx")

Is there a particular reason you can't use `df.drop_duplicates`? https://stackoverflow.com/questions/23667369/drop-all-duplicate-rows-in-python-pandas — DeepSpace, Jul 01 '18 at 12:36
output check if row 1 animal = row 2 animal , then check row 1 age = 1, then print row2 .if row1 animal != 1,then print row1 — Shravya Vaggu, Jul 01 '18 at 18:53

score 0 · Answer 1 · answered Jul 01 '18 at 12:48

0

Hej Shrvya. Pandas is awesome and can do all you are asking without loops :) You could do that in one line

df = data[data['age'] != 1].drop_duplicates()

We have made a new df that removes all records where 'age' != 1 and then we drop duplicates :)

I am not sure what is the aim of printing values out. Why do you want to print values on screen?

answered Jul 01 '18 at 12:48

Prayson W. Daniel

14,191
4
51
57

but i need animal whose age = 1 when there are no duplicates – Shravya Vaggu Jul 01 '18 at 13:54
Ah! age_one = data[(data['age']==1)&(~data.duplicated())] – Prayson W. Daniel Jul 01 '18 at 14:05
I need if there are no duplicates for age =1 that animal and other animals which has duplicates should remove ,where age =1 and print next age . – Shravya Vaggu Jul 01 '18 at 14:14
In above I have used two conditions, one asking for the condition that the animal has to be 1 years old and there should not be duplicates – Prayson W. Daniel Jul 01 '18 at 14:14
I am not sure what you want, but if you look at the above answers, you can build your own conditions ;) – Prayson W. Daniel Jul 01 '18 at 14:20
i am looking if first row age is one skip and print next row age, with same animal.if animal has no duplicates then print that row . thanks prayson daniel – Shravya Vaggu Jul 01 '18 at 14:24
Should we begin by removing dublicates, then find animals with age one, then print the next animal’s age of the same kind? Is this correct? – Prayson W. Daniel Jul 01 '18 at 14:32
no first we should check ,if age is 1 then print next animal and drop duplicates. – Shravya Vaggu Jul 01 '18 at 14:39
if no duplicates for animal with age = one then print that animal – Shravya Vaggu Jul 01 '18 at 14:40
Why are we printing those values? – Prayson W. Daniel Jul 01 '18 at 14:45
in a new excelsheet – Shravya Vaggu Jul 01 '18 at 14:55
print(x) just displays the values, it is not needed to generate new file? – Prayson W. Daniel Jul 01 '18 at 16:23
print(x) is not generate file. – Shravya Vaggu Jul 01 '18 at 17:45
df.to_excel(r"") this will generate file – Shravya Vaggu Jul 01 '18 at 17:45
i need just logic how to check check the first row with second , if first row == second row if the column age =1 , if equls print second row else print firstrow – Shravya Vaggu Jul 01 '18 at 17:48

score 0 · Answer 2 · answered Jun 24 '22 at 15:38

0

Or you can use this simple:

df.drop_duplicates(subset=['column'], keep='first', inplace=True)

answered Jun 24 '22 at 15:38

Bruno Slamek

1

How to remove duplicates from Excel file using pandas

2 Answers2