Problem using Pandas for joining dataframes

Question

I am trying to join a lot of CSV files into a single dataframe after doing some conversions and filters, when I use the append method for the sn2 dataframe, the exported CSV contains all the data I want, however when I use the append method for the sn3 dataframe, only the data from the last CSV is exported, what am I missing?

sn2=pd.DataFrame()
sn3=pd.DataFrame()
files=os.listdir(load_path)
for file in files:
    df_temp=pd.read_csv(load_path+file)
    df_temp['Date']=file.split('.')[0]
    df_temp['Date']=pd.to_datetime(df_temp['Date'],format='%Y%m%d%H%M')
    filter1=df_temp['Name']=='Atribute1'
    temp1=df_temp[filter1]
    sn2=sn2.append(temp1)
    filter2=df_temp['Name']=='Atribute2'
    temp2=df_temp[filter2]    
    sn3=pd.concat([temp2])

Please provide enough code so others can better understand or reproduce the problem. — Community, Jun 14 '22 at 01:06

score 1 · Answer 1 · answered Jun 14 '22 at 01:07

1

You have to pass all the dataframes that you want to concatenate to concat:

sn3 = pd.concat([sn3, temp2])

answered Jun 14 '22 at 01:07

Nick

138,499
22
57
95

Problem using Pandas for joining dataframes

1 Answers1