How do i combine the results from a while loop winto one output?

Question

I want to get one table out of the while loop instead of multiple tables. I have tried to merge the dataframes, but that only gives multiple tables with 2 rows and not 1 big table with all data. The while loop is used to get the data from each sensor and this data needs to get displayed in one table.

I tried to merge the data in different ways like df.merge and also used other statements to get the data like the if/else and the for statement. The closest I came to the wanted result was using the while statement and combining the results via pd.concat. This gave me tables of 2 rows but not the wanted table with all the results.

The dimensions in both dataframes is the same. The dataframes are made using this:

d = {
    "name": [loc_name], 
    "onder(<"+ str(CO2_low)+ "ppm)": [lessperc], 
    "tussen("+str(CO2_low)+"-"+str(CO2_high)+"ppm)": [betweenperc], 
    "boven(>"+ str(CO2_high)+"ppm)": [moreperc]
}

dummy data before a dataframe is made so this is d in my code:

{'name': ['1.04'], 'onder(<950ppm)': [1.0], 'tussen(950-1100ppm)': [0.0], 'boven(>1100ppm)': [0.0]}
{'name': ['1.05'], 'onder(<950ppm)': [0.98], 'tussen(950-1100ppm)': [0.2], 'boven(>1100ppm)': [0.0]}
{'name': ['1.06'], 'onder(<950ppm)': [0.93], 'tussen(950-1100ppm)': [0.4], 'boven(>1100ppm)': [0.3]}

Here is an example of the wanted result:

Name	<950	950-1100	>1100
1.04	1.0	0.0	0.0
1.05	0.98	0.2	0.0
1.06	0.93	0.4	0.3

Code I am currently using:

#Create the dataframe
while(k==0):
    k = k + 1
#d is used to get the data from the sensors
    d = (sensordata)
    df1 = pd.DataFrame(data=d)

#Add rows for the next sensor/location
else:
    d = (sensordata)
    df2 = pd.DataFrame(data=d)

frames = [df1, df2]
result = pd.concat(frames)
print(result)

what dimensions will `df1` and `df2` have? Are the indices / columns going to be the same? Could you maybe add some dummy data so that we can reproduce the issue? — zeawoas, Jan 20 '22 at 13:56
Please check https://stackoverflow.com/questions/20109391/how-to-make-good-reproducible-pandas-examples/20159305#20159305 — Marco_CH, Jan 20 '22 at 13:56
Simply initialize a list before the loop, append list with each data frame in loop, and run `pd.concat` on list outside loop. — Parfait, Jan 20 '22 at 15:12

Kcode · Accepted Answer · 2022-01-20T16:06:39.580

To illustrate I made a for loop, since the provided code example is limited. What you probably want to do, like @Parfait said, is that you initialize a variable before the loop.

This could be a list which you fill and later on create a dataframe with. Or you can directly append to the dataframe for every d value coming in. The use-case here is not very clear, but it seems to me you want to update for every new d immediately.

Hope this example can help:

d_values = [
{'name': ['1.04'], 'onder(<950ppm)': [1.0], 'tussen(950-1100ppm)': [0.0], 'boven(>1100ppm)': [0.0]},
{'name': ['1.05'], 'onder(<950ppm)': [0.98], 'tussen(950-1100ppm)': [0.2], 'boven(>1100ppm)': [0.0]},
{'name': ['1.06'], 'onder(<950ppm)': [0.93], 'tussen(950-1100ppm)': [0.4], 'boven(>1100ppm)': [0.3]}
]

result = None
for d in d_values:
    if result is None:
        result = pd.DataFrame(data=d)
    else:
        df2 = pd.DataFrame(data=d)
        result = pd.concat([result,df2], ignore_index=True)

Now you will be doing the concat on the result Dataframe which lives outside of the loop.

The reason why you probably only receive 2 results, is because in the next run you are not using the result to update on but again df1 and df2.

Output

   name  onder(<950ppm)  tussen(950-1100ppm)  boven(>1100ppm)
0  1.04            1.00                  0.0              0.0
1  1.05            0.98                  0.2              0.0
2  1.06            0.93                  0.4              0.3

How do i combine the results from a while loop winto one output?

1 Answers1