Pandas - add a row at the end of a for loop iteration

Question

So I have a for loop that gets a series of values and makes some tests:

list = [1, 2, 3, 4, 5, 6]
df = pd.DataFrame(columns=['columnX','columnY', 'columnZ'])
for value in list:
    if value > 3:
       df['columnX']="A"
    else:
       df['columnX']="B"
       df['columnZ']="Another value only to be filled in this condition"
    df['columnY']=value-1

How can I do this and keep all the values in a single row for each loop iteration no matter what's the if outcome? Can I keep some columns empty?

I mean something like the following process:

[create empty row] -> [process] -> [fill column X] -> [process] -> [fill column Y if true] ...

Like:

[index columnX columnY columnZ]
[0        A       0      NULL ]
[1        A       1      NULL ]
[2        B       2     "..." ]
[3        B       3     "..." ]
[4        B       4     "..." ]

*keep all the values in a single row for each loop iteration* - what do you mean? — RomanPerekhrest, Jul 04 '19 at 15:25
already answered here : https://stackoverflow.com/questions/10715965/add-one-row-to-pandas-dataframe — SM Abu Taher Asif, Jul 04 '19 at 15:27
@Asif I'm getting: KeyError: 'the label [0] is not in the [index]' when attempting to "append" a new row in the fashion from the question you mentioned. What I want is more like: create emtpy row and fill the row according to a series of procedures. — Tiago Duque, Jul 04 '19 at 15:30

Donbeo · Accepted Answer · 2019-07-04T15:36:47.943

I am not sure to understand exactly but I think this may be a solution:

list = [1, 2, 3, 4, 5, 6]
d = {'columnX':[],'columnY':[]}
for value in list:
    if value > 3:
       d['columnX'].append("A")
    else:
       d['columnX'].append("B")
    d['columnY'].append(value-1)
df = pd.DataFrame(d)

for the second question just add another condition

list = [1, 2, 3, 4, 5, 6]
d = {'columnX':[],'columnY':[], 'columnZ':[]}
for value in list:
    if value > 3:
       d['columnX'].append("A")
    else:
       d['columnX'].append("B")
    if condition:
        d['columnZ'].append(xxx)
    else:
        d['columnZ'].append(None)
df = pd.DataFrame(d)

Please, check my edits, I added another question related to "leaving empty columns". — Tiago Duque, Jul 04 '19 at 15:34

score 0 · Answer 2 · answered Jul 04 '19 at 15:51

According to the example you have given I have changed your code a bit to achieve the result you shared:

list = [1, 2, 3, 4, 5, 6]
df = pd.DataFrame(columns=['columnX','columnY', 'columnZ'])
for index, value in enumerate(list):
    temp = []
    if value > 3:
       #df['columnX']="A"
       temp.append("A")
       temp.append(None)
    else:
       #df['columnX']="B"
       temp.append("B")
       temp.append("Another value") # or you can add any conditions
    #df['columnY']=value-1
    temp.append(value-1)
    df.loc[index] = temp

print(df)

this produce the result:

  columnX        columnY  columnZ
0       B  Another value      0.0
1       B  Another value      1.0
2       B  Another value      2.0
3       A           None      3.0
4       A           None      4.0
5       A           None      5.0

df.index is printed as : Int64Index([0, 1, 2, 3, 4, 5], dtype='int64')

So i'm forced to fill all the columns, even the ones that I have no value? Im asking this because in the original solution I have many columns and many of them will be empty. — Tiago Duque, Jul 04 '19 at 16:52
None is equivalent to null / no value in python (https://stackoverflow.com/questions/3289601/null-object-in-python) — SM Abu Taher Asif, Jul 04 '19 at 17:03

score 0 · Answer 3 · answered Jul 04 '19 at 15:53

You may just prepare/initialize your Dataframe with an index depending on input list size, then getting power from np.where routine:

In [111]: lst = [1, 2, 3, 4, 5, 6]
     ...: df = pd.DataFrame(columns=['columnX','columnY', 'columnZ'], index=range(len(lst)))

In [112]: int_arr = np.array(lst)

In [113]: df['columnX'] = np.where(int_arr > 3, 'A', 'B')

In [114]: df['columnZ'] = np.where(int_arr > 3, df['columnZ'], '...')

In [115]: df['columnY'] = int_arr - 1

In [116]: df
Out[116]: 
  columnX  columnY columnZ
0       B        0     ...
1       B        1     ...
2       B        2     ...
3       A        3     NaN
4       A        4     NaN
5       A        5     NaN

Pandas - add a row at the end of a for loop iteration

3 Answers3