0

I have two dataframes which I need to concat but for some reason the usual concat, append, joins and merge are all failing;


df1
            A  B  C  D  E  F
01/02/2021  2  3  4  3  5  3
.
.
.
10/02/2021 3  4  5 6 7  8 9

df2
           A  B  C
11/02/2021 3  5  4
12/02/2021 4  6  4
.
.
.
17/04/2021 4  5  3

The second dataframe has values for only 3 of the variables that the first dataframe has but I didn't think this would be an issue if it was joined on the index?

Tried this and getting the below error;


df3 = pd.concat([df,df2], axis=0, ignore_index=True)

ValueError: Plan shapes are not aligned

Any help much appreciated!

spcol
  • 437
  • 4
  • 15
  • Do you mean `df3 = pd.concat([df1, df2], ...` (instead of `df3 = pd.concat([df,df2]`)? I can't replicate your problem? And the examples [here](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.concat.html) suggest that it should work. – Timus Sep 03 '21 at 13:13
  • I found in the issue in that one dataframe had a duplicate column header; Found the problem here; https://stackoverflow.com/questions/26226343/pandas-concat-yields-valueerror-plan-shapes-are-not-aligned – spcol Sep 03 '21 at 13:27

1 Answers1

1

For a "quick and dirty" solution, you can add new columns to df2 with empty/None values.

import pandas as pd
df1 = pd.DataFrame({'A': [1, 2, 3, 4],
                            'B': [1, 2, 3, 4],
                            'C': [1, 2, 3, 4],
                             })          
        
df2 = pd.DataFrame({'A': [1, 2, 3, 4],
                'B': [1, 2, 3, 4]})
                   
df2['C']= None
        
df_final = pd.concat([df1,df2])

Output :

   A  B    C
   1  1  1.0
   2  2  2.0
   3  3  3.0
   4  4  4.0
   1  1  NaN
   2  2  NaN
   3  3  NaN
   4  4  NaN
BlackMath
  • 1,708
  • 1
  • 11
  • 14