0

I have two dataframes which are obtained after a lot of pre-processing. I want to combine/merge them row by row and write them directly to a csv file. As the dataframes are huge so using concat and merge options of Pandas results in MemoryError.

Edit: The solutions available on Stackoverflow using merge and concat functions of the pandas are not applicable and results in MemoryError. The dataframes are too large.

Here is an MWE:

DF1

ID F1 F2
1 0 1
2 1 1

DF2

ID F3 F4 Label
1 A B L1
2 C D L2

Resultant CSV File

ID,F1,F2,ID,F3,F4,Label <br>
1,0,1,1,A,B,L1 <br>
2,1,1,2,C,D,L2

Note: Both the dataframes have the same number of rows.

Ahmad
  • 163
  • 1
  • 2
  • 9
  • You would probably need to rephrase the question in terms of possible ways to work around the memory issue for your specific case (with code), or to ask why this takes up so much memory. Otherwise, this could just mean you don't have enough memory, and it's not clear why you expect that you do have enough. – Sasha Kondrashov Sep 24 '22 at 02:26

0 Answers0