5

AttributeError: 'psycopg2.extensions.cursor' object has no attribute 'fast_executemany'

to_sql() is too slow. so trying to resolve the problem. but when I run the following code I am getting :-

AttributeError: 'psycopg2.extensions.cursor' object has no attribute 'fast_executemany'

@event.listens_for(conn, 'before_cursor_execute')
def receive_before_cursor_execute(conn, cursor, statement, params, context, executemany):
    if executemany:
        cursor.fast_executemany = True
        cursor.commit()
Rajesh Pandya
  • 1,540
  • 4
  • 18
  • 31
RAVI VERMA
  • 93
  • 1
  • 10

1 Answers1

4

use insert with tuples it around 200 time faster than executemany in psycopg

args_str = ','.join(cur.mogrify("(%s,%s,%s,%s,%s,%s,%s,%s,%s)", x) for x in tup)
cur.execute("INSERT INTO table VALUES " + args_str)

its equivalent of 

INSERT INTO table VALUES ('a', 'b', 'c'), ('a', 'b', 'c'), ('a', 'b', 'c'), ('a', 'b', 'c');
frankegoesdown
  • 1,898
  • 1
  • 20
  • 38
  • 1
    I have 13 columns and 900K rows in csv format. I am reading it with pandas. when I run the above code i am geting the mentioned error. I am new to mogrify(). would you plz make your point in detail? https://github.com/pandas-dev/pandas/issues/15276 – RAVI VERMA Dec 26 '18 at 18:21
  • cut it by batches 1k rows and iterate by them @RAVIVERMA – frankegoesdown Dec 26 '18 at 18:24