3

I am working in SQLite3 using DataMapper.

I was populating a table with almost 1.4 million records and it was taking a long time. I found out that inserts in SQLite are inherently slow and can be improved by:

  1. Setting pragma Synchronous = OFF
  2. Doing inserts in bulk by embedding them in BEGIN..COMMIT blocks.

There are a lot of ways to do this.

I have been trying to find ways to do the same with DataMapper, but the documentation has not helped so far. I searched the internet and there are ways explained to do the same in other environments but no one has talked about DataMapper in Ruby's context.

My Code so far is :

 DataMapper.setup(:default, 'sqlite:///path/to/DataBase.db')

 //Definition of Model

 File.open(fileName, "r") do |infile| 
    while (line = infile.gets) 
              // Populate data in database here
            end
      end
Community
  • 1
  • 1
bhuwansahni
  • 1,834
  • 1
  • 14
  • 20

1 Answers1

0

The problem is not with the code but sqlite itself. If you are populating it with 1.4 million records chances are you are going to be using it in production database.

sqlite is inherently slow for large database querying even with datamapper so it is advisable to use mysql for quick access

http://datamapper.org/getting-started.html

Naman Kamra
  • 21
  • 6