QListView takes too long to update when given 100k items

Question

I have a problem with reading file, specific is I want to make a small dictionary. In file which I need to read has content like this:

a   Ph  P6
a   snsr    CA
a b c   fb  Dj
a b c - book    i+  BS
A except B gate oOPa    y
a font  kQ  BU
[....]

It has about 109.000 lines, and file just has size about 2MB. In my QT app, I coded like this to read and add items to QListWidget:

QString sWord;
QFile inFile("C:\\EV\\ev.index");

inFile.open(QIODevice::ReadOnly|QIODevice::Text);
QTextStream in(&inFile);
while(!in.atEnd())
{
     sWord = in.readLine();
     myListWidget->addItem(sWord); //myListWidget is a QListWidget
}

But it reads too long! At first I think reason is my app reads line by line, so I coded its again like this:

QString data;
QStringList listWord;
QFile inFile("C:\\EV\\ev.index");


inFile.open(QIODevice::ReadOnly|QIODevice::Text);
QTextStream in(&inFile);
data.append(in.readAll());
listWord.append(data.split('\n'));
myListWidget->addItems(listWord);
inFile.close();

It works faster!(about 5 seconds since app launched), still long, I want it to read faster. What I have to do?

Answer 1

1. The list widget's layout takes too long. Set your list widget's uniformItemSizes property to true. This avoids expensive layout operations. Another way would be to set the layoutMode property to QListView::Batched. This avoids having to expensively lay out all of the items at once.

2. Don't use a QListWidget if a lower overhead QListView would do.

3. Additions of large numbers of elements should be batched, i.e. do not insert the elements into the model one-by-one. Insert elements from each batch in an atomic operation that emits the rowsInserted or columnsInserted signal only once.

4. You must not do any file loading in the GUI thread. This is a source of bad user experience in a lot of applications and must be discouraged with a heapful of scorn. Don't do it.

Below is a minimal example that takes all of those into account.

// https://github.com/KubaO/stackoverflown/tree/master/questions/filemodel-18548048
#include <QtWidgets>
#include <QtConcurrent>

void makeLines(QBuffer &buf, int count = 1000000) {
   buf.open(QIODevice::WriteOnly | QIODevice::Text);
   char line[16];
   for (int i = 0; i < count; ++i) {
      int n = qsnprintf(line, sizeof(line), "Item %d\n", i);
      buf.write(line, n);
   }
   buf.close();
}

struct StringListSource : QObject {
   Q_SIGNAL void signal(const QStringList &);
   void operator()(const QStringList &data) { emit signal(data); }
   Q_OBJECT
};

int main(int argc, char *argv[]) {
   QApplication app(argc, argv);
   QListView view;
   QStringListModel model;
   StringListSource signal;
   QObject::connect(&signal, &StringListSource::signal, &model, &QStringListModel::setStringList);
   QtConcurrent::run([&signal]{
      QBuffer file;
      signal({"Generating Data..."});
      makeLines(file);
      signal({"Loading Data..."});
      QStringList lines;
      if (file.open(QIODevice::ReadOnly | QIODevice::Text))
         while (!file.atEnd())
            lines.append(QString::fromLatin1(file.readLine()));
      file.close();
      signal(lines);
   });
   view.setModel(&model);
   view.setUniformItemSizes(true);
   view.show();
   return app.exec();
}
#include "main.moc"