How to list all files inside a directory including sub-directories efficiently?

Question

I'm working on a gallery app which displays all the images in a phone or a pen-drive. I successfully managed to list all the images and displays it into the app. But I think it is pretty slow. I'm using Depth First Search technique inside an AsyncTask. So is there any other method which can be used inside AsyncTask which is much faster. Here root is a DocumentFile which is made from a tree URI.

Here is the code I've used.

public class ImageBackgroundTask extends AsyncTask<Object, Object, ArrayList<DocumentFile>> {
DocumentFile root;
ArrayList<DocumentFile> result;
ProgressDialog pg;
Context context;
private AsyncTaskCompleteListener<ArrayList<DocumentFile> > callback;

ImageBackgroundTask(DocumentFile root, Context context, AsyncTaskCompleteListener<ArrayList<DocumentFile>> cb){
    this.context=context;
    this.root=root;
    this.callback = cb;

}
@Override
protected ArrayList<DocumentFile> doInBackground(Object... voids) {
    Queue<DocumentFile> stack=new ArrayDeque<>();
    ArrayList<DocumentFile> list=new ArrayList<>();
    for(DocumentFile f:root.listFiles()){
        stack.add(f);
    }
    while(!stack.isEmpty()){
        DocumentFile child=stack.remove();
        if(child.isDirectory()){
            for(DocumentFile file:child.listFiles()){
                stack.add(file);
            }
        }
        else if(child.isFile()){
            String name=child.getName();
            if(name.endsWith(".jpg")
                    || name.endsWith(".png")
                    || name.endsWith("jpeg")
                    || name.endsWith("JPG")
                    || name.endsWith("JPEG")
                    || name.endsWith("PNG"))
                list.add(child);
        }
    }
    return list;
}

@Override
protected void onPreExecute() {
    pg=new ProgressDialog(context);
    pg.setMessage("Loading...");
    pg.show();

}

@Override
protected void onProgressUpdate(Object... values) {
    super.onProgressUpdate(values);
}

@Override
protected void onPostExecute(ArrayList<DocumentFile> aVoid) {
    pg.dismiss();
    result=aVoid;
    callback.onTaskComplete(result);

}

Here is the output.

Checkout the GIF

Well shouldn't you tell how you obtained 'root' and what it contains? Your post is pretty unclear as you might know meanwhile. — greenapps, Nov 18 '17 at 14:20
Here root is a DocumentFile which i have created from TreUri so basically when user picks a drirectory from DocumentProvider that Directory's Uri is converted into a DocumentFile which is a root file int this case it is send to the ImageBackgroundTask class for scanning Images. — Vivek Patel, Nov 18 '17 at 14:37
Listing files using DocumentFile::listFiles() is notoriously slow and you should use a completely different technique for listing files. But before that: `child.getName()` is very very very slow. And you call it six times for every file. I bed that if you change that to String name = child.getName(); And then compare your variable name to the extensions that you need less than half the time you need now. Please try and report. — greenapps, Nov 18 '17 at 14:41
`when user picks a drirectory from DocumentProvider` Are you not using Intent.ACTION_OPEN_DOCUMENT_TREE? — greenapps, Nov 18 '17 at 14:44
Updated `child.getName()` part, I think there is a slight performance improvement. Thanks for it. — Vivek Patel, Nov 18 '17 at 14:52
You should have started your post with that. How many seconds does it take for how many files? — greenapps, Nov 18 '17 at 14:56
6 seconds to be precise and that progress bar is taking extra time to dismiss. — Vivek Patel, Nov 18 '17 at 15:30

greenapps · Accepted Answer · 2017-11-18T15:41:50.177

8

Do not use DocumentFile.listFiles() to list the files for a tree uri you obtained with Intent.ACTION_OPEN_DOCUMENT_TREE.

As it is notoriously slow.

Instead use functions from DocumentsContract.

Look at the void traverseDirectoryEntries(Uri rootUri) function from Issues traversing through directory hierarchy with Android Storage Access Framework / DocumentProvider using MTP

Collect the child uri for every file instead of trying to obtain a DocumentFile for it.

Then later you can use that child uri to load the image.

If you need six seconds now then it will be less than a second with DocumentsContract i think.

edited Nov 18 '17 at 15:41

answered Nov 18 '17 at 15:36

greenapps

11,154
2
16
19

HolyMoly!!!! Man this is awesome. Like Lightning Fast. Can't believe it is loading in under a second. A big thumbs up. Thank you man. Really Appreciate. – Vivek Patel Nov 18 '17 at 18:38

Mohammad Zarei · Answer 2 · 2017-11-18T15:00:26.603

-1

This is how I usually do these kinds of things:

public ArrayList<String> getFile(File directory) {
File listFile[] = directory.listFiles();
if (listFile != null && listFile.length > 0) {
    for (File file : listFile) {
        if (file.isDirectory()) {
            getFile(file);
        }
        else {
            if (file.getName().endsWith(".png")
                    || file.getName().endsWith(".jpg"))
            {
                String temp = file.getPath().substring(0, file.getPath().lastIndexOf('/'));
                if (!fileList.contains(temp))
                    fileList.add(temp);
            }
        }
    }
}
return fileList;
}

edited Nov 18 '17 at 15:00

answered Nov 18 '17 at 14:29

Mohammad Zarei

1,773
14
33

I've used this method earlier but as it is a recursive method and faster than my solution but it can't be used in AsyncTask. – Vivek Patel Nov 18 '17 at 14:42
Yeah, you're right....just call this method in your `doInBackground` method....It would be ok I think. – Mohammad Zarei Nov 18 '17 at 14:53
Of course it wil work in an AsyncTask too. But it is a nonsense answer as it uses the File class. Your question is about DocumentFile class. – greenapps Nov 18 '17 at 14:53
He can easily change each file to DocumentFile just by using `fromFile(File file)` method. – Mohammad Zarei Nov 18 '17 at 15:02
I think its a matter of logic. This recursive method also works for DocumentFile. I just want to run in `doInBackground` method but as we know it is not possible. – Vivek Patel Nov 18 '17 at 15:05
I used this method in AsyncTask now just to test it and it worked.... :) – Mohammad Zarei Nov 18 '17 at 15:13
Could you edit your post so that i can try it out. @MohammadZarei – Vivek Patel Nov 18 '17 at 15:21
It does not make much sense to stay with DocumentFile as it is notoriously slow. You already know that when you read my comments. – greenapps Nov 18 '17 at 15:22
Yes @greenapps you are right. But the intention of this app is to load images from USB drives so DocumentFile provides uri directly which make it easy to deal with Extrenal Drive. – Vivek Patel Nov 18 '17 at 15:27
No. Not at all. Stop with DocumentFile. – greenapps Nov 18 '17 at 15:30

How to list all files inside a directory including sub-directories efficiently?

2 Answers2

Linked