Xcode loop array of URLs with NSURLSessionDataTask

Question

I must be making this harder than it is... or implementing the solutions I see online incorrectly.

I have an array of URLs which I would like to loop through and push the results to a dictionary in order or the array. How can I make it wait for the dictionary to be updated before running the next request? Basically I want to make the calls synchronously in a background thread.

Here is where I call the download:

for (NSString *path in paths) {

    NSURLSession *session = [NSURLSession sessionWithConfiguration [NSURLSessionConfiguration defaultSessionConfiguration]];

    NSURLRequest *request = [[NSURLRequest alloc] initWithURL:[NSURL URLWithString:path]
                                              cachePolicy:NSURLRequestUseProtocolCachePolicy
                                          timeoutInterval:10];

    NSURLSessionDataTask *task = [session dataTaskWithRequest:request
                                            completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
                                                if (error)
                                                {

                                                }
                                                else
                                                {
                                                    NSError *parsingError = nil;
                                                    NSDictionary *dict = [NSJSONSerialization JSONObjectWithData:data
                                                                                                         options:NSJSONReadingAllowFragments
                                                                                                           error:&error];
                                                    if (parsingError)
                                                    {

                                                    }
                                                    else
                                                    {
                                                        [myDictionary addObject:dict];
                                                    }
                                                }
                                            }];
    [task resume];
}

Answer 1

Unless one request really requires the results of the prior request before being issued (which is not the case here), you should not run them sequentially. It may feel more logical to issue the sequentially, but you pay a huge performance penalty to do so. Issue them concurrently, save the results in some unordered structure (like a dictionary), and then when all done, build your ordered structure.

NSMutableDictionary *results = [NSMutableDictionary dictionaryWithCapacity:[paths count]];

// don't create new session for each request ... initialize this outside of the loop

NSURLSession *session = [NSURLSession sessionWithConfiguration:[NSURLSessionConfiguration defaultSessionConfiguration]]; // or since you're not doing anything special here, use `sharedSession`

// since we're going to block a thread in the process of controlling the degree of 
// concurrency, let's do this on a background queue; we're still blocking
// a GCD worker thread as these run (which isn't ideal), but we're only using
// one worker thread.

dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{

    // only do for requests at a time, so create queue a semaphore with set count

    dispatch_semaphore_t semaphore = dispatch_semaphore_create(4); // only do four requests at a time

    // let's keep track of when they're all done

    dispatch_group_t group = dispatch_group_create();

    // now let's loop through issuing the requests

    for (NSString *path in paths) {
        dispatch_group_enter(group);                               // tell the group that we're starting another request

        dispatch_semaphore_wait(semaphore, DISPATCH_TIME_FOREVER); // wait for one of the four slots to open up

        NSURLRequest *request = [[NSURLRequest alloc] initWithURL:[NSURL URLWithString:path]
                                                      cachePolicy:NSURLRequestUseProtocolCachePolicy
                                                  timeoutInterval:10];

        NSURLSessionDataTask *task = [session dataTaskWithRequest:request completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
            if (error) {
                // ...
            } else {
                NSError *parsingError = nil;
                NSDictionary *dict = [NSJSONSerialization JSONObjectWithData:data options:0 error:&error];
                if (parsingError) {

                } else {
                    // synchronize updating of dictionary

                    dispatch_async(dispatch_get_main_queue(), ^{
                        results[path] = dict;
                    });
                }
            }

            dispatch_semaphore_signal(semaphore);                  // when done, flag task as complete so one of the waiting ones can start
            dispatch_group_leave(group);                           // tell the group that we're done
        }];
        [task resume];
    }

    dispatch_group_notify(group, dispatch_get_main_queue(), ^{
        // trigger whatever you want when they're all done

        // and if you want them in order, iterate through the paths and pull out the appropriate result in order
        for (NSString *path in paths) {
            // do something with `results[path]`
        }
    });
});

I tried to reduce the amount of extra dependencies here, so I used dispatch groups and semaphores. In the above, I use semaphores to constrain the degree of concurrency and I use dispatch group to identify when it's all done.

Personally, I wouldn't use semaphores and groups, but rather I'd wrap these requests in asynchronous NSOperation subclass, but I was trying to limit the changes I made to your code. But the NSOperation idea is logically the same as the above: Run them concurrently, but constrain the degree of concurrency so you don't end up with them timing out on you, and trigger the retrieval of the results only when all the results are retrieved.