0

I have a script that looks for specific images in a network drive, copies them, and zips them into a separate folder (assuming everything is where it needs to be). This worked great when I was testing this on the same set of images locally on my machine; it zips through in less than a second. But then I connected it to the network drive--same exact set of images--and now it inexplicably takes several minutes.

I'm inclined to assume it's the network drive and not my script... but just to be sure, is there anything I'm doing that might potentially be causing a problem? e.g., I'm fairly new to os.walk; is it possible I'm looking in irrelevant folders that's wasting time? (I don't think I am, but)

    import pyodbc, shutil, zipfile, os, pandas as pd
    lanid = input('Please enter your lanid.')
    src = blah\\blah\\blah\\photos
    dst = "C:\\Users\\"+lanid+"\\Documents\\survey"
    os.chdir(src)


    if not os.path.exists(dst):
        os.makedirs(dst)

    [...some stuff about connecting to a sql database here...]

#looks for images in the network drive based off a table taken from the sql database.  if the images exist, they're copied into another folder; if they're not, it adds the missing images to a dictionary, to be reported out on later
    missingimages = {}
    def copyimages():
        for index, row in df.iterrows():
            personalityID = row['personalityID']
            personalityID = str(personalityID)
            name = row['name']
            try:
                shutil.copy(str(personalityID)+'.jpg', dst)
            except:
                try:
                    shutil.copy(str(personalityID)+'.png', dst)
                except:
                    try:
                        shutil.copy(str(personalityID)+'.jpeg', dst)
                    except:
                        missingimages[personalityID] = name
                        continue
        return missingimages

#if the missingimages dictionary is empty, the copied images are zipped.                       
    def zipimages():        
        if not bool(missingimages):
            zipzip = zipfile.ZipFile("C:\\Users\\"+lanid+"\\Documents\\surveyID_"+surveyID+"_"+date+".zip", mode='w')
            for foldername, subfolders, filenames in os.walk(dst):
                for filename in filenames:
                    zipzip.write(filename)
            zipzip.close()
            print("The file is zipped and ready to go!")
            status = 'completed'

#if the dictionary indicates some images are missing, then it says so
        else:
            print("There are no images for the following personalities.  Please save images of them in the Photos folder. \n")
            missingnames = pd.DataFrame.from_dict(missingimages,orient='index')
            missingnames.reset_index(inplace=True)
            missingnames.columns = ['personalityID','name']
            print(missingnames)
            status = 'not completed'
        return status

#run the above functions   
    copyimages()
    status = zipimages()

#prompts the user to restart the script if it didn't complete successfully
    while status == 'not completed':
        cont = str(input('\n Some images are missing.  Once they have been placed in the Photos folder, please enter Y to restart the script.'))
        if cont == 'Y':
            copyimages()
            status = zipimages()
        elif cont != 'Y':
            break
oddnumberedcat
  • 91
  • 3
  • 9
  • 1
    Where is the script hanging? If it's taking minutes `Ctrl+C` should be enough not interrupt the operation and look at the traceback for more info. Some [simple profiling](https://stackoverflow.com/a/582337/7675174) would help you identify the problem. – import random Jul 18 '17 at 03:22
  • How large are these files? – Grimmy Jul 18 '17 at 03:56
  • Maybe this? https://stackoverflow.com/questions/21799210/python-copy-larger-file-too-slow – Grimmy Jul 18 '17 at 03:57
  • You should also monitor your network speeds and try to do a rough estimate of how long a copy should take. It's easy to overestimate how fast data can travel over a network compared to doing copies locally. – Grimmy Jul 18 '17 at 04:01
  • These are some pretty small files; about 50 images, around ~5kb each. So, nothing too crazy here. I don't think it's even the copying that's the problem, though. I tried rewriting my script just to check whether the files exist, and it's still just as slow. To your point, however, perhaps my VPN is mucking things up...? – oddnumberedcat Jul 18 '17 at 04:13

0 Answers0