-3

I have a script that downloads about 1000 csv files, sorts them into 5 csv files and sorts those 5 files into 1 csv file.

The script works fine but obviously when you run it you end up with like a thousand csv files wherever you ran the program from. Is there a way to once the script has finished move all the csv files created into a new folder called like "data" or something like that?

Thanks.
user8261831
  • 464
  • 1
  • 4
  • 20

2 Answers2

1

This code worked for me:

import glob, os, shutil

source_dir = 'C:/Users/george/Desktop/my aemo app/a'
dst = 'C:/Users/george/Desktop/my aemo app/b' 
files = glob.iglob(os.path.join(source_dir, "*.csv"))
for file in files:
    if os.path.isfile(file):
        shutil.copy2(file, dst)
user8261831
  • 464
  • 1
  • 4
  • 20
  • Right, thanks. Would you explain how it answers the question, by editing the answer? Code answers are sometimes deleted here, because it is generally not clear what feature of the posted code was the solution. – halfer Jan 23 '19 at 00:26
  • Is that okay with you? – user8261831 Jan 23 '19 at 00:29
  • Would you explain how it is different to what you had originally? Put another way, how would a future reader best learn from this? See Bill's answer for a good example - see how the code is interspersed with explanation. – halfer Jan 23 '19 at 08:35
0

The shutil module lets you move files in Python. Here's the syntax:

# Preferably put this near the top of your script:
import shutil  

# Use this to move a file:
shutil.move("path/to/original/file.csv", "path/to/destination/file.csv")

If all of your .csv files are currently in a directory and you want to generate a list of all their paths (basically the equivalent of "ls *.csv" or "dir *.csv"), you can use Python's os module:

import os

my_csv_files = []
for file in os.listdir("/my_csv_dir"):
    if file.endswith(".csv"):
        my_csv_files.append(os.path.join("/my_csv_dir", file))

You could then iterate through the generated list. Or just replace the append command with the aforementioned move command to move each one to a new folder.

By the way, I see a lot of repeating code in your example. You may want to consider writing a couple of functions to take care of these tasks so that your code is easier to maintain and is 1/3 of the size.

Bill M.
  • 1,388
  • 1
  • 8
  • 16
  • Thanks for your help, as for the repeated code, I am quite new to Python so that is why – user8261831 Jan 22 '19 at 22:33
  • what is `dir` in `my_csv_dir` ? – user8261831 Jan 22 '19 at 22:35
  • I'm just using "my_csv_dir" as a fake name and placeholder. Just replace "`/my_csv_dir`" with the location of where your CSV files are sitting. Assuming they're all being written to the same place. – Bill M. Jan 22 '19 at 22:37
  • Okay, but now where do the files go? – user8261831 Jan 22 '19 at 22:41
  • You mean where are you moving them to? Wherever you tell them to go in your `shutil.move` command. – Bill M. Jan 22 '19 at 22:47
  • Yeah, I edited my question to include my code, can you take a look? Thanks btw – user8261831 Jan 22 '19 at 22:49
  • I'm confused. In your code, you first have `shutil.move("'C:/Users/lehaner/Desktop/my aemo app/a", "C:/Users/lehaner/Desktop/my aemo app/b/a")`, which is saying to look for a FILE named "`a`" in that's in the folder named "`my aemo app`", and to effectively move it to the folder named "`b`". But then AFTER that you're using `os.listdir` to generate a list of .csv files in a FOLDER named `a`. Did you understand my descriptions of what shutil.move and os.listdir actually do? – Bill M. Jan 22 '19 at 23:14
  • Not really, could you explain it? – user8261831 Jan 22 '19 at 23:29
  • How can I achieve my desired result? – user8261831 Jan 22 '19 at 23:32
  • Well you said you end up about 1000 CSV files in the current working directory, and you'd like to move all of them into a folder named, say, "`data`". Right? Then one way to do this is to loop through the names of all of your csv files, then run the `shutil.move` command on each file to move it to the folder. Being able to get lists of files from a directory or move files from one directory to another is something you can learn from looking up what `shutil.move` and `os.listdir` do. – Bill M. Jan 23 '19 at 00:04
  • I have done some more research, what do you think of this (edited in my question now) – user8261831 Jan 23 '19 at 00:05
  • Looks like that would work. Give it a try! – Bill M. Jan 23 '19 at 00:06