0

I have to transfer roughly 250gb of data from a Master Drive to lots of USB drives, all of identical size and brand. When I copy using the GUI in Mac OSX, it waits to copy all the files to one drive and then copies the same files to drive 2.

Is there a way to copy the files across so that each file/bit is only read once and then written multiple times? This would presumably be faster than reading the same data over and over again. I have seen this question asked before, but I couldn't understand how the terminal commands worked.

Just so you know, I have very little experience with terminal, so if anyone has any great code, could they please explain how I would adjust it work with for me?

Many thanks

James

James Harding
  • 1
  • 1

2 Answers2

2

I know little about Mac OSX, still I believe its standard tools are similar to ones in Linux.

If it was for Linux/Bash I would try:

tar -c /source/dirA/ /source/file1 | tee >(cd /foo/destination3/ && tar -x) >(cd /bar/destination2/ && tar -x) >(cd /foobar/destination1/ && tar -x) > /dev/null

Every destination will receive dirA/ and file1. You may give more arguments to the first tar (directories or files). In order to add another destination simply put additional tee argument in similar manner.

How does it work? First tar converts directories and files to a single bitstream that can be used in a pipe. The tee command forks that stream; every copy but one is extracted by tar in proper destination. The last copy moves down the pipe; it is discarded into /dev/null. (One may use the last copy for destination0 but the syntax would be different so I decided to keep it simple with tee only).

The code is a starting point. You may need to adjust it to work with OSX.

Kamil Maciorowski
  • 81,893
0

After a little head scratching, my boss reminded me that the free software Davinci Resolve (a colour grading application) now has some data wrangling facilities included. This allows you to select a source folder, multiple destinations and then it will copy each file simultaneously to each drive with one read and multiple writes.

I confirmed this through activity monitor which showed the data written per second to be three times greater than the read, and I was writing to three drives

https://www.blackmagicdesign.com/uk/products/davinciresolve

James Harding
  • 1
  • 1