Is there a way to append files efficiently using the DOS copy command?

Question

Using the DOS copy command syntax to concatenate files:

copy file1.txt+file2.txt all.txt

I know I can do this...

copy file1.txt+file2.txt file1.txt

Is this efficient? Is it doing what I'm expecting? It works, but I want to know is it actually appending to file1.txt or is it copying file1.txt (bad), concatenating file2 and then renaming to file1.txt (which is not efficient)?

Answer 1

copy is copying file1.txt and file2.txt into memory, concatenating them then writing out to file1.txt. It's not copying to a new file then renaming that file so really there's not much extra disk I/O.

You can also use type.

type file2.txt >> file1.txt

The >> operator appends text. But that will, of course, not work for binary files.

Answer 2

Is this efficient?

Sure. However, using the the /b switch can/may increase performance by simply concatenating the bytes instead of processing the files as text. This is particularly noticeable when concatenating very large text files.

Is it doing what I'm expecting?

Usually yes, but if the file was made in Linux, Mac, or other system with differing file-/line-terminators, then it may give unexpected results. It is a good idea to use the /b switch in general, even for text files.

I want to know is it actually appending to file1.txt or is it copying file1.txt (bad), concatenating file2 and then renaming to file1.txt (which is not efficient)?

Yes, it is creating a new, temporary file, deleting the original, and renaming the temp file to the original name, but deleting and renaming take no time and unless the original file is massive, you won’t normally notice even the (redundant) copying of the original file.

Answer 3

Find base directory where files are located

0. dir /b /s *.xml > dirlist.txt
1. edit each line of dirlist.txt using copy+replace, i.e.-
2. l:\dir1\dir2\file.xml
3. replace l:\ with " m:\dest.txt<br>copy /b m:\dest.txt+"
4. save file as dirlist.htm
5. open file in browser
6. copy the results back to editor
7. fix the first and last lines
8. save the file as .bat
9. open command prompt and run the .bat file

Results of binary copy appended to m:\dest.txt

*Note: I do recall that old MS-DOS would internally hack up files with the same source and destinations (64k). For very large file products it would be better safer to use a temp file. If the OS can't handle the task using a temp file due to internal buffer limits or poor coding, the DOS will fail the task.

*Note: For any appended target file approaching 1 GB, I would suggest copying all of the target files to a temp directory and committing copy /b *.*; target.file.

Answer 4

Files are streams--meaning they really work like a tape deck that you can instantly move anywhere, but you still have to "press play" and consume data serially. So you can do the following:

• Move a seek pointer somewhere between 0 and the end of the file.
• Read where the seek pointer is, moving the seek pointer forward.
• Write where the seek pointer is, moving the seek pointer forward. If you keep going past the end of the file you are making the file bigger.

That's it. Well, one more thing:

• Tell the OS that the seek pointer is the new end of the file. This is a truncate and everything after the end is lost.

If you want to "delete" something in the middle or beginning of the file, you've got to rewrite the whole file - starting from the end and moving to the beginning.

So one thing that traditional files do not do well is shrink from the beginning.

So, it's hard to do an "efficient" append that shrinks the 2nd source file while stitching on the first.

It's definitely possible but you'd have to manipulate filesystem structures to do it in a way that would be worth it timewise - the filesystem strctures are the OS and filesystem-specific stuff you don't see with dir and File Exmplorer. That means each filesystem in existence--NTFS, FAT32, ext2 (Linux)--would need its own method of doing this.

I'm not aware of any Windows "stitching" utilities that exist that allow you to combine files without having space for a third file. There may be some out there, if you do find any, be very wary and test thoroughly as it's basically manipulating files in a nonstandard unsafe way and if the utility doesn't work 100% as advertised, you will lose data!

Answer 5

If they are of the same extension you could do this -

type *.txt >> fileout.tmp.

Then rename fileout.tmp to the proper extension.