37

I am using PowerShell 3.

What is best practice for concatenating files?

file1.txt + file2.txt = file3.txt

Does PowerShell provide a facility for performing this operation directly? Or do I need each file's contents be loaded into local variables?

user4003407
  • 21,204
  • 4
  • 50
  • 60
BaltoStar
  • 8,165
  • 17
  • 59
  • 91
  • Possible duplicate of [How do I concatenate two text files in PowerShell?](http://stackoverflow.com/questions/8749929/how-do-i-concatenate-two-text-files-in-powershell) – phuclv Mar 29 '16 at 16:46

9 Answers9

54

If all the files exist in the same directory and can be matched by a simple pattern, the following code will combine all files into one.

Get-Content .\File?.txt | Out-File .\Combined.txt
nabrond
  • 1,368
  • 8
  • 17
  • 1
    `Get-Content ... | Out-File ... ` is arguably the clearest expression of *my* intent when I'm doing something like this. It doesn't matter to me whether I can match a simple pattern; there are a lot of ways to select files in PowerShell. The default encoding for Out-File is more often correct for me than the default encoding for Set-Content; Out-File defaults to Unicode, and Set-Content defaults to ASCII. – Mike Sherrill 'Cat Recall' Dec 17 '15 at 15:14
  • 4
    For everyone wondering why this fails on Windows 8+, use this instead: `Get-Content .\*.txt | Out-File .\Combined.txt` – omni Jun 26 '18 at 09:18
  • 1
    Like @masi says, I was confused by the pattern expression. For everyone as dense as me, the question mark (`?`) only matches one character. Use an asterisk (`*`) if you need to match multiple. – Jason Apr 18 '19 at 17:38
  • 6
    That produced a recursive file bomb for me, don't put the output file in the same directory as the input files. – Lennart Jan 17 '20 at 06:45
  • Not very useful because it doesn't show that input files must be separated by commas. – Pedro Machado Nov 07 '21 at 19:58
  • The outfile will be UTF-16, which doubles the file size – anotherfred May 15 '23 at 12:25
33

I would go this route:

Get-Content file1.txt, file2.txt | Set-Content file3.txt

Use the -Encoding parameter on Set-Content if you need something other than ASCII which is the default for Set-Content.

Keith Hill
  • 194,368
  • 42
  • 353
  • 369
8

If you need more flexibility, you could use something like

Get-ChildItem -Recurse *.cs | ForEach-Object { Get-Content $_ } | Out-File -Path .\all.txt
Dan Friedman
  • 4,941
  • 2
  • 41
  • 65
7

Warning: Concatenation using a simple Get-Content (whether or not using -Raw flag) works for text files; Powershell is too helpful for that:

  • Without -Raw, it "fixes" (i.e. breaks, pun intended) line breaks, or what Powershell thinks is a line break.
  • With -Raw, you get a terminating line end (normally CR+LF) at the end of each file part, which is added at the end of the pipeline. There's an option for that in newer Powershells' Set-Content.

To concatenate a binary file (that is, an arbitrary file that was split for some reason and needs to be put together again), use either this:

Get-Content -Raw file1, file2 | Set-Content -NoNewline destination

or something like this:

Get-Content file1 -Encoding Byte -Raw | Set-Content destination -Encoding Byte
Get-Content file2 -Encoding Byte -Raw | Add-Content destination -Encoding Byte

An alternative is to use the CMD shell and use

copy file1 /b + file2 /b + file3 /b + ... destinationfile

You must not overwrite any part, that is, use any of the parts as destination. The destination file must be different from any of the parts. Otherwise you're up for a surprise and must find a backup copy of the file part.

Peter Lindgren
  • 173
  • 1
  • 10
1
gc file1.txt, file2.txt > output.txt

I think this is as short as it gets.

Shadow Man
  • 11
  • 2
  • Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Feb 06 '22 at 09:57
1

In case you would like to ensure the concatenation is done in a specific order, use the Sort-Object -Property <Some Name> argument. For example, concatenate based on the name sorting in an ascending order:

Get-ChildItem -Path ./* -Include *.txt -Exclude output.txt | Sort-Object -Property Name | ForEach-Object { Get-Content $_ } | Out-File output.txt

IMPORTANT: -Exclude and Out-File MUST contain the same values, otherwise, it will recursively keep on adding to output.txt until your disk is full.

Note that you must append a * at the end of the -Path argument because you are using -Include, as mentioned in Get-ChildItem documentation.

M. Al Jumaily
  • 731
  • 1
  • 6
  • 21
0

a generalization based on @Keith answer:

gc <some regex expression> | sc output

yaitloutou
  • 1,691
  • 19
  • 23
  • 2
    while this answer looks like it meant `bash`, the reason for short names are aliases in PowerShell - where `gc` is Get-Content and `sc` is Set-Content. – c69 Aug 14 '17 at 19:21
0

Here is an interesting example of how to make a zip-in-image file based on Powershell 7

Get-Content -AsByteStream file1.png, file2.7z | Set-Content -AsByteStream file3.png
Get-Content -AsByteStream file1.png, file2.7z | Add-Content -AsByteStream file3.png
SekiBetu
  • 1
  • 1
0

None of the examples above worked very well for me because I was dealing with large (5GB+) files. Set-Content makes it first be all loaded in memory (explodes RAM use) and then write it all. Instead, a text stream write is better. I put the below approach to combine a header file and a data file after looking at the answers to this other question!

$file = [system.io.file]::OpenWrite("$($pwd.Path)\out.txt")
$writer = New-Object System.IO.StreamWriter($file)

cat headers.txt,data.txt | ForEach-Object { $writer.WriteLine($_) }

$writer.Close()
$file.Close()
user1690166
  • 55
  • 2
  • 6