What is the easiest way to compare 2 files and deleting the duplicates found on both files. I'm aware you can do this easily on a linux box with certain commands, but how would I go about achieving this on a windows computer? I would of thought there would be a way to do this on Notepad++ but I can't seem to find one.
The other answers on here don't support a user on windows or don't work.
I'm aware you can do this easily on a Linux box with certain commands, but how would I go about achieving this on a Windows computer?
GNUWin32
If you're willing to install additional third-party software, the GNUWin (GNUWin32) project offers Windows ports of a number of these utilities (commands).
The utilities themselves are offered in packages with either a single utility or several utilities bundled in each. Each package can be installed separately 1, with the exception of certain packages where the included utilities are dependent on others.
The basic package you will most likely want to install is the CoreUtils package but there are other packages available as well (depending on what utilities you may wish to use).
For what it's worth, I've included some simple example batch files using GNUWin32 utilities which may help you in another section of this answer.
GNUWin32 Alternatives
Setting aside the Windows Subsystem for Linux (WSL) in Windows 10, there are other possible options for using these types of utilities on Windows, such as Git for Windows and Cygwin.
These versions often have the advantage of being more up-to-date than their GNUWin32 counterparts, but arguably may be lacking in ease of installation and use.
1 The GNUWin32 Project offers a download utility called GetGnuWin32, which can handle downloading and installing all available GNUWin32 packages in one go (if that is your preferred option).
Notepad++
I am personally not aware of any Notepad++ plugins or features that fully support what you're trying to accomplish. However, Notepad++ can run batch scripts, commands and executables directly via the Notepad++ Run... menu ( F5 ) or the NppExec plugin and thus can be used (to some degree) in concert with e.g. GNUWin32 utilities.
With that said, I am not sure of a good mechanism for feeding separate open files as a group to anything listed above. One partial workaround is to combine the two files into one, sort them, save the file and then run a variation of the "Single File Output" example batch file given later in this answer against that saved file.
Notepad++ does support more low-level interactions with the editor via the Python Script for Notepad++ plugin (based around a self-contained version of Python 2.7). Unfortunately, my experience with that plugin is limited so I am uncertain how it might be able to help you towards the desired functionality (if at all).
If you want to experiment with the Python Script for Notepad++ plugin yourself, you can install it under the Plugins → Plugins Admin... interface in Notepad++.
GNUWin32 Batch Examples
The batch (.bat) file examples given below leverage GNUWin32 utilities and are intended show some rudimentary approaches to (hopefully) accomplishing your goals. But they are simply references. They can almost certainly be improved upon or even ignored all together in favor of other approaches.
These batch files assume the GNUWin32 CoreUtils package is installed (for the sort, uniq and comm utilities). The example which uses comm also requires sed (a separate package).
For completeness, the following (minimal) text files were used as test input for the example scripts outlined below:
ex. dup1.txt
abc
def
123
ghi
Unique Line: 123
--
ex. dup2.txt
ghi
abc
jkl
def
Unique Line: jkl
Example Batch - Single File Output
In conjunction with e.g. the GNUWin32 CoreUtils package, you can create a Windows batch file that supports dragging/dropping two text files onto it, then having the non-duplicated lines displayed as a third, single file opened in ex. Notepad++:
ex. nodupes.bat
@ECHO Off
@REM Two input files (paths)
gnu-sort %1 %2 | uniq -u > uniq_text.txt
@REM Check our command output.
:: PAUSE
START notepad++.exe uniq_text.txt
Note that in the example above, gnu-sort is the GNUWin32 sort utility, renamed to avoid conflicts in Windows (see the Windows Conflicts section).
While this batch file can be used in a "drag and drop" mode for convenience, you can also pass the relevant file names directly at the command line e.g. nodupes.bat dup1.txt dup2.txt. This is true for the other batch examples in this section as well.
Example Batch - Single File Output (Notepad++)
To achieve something similar more directly in Notepad++, you can create a new batch file similar to the one already described above but with a single input:
ex. nodupes-npp.bat
@ECHO Off
@REM One input file (path)
gnu-sort %1 | uniq -u > C:\path\to\uniq_text.txt
START notepad++ C:\path\to\uniq_text.txt
You would then save this .bat file in a static location for Notepad++ to reference. Then, in Notepad++ e.g. under the Run... menu ( F5 ), you could invoke this batch file for whatever (single) file was currently open:
cmd /c C:\path\to\nodupes-npp.bat "$(FULL_CURRENT_PATH)"
A couple caveats exist to this approach, however:
1) The current file open would need to be a combined version of the two files you wished to compare data from.
2) The file you wish to apply this to must be saved before using this .bat/Run... approach. This is because:
"$(FULL_CURRENT_PATH)" isn't available in Notepad++ until the file is saved.
If changes are made to the file, any updates need to be applied to the actual file being accessed (i.e. the version on disk, not the version currently open in Notepad++).
Example Batch - Multiple File Output
If you want unique lines divided (relatively) by their originating files, you can instead use a combination of comm and sed:
ex. nodupes-separated.bat
@ECHO Off
@REM Use sed to divide our comm -3 output into two files,
@REM using a simple sed script (filter.sed).
gnu-sort %1 > sorted_duplicates_1.txt
gnu-sort %2 > sorted_duplicates_2.txt
comm -3 sorted_duplicates_1.txt sorted_duplicates_2.txt | sed -f filter.sed
@REM Check our command output.
:: PAUSE
@REM Clean up our temporary files.
DEL sorted_duplicates_1.txt
DEL sorted_duplicates_2.txt
@REM These file names are defined in ex. filter.sed
START notepad++ uniq_text_1.txt
START notepad++ uniq_text_2.txt
ex. filter.sed
/^\t/ {
s///
w uniq_text_2.txt
d
}
/^[^\t]/ {
w uniq_text_1.txt
d
}
Note that while comm is part of the CoreUtils package, sed is a separate download with GNUWin32. Keep in mind that the batch file above won't alter your original files, it will create two new ones (so renaming them is up to your). Also, as written, ex. filter.sed need to be in the same directory as your batch file (e.g. nodupes-separated.bat).
Windows Conflicts (GNUWin32)
GNUWin32 contains some utilities with same name as Windows utilities or commands (e.g. sort). This can present a (mild) issue, as each utility may operate differently. The conflicts I am personally aware of in the GNUWin32 CoreUtils package are:
Keep in mind there may be other conflicts in separate packages as well, e.g. tree.
If you want to access the GNUWin32 versions from your environment path (i.e. without typing their full path), you can either rename the appropriate GNUWin32 executables themselves (recommended) or use surrogate batch files whose names are different (less desirable). A surrogate batch file would simply contain e.g.:
ex. gnu-utility.bat
@ECHO Off
C:\path\to\GNUWin32\bin\utility.exe %*
Using Windows Utility Versions
You may be able to use the Windows version of a utility, but they can require reworking how you approach things. For instance, Windows sort by default can only take one input at a time, so the original nodupes.bat example above might look like the following when using it:
ex. nodupes-win-sort.bat
@ECHO Off
@REM Helps compensate for a minor issue with cat and newlines
ECHO. > blank
@REM Using Windows sort, not GNUWin32 sort
cat %1 blank %2 | sort | uniq -u > uniq_text.txt
@REM Clean up our temporary file.
DEL blank
START notepad++.exe uniq_text.txt
Notes (GNUWin32)
For the GNUWin32 utilities to work without specifying a full path, you will want to add them to your environment path in Windows. Otherwise, you will need to specify full paths to each executable (be aware this may cause headaches ex. with piping).
GNUWin32 packages are arguably best placed in a "regular" Windows folder you create (i.e. not C:\Program Files, C:\Program Files (x86) or your User folder).
You should likewise try to install GNUWin32 into a path without spaces (e.g. C:\Programs\GNUWin32).
While the GNUWin32 CoreUtils package supports a variety of basic tools, there are still quite a number of utilities that are available as their own individual packages or groups of utilities. Also, some utilities may work slightly differently or have more limited options/syntax simply because the underlying OS (Windows) is not e.g. Linux.
