353

I need to recursively search for a specified string within all files and subdirectories within a directory and replace this string with another string.

I know that the command to find it might look like this:

grep 'string_to_find' -r ./*

But how can I replace every instance of string_to_find with another string?

Walf
  • 8,535
  • 2
  • 44
  • 59
billtian
  • 6,589
  • 4
  • 18
  • 28
  • I don't believe grep can do this (I could be wrong). Easier ways would be to use sed or perl to do the replacing – Memento Mori Mar 14 '13 at 06:41
  • 2
    Try to use `sed -i 's/.*substring.*/replace/'` – Eddy_Em Mar 14 '13 at 06:42
  • 2
    @Eddy_Em That will replace the entire line with replace. You need to use grouping to capture the part of the line before and after the substring and then put that in the replacement line. `sed -i 's/\(.*\)substring\(.*\)/\1replace\2/'` – JStrahl Aug 05 '14 at 07:50
  • 2
    Possible duplicate of [Using grep and sed to find and replace a string](https://stackoverflow.com/questions/6178498/using-grep-and-sed-to-find-and-replace-a-string) – jww Oct 25 '17 at 00:25
  • @see https://stackoverflow.com/questions/5171901/sed-command-find-and-replace-in-file-and-overwrite-file-doesnt-work-it-empties – softwarevamp Jun 06 '18 at 01:55

10 Answers10

306

Another option is to use find and then pass it through sed.

find /path/to/files -type f -exec sed -i 's/oldstring/new string/g' {} \;
rezizter
  • 4,908
  • 4
  • 24
  • 32
  • 44
    On OS X 10.10 Terminal, a proper extension string to parameter `-i` is required. For example, `find /path/to/files -type f -exec sed -i "" "s/oldstring/new string/g" {} \;` Anyway, providing empty string still creates a backup file unlike described in manual... – eonil Jun 25 '15 at 02:41
  • 11
    Why do I get "sed: RE error: illegal byte sequence". And yes, I added the `-i ""` for OS X. It works otherwise. – taco Jun 16 '16 at 19:51
  • 2
    I had the illegal byte sequence issue on macOS 10.12, and this question/answer solved my issue: http://stackoverflow.com/questions/19242275/re-error-illegal-byte-sequence-on-mac-os-x. – abeboparebop Apr 11 '17 at 08:12
  • 4
    This touches every file so file times are modified; and converts line endings from `CRLF` to `LF` on Windows. – jww Oct 25 '17 at 00:21
  • My old and new strings have / because they are paths... this won't work. – Emmanuel Goldstein Feb 24 '23 at 06:29
  • Could you not escape the forward slash(/) with a back slash (\)? Please paste what you are searching for and I can send you the query. – rezizter Feb 25 '23 at 07:38
254

I got the answer.

grep -rl matchstring somedir/ | xargs sed -i 's/string1/string2/g'
billtian
  • 6,589
  • 4
  • 18
  • 28
  • 21
    This would scan through the matching files twice... once with `grep` and then again with `sed`. Using `find` method is more efficient but this method you mention does work. – cmevoli Mar 14 '13 at 10:27
  • 54
    On OS X you will need to change `sed -i 's/str1/str2/g'` to `sed -i "" 's/str1/str2/g'` for this to work. – jdf Jul 07 '15 at 22:53
  • 14
    @cmevoli with this method, `grep` goes through all the files and `sed` only scans the files matched by `grep`. With the `find` method in the other answer, `find` first lists all files, and then `sed` will scan through all the files in that directory. So this method is not necessarily slower, it depends on how many matches there are and the differences in search speeds between `sed`, `grep` and `find`. – joelostblom Dec 28 '16 at 17:08
  • 8
    OTOH this way lets you PREVIEW what grep finds BEFORE actually replacing, reducing the risk of failure greatly, especially for regex n00bs like myself – Mr. Developerdude Jan 31 '17 at 00:47
  • 3
    This is also useful when your grep replacement is more clever than sed. For example ripgrep obeys .gitignore while sed doesn't. – user31389 Oct 31 '17 at 13:52
  • What does the `1` after `-r` do? Recursive depth? – Yankee Sep 10 '19 at 01:27
  • @Yankee it's lower-case L, and it means "Suppress normal output; instead print the name of each input file from which output would normally have been printed. The scanning will stop on the first match. (-l is specified by POSIX." http://linuxcommand.org/lc3_man_pages/grep1.html – sanitycheck Jan 06 '20 at 15:59
  • For this reason, it might be a good idea to pass the grep output as a literal string to sed. https://stackoverflow.com/questions/13055889/sed-with-literal-string-not-input-file#13055913 – WoodrowShigeru Aug 04 '21 at 17:33
  • to make the whole thing case-insensitive add `i` flags on both `grep` and `sed`: `grep -ril matchstring somedir/ | xargs sed -i 's/string1/string2/i'` – Nick Andriopoulos Oct 25 '21 at 10:59
71

You could even do it like this:

Example

grep -rl 'windows' ./ | xargs sed -i 's/windows/linux/g'

This will search for the string 'windows' in all files relative to the current directory and replace 'windows' with 'linux' for each occurrence of the string in each file.

Matthias Braun
  • 32,039
  • 22
  • 142
  • 171
Du-Lacoste
  • 11,530
  • 2
  • 71
  • 51
  • 3
    The `grep` is only useful if there are files which should not be modified. Running `sed` on all files will update the file's modification date but leave the contents unchanged if there are no matches. – tripleee Jun 11 '16 at 08:10
  • 1
    @tripleee: Be careful with *... but [sed] leave the contents unchanged if there are no matches"*. When using `-i`, I believe `sed` changes the file time of every file it touches, even though the contents are unchanged. `sed` also converts line endings. I don't use `sed` on Windows in a Git repo because all `CRLF` are changed to `LF`. – jww Oct 25 '17 at 00:19
  • 1
    This command needs a "" after the -i to denote that no backup files will be made after the in-place substitution takes place, at least in macosx. Check the man page for details. If you want a backup, this is where you put the extension of the file to be created. – spinyBabbler Apr 16 '20 at 17:46
  • For spaces is file names you need to do NULL termination on grep and xargs. https://stackoverflow.com/questions/17296525/grep-and-sed-with-spaces-in-filenames – IGRACH Aug 20 '21 at 18:33
  • 1
    I love the funny subtle subliminal command to replace Windows with Linux – Kostrahb Oct 21 '21 at 10:45
42

This works best for me on OS X:

grep -r -l 'searchtext' . | sort | uniq | xargs perl -e "s/matchtext/replacetext/" -pi

Source: http://www.praj.com.au/post/23691181208/grep-replace-text-string-in-files

Marc Juchli
  • 2,240
  • 24
  • 20
  • this is perfect! also works with ag: `ag "search" -l -r . | sort | uniq | xargs perl -e 's/search/replace' -pi` –  Mar 04 '16 at 11:24
  • @sebastiankeller Your Perl command lacks the final slash, which is a syntax error. – tripleee Jun 11 '16 at 08:12
  • 4
    Why is the `sort -u` even part of this? In what circumstances would you expect `grep -rl` to produce the same file name twice? – tripleee Oct 25 '17 at 03:25
6

Usually not with grep, but rather with sed -i 's/string_to_find/another_string/g' or perl -i.bak -pe 's/string_to_find/another_string/g'.

minopret
  • 4,726
  • 21
  • 34
  • I think this is probably the easiest method on here to get the job done. Forcing grep when it is not needed is unnecessary. – Daniel Jun 04 '21 at 01:44
6

Other solutions mix regex syntaxes. To use perl/PCRE patterns for both search and replace, and process only matching files, this works quite well:

grep -rlIZPi 'match1' | xargs -0r perl -pi -e 's/match2/replace/gi;'

match1 and match2 are usually identical but match2 can contain more advanced features that are only relevant to the substitution, e.g. capturing groups.

Translation: grep recursively and list matching filenames, each separated by null to protect any special characters; pipe any filenames to xargs which is expecting a null-separated list; if any filenames are received, pass them to perl to perform the actual substitutions.

For case-sensitive matching, drop the i flag from grep and the i pattern modifier from the s/// expression, but not the i flag from perl itself. To include binary files, remove the I flag from grep.

Walf
  • 8,535
  • 2
  • 44
  • 59
  • Perl itself is quite capable of recursing a file structure. In fact, there is a tool [`find2perl`](https://perldoc.perl.org/5.8.8/find2perl.html) which ships with Perl which does this sort of thing without any `xargs` trickery. – tripleee Oct 25 '17 at 03:24
  • @tripleee `find` doesn't search file contents, and the point is to only process matching files without writing a Perl program. – Walf Oct 25 '17 at 05:30
  • This is a nice solution for Windows, as it avoids the sed-based solutions' issue of converting the line endings. Thanks! – JamHandy Apr 22 '19 at 12:51
5

Be very careful when using find and sed in a git repo! If you don't exclude the binary files you can end up with this error:

error: bad index file sha1 signature 
fatal: index file corrupt

To solve this error you need to revert the sed by replacing your new_string with your old_string. This will revert your replaced strings, so you will be back to the beginning of the problem.

The correct way to search for a string and replace it is to skip find and use grep instead in order to ignore the binary files:

sed -ri -e "s/old_string/new_string/g" $(grep -Elr --binary-files=without-match "old_string" "/files_dir")

Credits for @hobs

tsveti_iko
  • 6,834
  • 3
  • 47
  • 39
1

Here is what I would do:

find /path/to/dir -type f -iname "*filename*" -print0 | xargs -0 sed -i '/searchstring/s/old/new/g'

this will look for all files containing filename in the file's name under the /path/to/dir, than for every file found, search for the line with searchstring and replace old with new.

Though if you want to omit looking for a specific file with a filename string in the file's name, than simply do:

find /path/to/dir -type f -print0 | xargs -0 sed -i '/searchstring/s/old/new/g'

This will do the same thing above, but to all files found under /path/to/dir.

tinnick
  • 316
  • 1
  • 2
  • 15
0

Modern rust tools can be used to do this job. For example to replace in all (non ignored) files "oldstring" and "oldString" with "newstring" and "newString" respectively you can :

Use fd and sd

fd -tf -x sd 'old([Ss]tring)' 'new$1' {}

Use ned

ned -R -p 'old([Ss]tring)' -r 'new$1' .

Use ruplacer

ruplacer --go 'old([Ss]tring)' 'new$1' .

Ignored files

To include ignored (by .gitignore) and hidden files you have to specify it :

  • use -IH for fd,
  • use --ignored --hiddenfor ruplacer.
Kpym
  • 3,743
  • 1
  • 21
  • 18
-1

Another option would be to just use perl with globstar.

Enabling shopt -s globstar in your .bashrc (or wherever) allows the ** glob pattern to match all sub-directories and files recursively.

Thus using perl -pXe 's/SEARCH/REPLACE/g' -i ** will recursively replace SEARCH with REPLACE.

The -X flag tells perl to "disable all warnings" - which means that it won't complain about directories.

The globstar also allows you to do things like sed -i 's/SEARCH/REPLACE/g' **/*.ext if you wanted to replace SEARCH with REPLACE in all child files with the extension .ext.

GuiltyDolphin
  • 768
  • 1
  • 7
  • 10
  • *"Another option would be to just use perl with globstar..."* - Not on Posixy machines, like Solaris. That's why I am specifically looking for `grep` and `sed`. – jww Oct 25 '17 at 00:20