193

Is there a way to exclude certain paths/directories/files when searching a git repository using git grep? Something similar to the --exclude option in the normal grep command?

I need to use git grep because using grep directly runs too slowly on large git repositories.

pants
  • 192
  • 13
Yogeshwer Sharma
  • 3,647
  • 5
  • 25
  • 27

5 Answers5

290

In git 1.9.0 the "magic word" exclude was added to pathspecs. So if you want to search for foobar in every file except for those matching *.java you can do:

git grep foobar -- ':(exclude)*.java'

Or using the ! "short form" for exclude:

git grep foobar -- ':!*.java'

Note that in git versions up to v2.12, when using an exclude pathspec, you must have at least one "inclusive" pathspec. In the above examples you'd want to add ./* (recursively include everything under the current directory) somewhere after the -- as well. In git v2.13 this restriction was lifted and git grep foobar -- ':!*.java' works without the ./*.

There's a good reference for all the "magic words" allowed in a pathspec at git-scm.com (or just git help glossary).

onlynone
  • 7,602
  • 3
  • 31
  • 50
  • 9
    `git grep clock.gettime -- './*' ':!arch/**' ':!drivers/**'` to exclude multiple entire directories. I don't think it prevents recursion though. – Ciro Santilli OurBigBook.com Apr 23 '16 at 09:55
  • 3
    For frequent use, you can make a git alias with the exclusions: `git config alias.mygrep '!git grep "$@" -- "${GIT_PREFIX}/*" ":!*.java*" #'`. Then just `git mygrep foobar`. (Using alias [shell # trick](http://stackoverflow.com/a/39523506/647002) and [current dir](http://stackoverflow.com/a/22039008/647002).) – medmunds Mar 20 '17 at 18:18
  • the problem i cant solve with this solution is that the reported paths of the files are relative to the WC root. So, if i'm in a sub dir of the WC, i cant just use the path of found file(s) as-is (e.g. for less) but have to junc common paths. Is there a solution to this (w/o having to emply sed myself) ? [git bash on win7] – elonderin Sep 26 '17 at 17:15
  • 2
    @elonderin this solution has nothing to do with how the matched files are reported. But I just tried a `git grep` and `git ls-files` from subdirectories and both report filenames relative to the current directory (even when you use the `':(top)'` include pathspec). Both commands have the `--full-name` option to report names relative to the root, but that's off by default. – onlynone Sep 28 '17 at 14:22
  • Or `git grep foobar -- './*' ':^*.java'`, since `:!` and `:^` are synonyms of `:(exclude)` – hIpPy Jul 12 '18 at 06:43
  • 1
    I don't use git aliases so I made a bash function, but possibly a git alias is better https://gist.github.com/cmdcolin/04e2378b60f4457a41904c659368066f – Colin D Apr 16 '19 at 15:56
  • @YogeshwerSharma Do you mind accepting this answer instead of the outdated one? – usr1234567 Mar 13 '20 at 10:10
  • 1
    I'm using git 2.19 and a simpler syntax than the one posted above that works is just: git grep foobar ':!*.java' – adriandz Apr 30 '20 at 11:08
  • could `':(exclude)` applied for specific files? – alper Jun 18 '21 at 12:27
66

Update: For git >= 1.9 there is native support for exclude patterns, see onlyone's answer.

This may seem backwards, but you can pass a list of files not matching your exclude pattern to git grep like this:

git grep <pattern> -- `git ls-files | grep -v <exclude-pattern>`

grep -v returns every path not matching <exclude-pattern>. Note that git ls-files also takes a --exclude parameter, but that is only applied to untracked files.

Community
  • 1
  • 1
kynan
  • 13,235
  • 6
  • 79
  • 81
  • Thanks for this! Git grep is so much faster than ack & co but not being able to exclude arbitrary paths was a bit too inconvenient so to speak :) – Tomasz Zieliński Jun 05 '13 at 13:37
  • 2
    Unfortunately my repo has a lot of files. When I try @kynan's approach I get: "-bash: /usr/bin/git: Argument list too long" – Benissimo May 19 '14 at 09:23
  • 2
    This should solve both the "Argument list too long" problem of Benissimo and my problem with filename caracters interpreted by bash (like []) or filenames containing spaces in the repository: git ls-files | grep -v | xargs -d '\n' git grep -- – Scout Oct 23 '14 at 14:48
  • 2
    Check onlynone's answer, it's possibly to do this entirely within (modern versions of) git now. – David Nov 16 '15 at 15:12
  • 1
    Why the downvotes? This answer still applies to git versions prior to 1.9. I've added a note referring to onlyone's answer. – kynan Dec 18 '15 at 00:03
19

It's not possible, but has been discussed recently. Proposed workaround in link:

You can put *.dll to .gitignore file then git grep --exclude-standard.

EDIT see onlynone's answer, since git 1.9.0 it's possible.

Jonathon Reinhart
  • 132,704
  • 33
  • 254
  • 328
CharlesB
  • 86,532
  • 28
  • 194
  • 218
  • 2
    This used to be true but no longer, it is now possible in git. See what should be the real answer below: http://stackoverflow.com/a/30084612/1391445 – user1391445 Dec 09 '15 at 23:38
19

You can mark files or directories as binary by creating an attributes file in your repository, e.g.

$ cat .git/info/attributes 
directory/to/ignore/*.* binary
directory/to/ignore/*/*.* binary
another_directory/to/also/ignore/*.* binary

Matches in binary files are listed without the including line, e.g.

$ git grep "bar"
Binary file directory/to/ignore/filename matches
other_directory/other_filename:      foo << bar - bazz[:whatnot]
coberlin
  • 508
  • 5
  • 7
2

With the example by @kynan as base I made this script and put it in my path (~/bin/) as gg. It does use git grep but avoids some specified filetypes.

In our repo its a lot of images so I have excluded the imagefiles, and this takes the serchtime down to 1/3 if I search the whole repo. But the script could easily be modified to exclude other filestypes or geleralpatterns.

#!/bin/bash                                                                    
#                                                                              
# Wrapper of git-grep that excludes certain filetypes.                         
# NOTE: The filetypes to exclude is hardcoded for my specific needs.           
#                                                                              
# The basic setup of this script is from here:                                 
#   https://stackoverflow.com/a/14226610/42580                                  
# But there is issues with giving extra path information to the script         
# therefor I crafted the while-thing that moves path-parts to the other side   
# of the '--'.                                                                 

# Declare the filetypes to ignore here                                         
EXCLUDES="png xcf jpg jpeg pdf ps"                                             

# Rebuild the list of fileendings to a good regexp                             
EXCLUDES=`echo $EXCLUDES | sed -e 's/ /\\\|/g' -e 's/.*/\\\.\\\(\0\\\)/'`      

# Store the stuff that is moved from the arguments.                            
moved=                                                                         

# If git-grep returns this "fatal..." then move the last element of the        
# arg-list to the list of files to search.                                     
err="fatal: bad flag '--' used after filename"                                 
while [ "$err" = "fatal: bad flag '--' used after filename" ]; do              
    {                                                                          
        err=$(git grep "$@" -- `git ls-files $moved | grep -iv "$EXCLUDES"` \  
            2>&1 1>&3-)                                                        
    } 3>&1                                                                     

    # The rest of the code in this loop is here to move the last argument in   
    # the arglist to a separate list $moved. I had issues with whitespace in   
    # the search-string, so this is loosely based on:                          
    #   http://www.linuxjournal.com/content/bash-preserving-whitespace-using-set-and-eval
    x=1                                                                        
    items=                                                                     
    for i in "$@"; do                                                          
        if [ $x -lt $# ]; then                                                 
            items="$items \"$i\""                                              
        else                                                                   
            moved="$i $moved"                                                  
        fi                                                                     
        x=$(($x+1))                                                            
    done                                                                       
    eval set -- $items                                                         
done                                                                           
# Show the error if there was any                                              
echo $err

Note 1

According to this it should be possible to name the thing git-gg and be able to call it as a regular git command like:

$ git gg searchstring

But I can not get this working. I created the script in my ~/bin/ and made a the git-gg symlink in /usr/lib/git-core/.

Note 2

The command can not be made into an regular sh git-alias since it will then be invoked at the root of the repo. And that is not what I want!

Community
  • 1
  • 1
UlfR
  • 4,175
  • 29
  • 45