Git grep multiple repos at once

Updated in 2020 to work with Python 3.

My company uses multiple git repos that all depend on one another. Often I find myself trying to grep through a few of them at the same time.

Regular grep works, but it takes a long time and displays a lot of noise. git grep only searches through files that are NOT listed in .gitignore file, but only works at per-repo level.

Few months ago I wrote a script to walk through my repos and generate a list of things I worked on that week.

I’ve modified that script to do git grep for all git repos.

i.e. git-grep-all "some string" will produce:

repo-1
/some/file: some string
/some/other/file: some string 123

repo-2
/yet/another/file: again some string

Few notes to get it working

1. You need to update the BASE variable. All my git repo are nested under one folder, and it looks as follows:
   • company-name
     • repo-1
     • repo-2
     • repo-3
2. I’ve saved this file to ~/bin/git-grep-all, made it executable, and added ~/bin to my PATH.
3. Now I can just run git-grep-all "my search string" to search through all repos that I have cloned.

#!/usr/bin/env python

import subprocess
import os
import sys

BASE = 'path/to/multiple/repos/to/search/'  # <----------- Fix base path

if len(sys.argv) < 2:
    print('used as: grep-all "some string"')
    exit(1)

search = sys.argv[1]

repos = os.listdir(BASE)
repos = [os.path.join(BASE, repo) for repo in repos]


results = ""
for repo in repos:
    command = "cd {} && git grep --color='always' '{}'".format(repo, search)
    results = subprocess.run(
        [command], shell=True, capture_output=True)
    results = results.stdout.decode("utf-8", 'ignore')
    if(
        len(results) > 0 and
        "Not a directory" not in results and
        "Not a git repository" not in results
    ):
        print("\n" + os.path.relpath(repo, BASE))
        print(results)

Code language: PHP (php)

Note: This method is not perfect. It will only grep repos at the state that you last left them in and it will not pull new changes from origin. Still, I find that it works really well for 80% of my use cases.

Hope it helps.