How to supercharge string search through a directory hierarchy on a Linux/Unix

Usually, I use the grep command for code searching on my box. Recently, I come across another cool tool called ag. It is an attempt to make something better than ack, which itself is better than grep command. Let us see how to install and use the ag tool on Unix-like operating systems.

The ag command can recursively search for PATTERN in PATH. Like grep or ack, but faster.

Why use ag tool?

It is faster than both grep and ack.
It ignores file patterns found in your ~/.gitignore and ~/.hgignore.
You can add custom ignore patterns to a ~/.ignore file.
Ag uses Pthreads to take advantage of multiple CPU cores and search files in parallel.

How to install

You can install ag as per the version of your Linux distro or Unix variant. Open the Terminal application and type the following command.

Install ag on on a Ubuntu/Debian Linux

Type the following apt command/apt-get command:
$ sudo apt install silversearcher-ag
Sample outputs:

[sudo] password for vivek: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  silversearcher-ag
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 37.3 kB of archives.
After this operation, 120 kB of additional disk space will be used.
Get:1 http://in.archive.ubuntu.com/ubuntu artful/universe amd64 silversearcher-ag amd64 2.0.0-1 [37.3 kB]
Fetched 37.3 kB in 1s (22.3 kB/s)            
Selecting previously unselected package silversearcher-ag.
(Reading database ... 196028 files and directories currently installed.)
Preparing to unpack .../silversearcher-ag_2.0.0-1_amd64.deb ...
Unpacking silversearcher-ag (2.0.0-1) ...
Setting up silversearcher-ag (2.0.0-1) ...
Processing triggers for man-db (2.7.6.1-2) ...

Install ag on on a RHEL/CentOS 7 or Fedora Linux

First enable the EPEL repo and type the following yum command:
$ sudo yum install the_silver_searcher
Sample outputs:

Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: centos.excellmedia.net
 * epel: mirror.nes.co.id
 * extras: mirrors.vinahost.vn
 * updates: mirrors.vonline.vn
Resolving Dependencies
--> Running transaction check
---> Package the_silver_searcher.x86_64 0:2.0.0-1.el7 will be installed
--> Finished Dependency Resolution
 
Dependencies Resolved
 
===============================================================================
 Package                    Arch          Version            Repository   Size
===============================================================================
Installing:
 the_silver_searcher        x86_64        2.0.0-1.el7        epel         48 k
 
Transaction Summary
===============================================================================
Install  1 Package
 
Total download size: 48 k
Installed size: 104 k
Is this ok [y/d/N]: y
Downloading packages:
the_silver_searcher-2.0.0-1.el7.x86_64.rpm                |  48 kB   00:01     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : the_silver_searcher-2.0.0-1.el7.x86_64                      1/1 
  Verifying  : the_silver_searcher-2.0.0-1.el7.x86_64                      1/1 
 
Installed:
  the_silver_searcher.x86_64 0:2.0.0-1.el7                                     
 
Complete!

Fedora Linux user type the following dnf command:
$ sudo dnf install the_silver_searcher

Install ag on an OpenSUSE Linux

Type the following zypper command:
# zypper install the_silver_searcher

Install ag on an OpenBSD

Type the following pkg_add command:
# pkg_add the_silver_searcher

Install ag on a FreeBSD

Type the following pkg command:
# pkg install the_silver_searcher

Install ag on a macOS Unix

Type the following brew command:
$ brew install the_silver_searcher
OR use the port command:
$ port install the_silver_searcher

ag syntax

ag’s basic syntax is
ag [option(s)] pattern files ag [option(s)] pattern file1 file2 ag [option(s)] pattern dir/

Examples

ag can search any number of files simultaneously. Thus, for example, the following would search the three files foo, bar and foobar for any line that contains the string UNIX:
$ ag UNIX foo bar foobar
Sample outputs:

Each result is displayed beginning on a separate line, and it is preceded by the name of the file, line number, and highlighted the pattern. ag is not limited to searching for just single words or strings. It can also search for sequences of strings or phrases. Just enclose the sequence of strings in quotation marks:
$ ag 'UNIX os' file1 foo bar
You can instructs ag to ignore case:

$ ag -i unix dir
 $ ag -i unix .
 $ ag -i unix foo bar foobar

foo
1:FreeBSD is a better unix
2:No macOS is Unix
5:UNIX
6:ios is UNIX too :P

bar
2:UNIX no good for me
5:OpenBSD as a Unix for me

foobar
2:I love unix
3:UNIX <3

ag can only report the number of times that the pattern has been matched for each file. Just pass the -c option:
$ ag -i -c unix foo bar foobar foobar:2 foo:4 bar:2
The search can be restricted to certain types of files. Example:
$ ag --shell 'word' ~/path/ $ ag --shell 'curl' ~/bin/
Above searches for 'curl' in files with suffix .sh, .bash, .csh, .tcsh, .ksh, .zsh and .fish shell scripts only. No need to use the find command and grep it. For a list of supported file types run:
$ ag --list-file-types $ ag --list-file-types | ag -A 'python'
One can search contents of compressed files too. Just pass the -z option:
$ ag -z 'main()' file.zip xkcd.tar.gz
No need to use the egrep command:
$ ag 'regex' path $ ag '[Cu]url' ~/bin/
To see all help option, type:
$ man ag $ ag --help
Sample outputs:

Usage: ag [FILE-TYPE] [OPTIONS] PATTERN [PATH]
 
Output Options:
     --ackmate            Print results in AckMate-parseable format
  -A --after [LINES]      Print lines after match (Default: 2)
  -B --before [LINES]     Print lines before match (Default: 2)
     --[no]break          Print newlines between matches in different files
                          (Enabled by default)
  -c --count              Only print the number of matches in each file.
                          (This often differs from the number of matching lines)
     --[no]color          Print color codes in results (Enabled by default)
     --color-line-number  Color codes for line numbers (Default: 1;33)
     --color-match        Color codes for result match numbers (Default: 30;43)
     --color-path         Color codes for path names (Default: 1;32)
     --column             Print column numbers in results
     --[no]filename       Print file names (Enabled unless searching a single file)
  -H --[no]heading        Print file names before each file's matches
                          (Enabled by default)
  -C --context [LINES]    Print lines before and after matches (Default: 2)
     --[no]group          Same as --[no]break --[no]heading
  -g --filename-pattern PATTERN
                          Print filenames matching PATTERN
  -l --files-with-matches Only print filenames that contain matches
                          (don't print the matching lines)
  -L --files-without-matches
                          Only print filenames that don't contain matches
     --print-all-files    Print headings for all files searched, even those that
                          don't contain matches
     --[no]numbers        Print line numbers. Default is to omit line numbers
                          when searching streams
  -o --only-matching      Prints only the matching part of the lines
     --print-long-lines   Print matches on very long lines (Default: >2k characters)
     --passthrough        When searching a stream, print all lines even if they
                          don't match
     --silent             Suppress all log messages, including errors
     --stats              Print stats (files scanned, time taken, etc.)
     --stats-only         Print stats and nothing else.
                          (Same as --count when searching a single file)
     --vimgrep            Print results like vim's :vimgrep /pattern/g would
                          (it reports every match on the line)
  -0 --null --print0      Separate filenames with null (for 'xargs -0')
 
Search Options:
  -a --all-types          Search all files (doesn't include hidden files
                          or patterns from ignore files)
  -D --debug              Ridiculous debugging (probably not useful)
     --depth NUM          Search up to NUM directories deep (Default: 25)
  -f --follow             Follow symlinks
  -F --fixed-strings      Alias for --literal for compatibility with grep
  -G --file-search-regex  PATTERN Limit search to filenames matching PATTERN
     --hidden             Search hidden files (obeys .*ignore files)
  -i --ignore-case        Match case insensitively
     --ignore PATTERN     Ignore files/directories matching PATTERN
                          (literal file/directory names also allowed)
     --ignore-dir NAME    Alias for --ignore for compatibility with ack.
  -m --max-count NUM      Skip the rest of a file after NUM matches (Default: 10,000)
     --one-device         Don't follow links to other devices.
  -p --path-to-ignore STRING
                          Use .ignore file at STRING
  -Q --literal            Don't parse PATTERN as a regular expression
  -s --case-sensitive     Match case sensitively
  -S --smart-case         Match case insensitively unless PATTERN contains
                          uppercase characters (Enabled by default)
     --search-binary      Search binary files for matches
  -t --all-text           Search all text files (doesn't include hidden files)
  -u --unrestricted       Search all files (ignore .ignore, .gitignore, etc.;
                          searches binary and hidden files as well)
  -U --skip-vcs-ignores   Ignore VCS ignore files
                          (.gitignore, .hgignore; still obey .ignore)
  -v --invert-match
  -w --word-regexp        Only match whole words
  -W --width NUM          Truncate match lines after NUM characters
  -z --search-zip         Search contents of compressed (e.g., gzip) files

Usage: ag [FILE-TYPE] [OPTIONS] PATTERN [PATH] Output Options: --ackmate Print results in AckMate-parseable format -A --after [LINES] Print lines after match (Default: 2) -B --before [LINES] Print lines before match (Default: 2) --[no]break Print newlines between matches in different files (Enabled by default) -c --count Only print the number of matches in each file. (This often differs from the number of matching lines) --[no]color Print color codes in results (Enabled by default) --color-line-number Color codes for line numbers (Default: 1;33) --color-match Color codes for result match numbers (Default: 30;43) --color-path Color codes for path names (Default: 1;32) --column Print column numbers in results --[no]filename Print file names (Enabled unless searching a single file) -H --[no]heading Print file names before each file's matches (Enabled by default) -C --context [LINES] Print lines before and after matches (Default: 2) --[no]group Same as --[no]break --[no]heading -g --filename-pattern PATTERN Print filenames matching PATTERN -l --files-with-matches Only print filenames that contain matches (don't print the matching lines) -L --files-without-matches Only print filenames that don't contain matches --print-all-files Print headings for all files searched, even those that don't contain matches --[no]numbers Print line numbers. Default is to omit line numbers when searching streams -o --only-matching Prints only the matching part of the lines --print-long-lines Print matches on very long lines (Default: >2k characters) --passthrough When searching a stream, print all lines even if they don't match --silent Suppress all log messages, including errors --stats Print stats (files scanned, time taken, etc.) --stats-only Print stats and nothing else. (Same as --count when searching a single file) --vimgrep Print results like vim's :vimgrep /pattern/g would (it reports every match on the line) -0 --null --print0 Separate filenames with null (for 'xargs -0') Search Options: -a --all-types Search all files (doesn't include hidden files or patterns from ignore files) -D --debug Ridiculous debugging (probably not useful) --depth NUM Search up to NUM directories deep (Default: 25) -f --follow Follow symlinks -F --fixed-strings Alias for --literal for compatibility with grep -G --file-search-regex PATTERN Limit search to filenames matching PATTERN --hidden Search hidden files (obeys .*ignore files) -i --ignore-case Match case insensitively --ignore PATTERN Ignore files/directories matching PATTERN (literal file/directory names also allowed) --ignore-dir NAME Alias for --ignore for compatibility with ack. -m --max-count NUM Skip the rest of a file after NUM matches (Default: 10,000) --one-device Don't follow links to other devices. -p --path-to-ignore STRING Use .ignore file at STRING -Q --literal Don't parse PATTERN as a regular expression -s --case-sensitive Match case sensitively -S --smart-case Match case insensitively unless PATTERN contains uppercase characters (Enabled by default) --search-binary Search binary files for matches -t --all-text Search all text files (doesn't include hidden files) -u --unrestricted Search all files (ignore .ignore, .gitignore, etc.; searches binary and hidden files as well) -U --skip-vcs-ignores Ignore VCS ignore files (.gitignore, .hgignore; still obey .ignore) -v --invert-match -w --word-regexp Only match whole words -W --width NUM Truncate match lines after NUM characters -z --search-zip Search contents of compressed (e.g., gzip) files

Conclusion

The ag comes with saner defaults than grep and easy to type. However, I will not install it on my server. It might be a good choice for your laptop or desktop computer dedicated to software development. ag was originally created by Geoff Greer. More information (and the latest release) can be found at https://geoff.greer.fm/ag.

UP NEXT

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin, DevOps engineer, and a trainer for the Linux operating system/Unix shell scripting. Get the latest tutorials on SysAdmin, Linux/Unix and open source topics via RSS/XML feed or weekly email newsletter.

4 comment

Jared K Still says:
November 2, 2017 at 2:29 am
Something seems to be missing – the speed comparison tests.
SC says:
November 1, 2017 at 3:16 pm
Thank you. I’ve been looking for a tool like this, plus it’s much faster than ack.
macxcool says:
November 1, 2017 at 1:41 pm
I went looking in the Archlinux repos for the Silver Searcher (aka ag) ;-). Apparently there are some very interesting alternatives. rg (RipGrep) is one of the most interesting and very fast, maybe even faster than ag.
There are also a couple of forks of Silver Searcher, namely The Platinum Searcher and Aswild’s Silver Searcher fork. RipGrep does look good, though. I’m going to install it when I get home.
1. stfl says:
  November 2, 2017 at 1:25 pm
  +1 for ripgrep (rg)