How to supercharge string search through a directory hierarchy on a Linux/Unix

Posted on in Categories Command Line Hacks last updated November 1, 2017

Usually, I use the grep command for code searching on my box. Recently, I come across another cool tool called ag. It is an attempt to make something better than ack, which itself is better than grep command. Let us see how to install and use the ag tool on Unix-like operating systems.

The ag command can recursively search for PATTERN in PATH. Like grep or ack, but faster.

Why use ag tool?

  1. It is faster than both grep and ack.
  2. It ignores file patterns found in your ~/.gitignore and ~/.hgignore.
  3. You can add custom ignore patterns to a ~/.ignore file.
  4. Ag uses Pthreads to take advantage of multiple CPU cores and search files in parallel.

How to install

You can install ag as per the version of your Linux distro or Unix variant. Open the Terminal application and type the following command.

Install ag on on a Ubuntu/Debian Linux

Type the following apt command/apt-get command:
$ sudo apt install silversearcher-ag
Sample outputs:

[sudo] password for vivek: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  silversearcher-ag
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 37.3 kB of archives.
After this operation, 120 kB of additional disk space will be used.
Get:1 http://in.archive.ubuntu.com/ubuntu artful/universe amd64 silversearcher-ag amd64 2.0.0-1 [37.3 kB]
Fetched 37.3 kB in 1s (22.3 kB/s)            
Selecting previously unselected package silversearcher-ag.
(Reading database ... 196028 files and directories currently installed.)
Preparing to unpack .../silversearcher-ag_2.0.0-1_amd64.deb ...
Unpacking silversearcher-ag (2.0.0-1) ...
Setting up silversearcher-ag (2.0.0-1) ...
Processing triggers for man-db (2.7.6.1-2) ...

Install ag on on a RHEL/CentOS 7 or Fedora Linux

First enable the EPEL repo and type the following yum command:
$ sudo yum install the_silver_searcher
Sample outputs:

Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: centos.excellmedia.net
 * epel: mirror.nes.co.id
 * extras: mirrors.vinahost.vn
 * updates: mirrors.vonline.vn
Resolving Dependencies
--> Running transaction check
---> Package the_silver_searcher.x86_64 0:2.0.0-1.el7 will be installed
--> Finished Dependency Resolution
 
Dependencies Resolved
 
===============================================================================
 Package                    Arch          Version            Repository   Size
===============================================================================
Installing:
 the_silver_searcher        x86_64        2.0.0-1.el7        epel         48 k
 
Transaction Summary
===============================================================================
Install  1 Package
 
Total download size: 48 k
Installed size: 104 k
Is this ok [y/d/N]: y
Downloading packages:
the_silver_searcher-2.0.0-1.el7.x86_64.rpm                |  48 kB   00:01     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : the_silver_searcher-2.0.0-1.el7.x86_64                      1/1 
  Verifying  : the_silver_searcher-2.0.0-1.el7.x86_64                      1/1 
 
Installed:
  the_silver_searcher.x86_64 0:2.0.0-1.el7                                     
 
Complete!

Fedora Linux user type the following dnf command:
$ sudo dnf install the_silver_searcher

Install ag on an OpenSUSE Linux

Type the following zypper command:
# zypper install the_silver_searcher

Install ag on an OpenBSD

Type the following pkg_add command:
# pkg_add the_silver_searcher

Install ag on a FreeBSD

Type the following pkg command:
# pkg install the_silver_searcher

Install ag on a macOS Unix

Type the following brew command:
$ brew install the_silver_searcher
OR use the port command:
$ port install the_silver_searcher

ag syntax

ag’s basic syntax is
ag [option(s)] pattern files
ag [option(s)] pattern file1 file2
ag [option(s)] pattern dir/

Examples

ag can search any number of files simultaneously. Thus, for example, the following would search the three files foo, bar and foobar for any line that contains the string UNIX:
$ ag UNIX foo bar foobar
Sample outputs:

Fig.01: ag command in action
Fig.01: ag command in action

Each result is displayed beginning on a separate line, and it is preceded by the name of the file, line number, and highlighted the pattern. ag is not limited to searching for just single words or strings. It can also search for sequences of strings or phrases. Just enclose the sequence of strings in quotation marks:
$ ag 'UNIX os' file1 foo bar
You can instructs ag to ignore case:
$ ag -i unix dir
$ ag -i unix .
$ ag -i unix foo bar foobar

foo
1:FreeBSD is a better unix
2:No macOS is Unix
5:UNIX
6:ios is UNIX too :P

bar
2:UNIX no good for me
5:OpenBSD as a Unix for me

foobar
2:I love unix
3:UNIX <3

ag can only report the number of times that the pattern has been matched for each file. Just pass the -c option:
$ ag -i -c unix foo bar foobar
foobar:2
foo:4
bar:2

The search can be restricted to certain types of files. Example:
$ ag --shell 'word' ~/path/
$ ag --shell 'curl' ~/bin/

Above searches for 'curl' in files with suffix .sh, .bash, .csh, .tcsh, .ksh, .zsh and .fish shell scripts only. No need to use the find command and grep it. For a list of supported file types run:
$ ag --list-file-types
$ ag --list-file-types | ag -A 'python'

One can search contents of compressed files too. Just pass the -z option:
$ ag -z 'main()' file.zip xkcd.tar.gz
No need to use the egrep command:
$ ag 'regex' path
$ ag '[Cu]url' ~/bin/

To see all help option, type:
$ man ag
$ ag --help

Sample outputs:

Usage: ag [FILE-TYPE] [OPTIONS] PATTERN [PATH]
 
Output Options:
     --ackmate            Print results in AckMate-parseable format
  -A --after [LINES]      Print lines after match (Default: 2)
  -B --before [LINES]     Print lines before match (Default: 2)
     --[no]break          Print newlines between matches in different files
                          (Enabled by default)
  -c --count              Only print the number of matches in each file.
                          (This often differs from the number of matching lines)
     --[no]color          Print color codes in results (Enabled by default)
     --color-line-number  Color codes for line numbers (Default: 1;33)
     --color-match        Color codes for result match numbers (Default: 30;43)
     --color-path         Color codes for path names (Default: 1;32)
     --column             Print column numbers in results
     --[no]filename       Print file names (Enabled unless searching a single file)
  -H --[no]heading        Print file names before each file's matches
                          (Enabled by default)
  -C --context [LINES]    Print lines before and after matches (Default: 2)
     --[no]group          Same as --[no]break --[no]heading
  -g --filename-pattern PATTERN
                          Print filenames matching PATTERN
  -l --files-with-matches Only print filenames that contain matches
                          (don't print the matching lines)
  -L --files-without-matches
                          Only print filenames that don't contain matches
     --print-all-files    Print headings for all files searched, even those that
                          don't contain matches
     --[no]numbers        Print line numbers. Default is to omit line numbers
                          when searching streams
  -o --only-matching      Prints only the matching part of the lines
     --print-long-lines   Print matches on very long lines (Default: >2k characters)
     --passthrough        When searching a stream, print all lines even if they
                          don't match
     --silent             Suppress all log messages, including errors
     --stats              Print stats (files scanned, time taken, etc.)
     --stats-only         Print stats and nothing else.
                          (Same as --count when searching a single file)
     --vimgrep            Print results like vim's :vimgrep /pattern/g would
                          (it reports every match on the line)
  -0 --null --print0      Separate filenames with null (for 'xargs -0')
 
Search Options:
  -a --all-types          Search all files (doesn't include hidden files
                          or patterns from ignore files)
  -D --debug              Ridiculous debugging (probably not useful)
     --depth NUM          Search up to NUM directories deep (Default: 25)
  -f --follow             Follow symlinks
  -F --fixed-strings      Alias for --literal for compatibility with grep
  -G --file-search-regex  PATTERN Limit search to filenames matching PATTERN
     --hidden             Search hidden files (obeys .*ignore files)
  -i --ignore-case        Match case insensitively
     --ignore PATTERN     Ignore files/directories matching PATTERN
                          (literal file/directory names also allowed)
     --ignore-dir NAME    Alias for --ignore for compatibility with ack.
  -m --max-count NUM      Skip the rest of a file after NUM matches (Default: 10,000)
     --one-device         Don't follow links to other devices.
  -p --path-to-ignore STRING
                          Use .ignore file at STRING
  -Q --literal            Don't parse PATTERN as a regular expression
  -s --case-sensitive     Match case sensitively
  -S --smart-case         Match case insensitively unless PATTERN contains
                          uppercase characters (Enabled by default)
     --search-binary      Search binary files for matches
  -t --all-text           Search all text files (doesn't include hidden files)
  -u --unrestricted       Search all files (ignore .ignore, .gitignore, etc.;
                          searches binary and hidden files as well)
  -U --skip-vcs-ignores   Ignore VCS ignore files
                          (.gitignore, .hgignore; still obey .ignore)
  -v --invert-match
  -w --word-regexp        Only match whole words
  -W --width NUM          Truncate match lines after NUM characters
  -z --search-zip         Search contents of compressed (e.g., gzip) files

Conclusion

The ag comes with saner defaults than grep and easy to type. However, I will not install it on my server. It might be a good choice for your laptop or desktop computer dedicated to software development. ag was originally created by Geoff Greer. More information (and the latest release) can be found at https://geoff.greer.fm/ag.

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin and a trainer for the Linux operating system/Unix shell scripting. He has worked with global clients and in various industries, including IT, education, defense and space research, and the nonprofit sector. Follow him on Twitter, Facebook, Google+.

4 comment

  1. I went looking in the Archlinux repos for the Silver Searcher (aka ag) ;-). Apparently there are some very interesting alternatives. rg (RipGrep) is one of the most interesting and very fast, maybe even faster than ag.
    There are also a couple of forks of Silver Searcher, namely The Platinum Searcher and Aswild’s Silver Searcher fork. RipGrep does look good, though. I’m going to install it when I get home.

Leave a Comment