remark Command: Highlight Any Text or Command Output Using Regular Expressions

by on August 31, 2012 · 8 comments· LAST UPDATED September 9, 2012

in Command Line Hacks

The remark command is a little know tool that can be used to color syslog file as well as output of of programs such as ping, traceroute, and much more.

You can highlight text using regular expressions and specific rule formats. The hierarchical rules-format is simple to understand, yet very powerful. The remark will read each line from stdin and each line is highlighted according to rules defined in FILE, and is then printed to stdout (screen).

Install remark

Type the following command under Debian / Ubuntu Linux to install regex-markup:
$ cd /tmp
$ wget http://savannah.nongnu.org/download/regex-markup/regex-markup_0.10.0-1_amd64.deb
$ sudo dpkg -i regex-markup_0.10.0-1_amd64.deb

If you are using RHEL / CentOS / Fedora Linux 64bit, type the following command to install regex-markup:
$ cd /tmp
$ wget http://savannah.nongnu.org/download/regex-markup/regex-markup-0.10.0-1.x86_64.rpm
$ rpm -ivh regex-markup-0.10.0-1.x86_64.rpm

Or you can compile and install it using source code:
$ cd /tmp
$ wget http://savannah.nongnu.org/download/regex-markup/regex-markup-0.10.0.tar.gz
$ tar -xvf regex-markup-0.10.0.tar.gz
$ cd regex-markup-0.10.
$ ./configure
$ make
$ sudo make install

Syntax

The syntax is:

 
command1 | remark /path/to/config
command2 arg1 arg2 | remark /path/to/config
 

Examples

Highlight ping output using the ping rules file:
$ ping -c 4 nixcraft.com | remark /usr/share/regex-markup/ping
Sample outputs:

Fig.01: ping command in action

Fig.01: ping command in action


You can create a bash shell function and add it to your ~/.bashrc file:

 
ping() { /bin/ping $@ | remark /usr/share/regex-markup/ping; }
 

More about /usr/share/regex-markup/ping

The /usr/share/regex-markup/ping is a rule file for ping command. It consists of style and macro definitions, and match statements. Order of match statements is important, since they are executed from top down. Styles and macros need to be defined before they are used. The syntax of rule files is similar to that of programming languages such as C and Java in that indentation and use of whitespace doesn't matter.:
$ cat /usr/share/regex-markup/ping
Sample outputs:

 
# Rules to highlight the output of ping(8)
 
include "common"
 
# Special: Color all lines that don't match any of the rules below red
/.*/ red
 
 
/^PING ([-.a-zA-Z0-9]+) \(([-.a-zA-Z0-9]+)\) ([0-9]+)\(([0-9]+)\) bytes of data\.$/ {
  default
  1 blue
  2 green
  3,4 red
  break		# this is merely to skip the matches below
}
 
/^PING ([-.a-zA-Z0-9]+) \(([-.a-zA-Z0-9]+)\): ([0-9]+) data bytes$/ {
  default
  1 blue
  2 green
  3 red
  break		# this is merely to skip the matches below
}
 
/^([0-9]+) bytes from ([-.a-zA-Z0-9]+) \(([-.a-zA-Z0-9]+)\): icmp_seq=([0-9]+) ttl=([0-9]+) time=(.*)$/ {
  default
  1,4,5,6 red
  2 blue
  3 green
  break
}
/^([0-9]+) bytes from ([-.a-zA-Z0-9]+): icmp_seq=([0-9]+) ttl=([0-9]+) time=(.*)$/ {
  default
  1,3,4,5 red
  2 green
  break
}
 
/^--- ([-.a-zA-Z0-9]+) ping statistics ---$/ {
  default
  1 blue
  break
}
 
/^([0-9]+) packets transmitted, ([0-9]+) packets received, ([0-9]+)% packet loss$/ {
  default
  1,2,3 red
  break
}
/^([0-9]+) packets transmitted, ([0-9]+) received, ([0-9]+)% packet loss, time ([0-9]+ms)$/ {
  default
  1,2,3,4 red
  break
}
 
/^round-trip min\/avg\/max = ([.0-9]+)\/([.0-9]+)\/(.*)$/ {
  default
  1,2,3 red
  break
}
/^rtt min\/avg\/max\/mdev = ([.0-9]+)\/([.0-9]+)\/([.0-9]+)\/(.*)$/ {
  default
  1,2,3,4 red
  break
}
 

The default style is defined in /usr/share/regex-markup/common file:
$ cat /usr/share/regex-markup/common
Sample outputs:

 
# Some styles for ANSI character attributes
 
style plain     { pre "\E[0m"  post "\E[0m"  }
 
style bold      { pre "\E[1m"  post "\E[22m" }
style underline { pre "\E[4m"  post "\E[24m" }
style blink     { pre "\E[5m"  post "\E[25m" }
style inverse   { pre "\E[7m"  post "\E[27m" }
 
style black     { pre "\E[30m" post "\E[39m" }
style red       { pre "\E[31m" post "\E[39m" }
style green     { pre "\E[32m" post "\E[39m" }
style yellow    { pre "\E[33m" post "\E[39m" }
style blue      { pre "\E[34m" post "\E[39m" }
style magenta   { pre "\E[35m" post "\E[39m" }
style cyan      { pre "\E[36m" post "\E[39m" }
style white     { pre "\E[37m" post "\E[39m" }
 
style blackbg   { pre "\E[40m" post "\E[49m" }
style redbg     { pre "\E[41m" post "\E[49m" }
style greenbg   { pre "\E[42m" post "\E[49m" }
style yellowbg  { pre "\E[43m" post "\E[49m" }
style bluebg    { pre "\E[44m" post "\E[49m" }
style magentabg { pre "\E[45m" post "\E[49m" }
style cyanbg    { pre "\E[46m" post "\E[49m" }
style whitebg   { pre "\E[47m" post "\E[49m" }
 
style standard   { pre "\E[38m" post "\E[39m" }         # XXX
style standardbg { pre "\E[48m" post "\E[49m" }         # XXX
style default    { pre "\E[39m" post "\E[39m" }         # XXX
style defaultbg  { pre "\E[49m" post "\E[49m" }         # XXX
 

Type the following command:
$ traceroute www.cyberciti.biz | remark /usr/local/share/regex-markup/traceroute
Sample outputs:

Fig.02: traceroute in action

Fig.02: traceroute in action

Highlight Linux server syslogs entries

Type the following command:

 
grep something /var/log/syslog | remark /usr/share/regex-markup/syslog
tail -f  /var/log/syslog | remark /usr/share/regex-markup/syslog
 

Highlight diff command output

Type the following command:

 
diff file1 file2 | remark /usr/share/regex-markup/diff
 

Highlight make command output

Type the following command:

 
cd /path/to/build
make | remark /usr/share/regex-markup/make
 

How do I create my own regex-markup?

You need to read remark command man page for rule format and style:
$ man remark

References:
  1. regex-markup
TwitterFacebookGoogle+PDF versionFound an error/typo on this page? Help us!

  • Sahand Mozaffari

    So one has to wait for the output to complete before it is printed out?

  • Chris F.A. Johnson

    For 32-bit, RPM-based distros, use:

    http://download.savannah.gnu.org/releases/regex-markup/regex-markup-0.10.0-1.i386.rpm
    
  • Umair

    It slapped me with this Error

    dpkg: error processing regex-markup_0.10.0-1_amd64.deb (–install):
    package architecture (amd64) does not match system (i386)
    Errors were encountered while processing:
    regex-markup_0.10.0-1_amd64.deb

    Im using ubuntu.

  • mb

    Is there a way to use remark in combination with alias?
    So you don’t have to write the full command but just use aliases.

  • http://www.cyberciti.biz/tips/about-us Vivek Gite

    You need to install 32bit deb file under Debian/Ubuntu for 32 bit systems:

    wget http://savannah.nongnu.org/download/regex-markup/regex-markup_0.10.0-1_i386.deb
    sudo dpkg -i regex-markup_0.10.0-1_i386.deb
  • http://www.cyberciti.biz/tips/about-us Vivek Gite

    There is no mechanism for using arguments in the replacement text. If arguments are needed, a shell function should be used see below:

    http://www.cyberciti.biz/faq/linux-unix-pass-argument-to-alias-command/
    

    For the lazy:

    ping() { /bin/ping $@ | remark /usr/share/regex-markup/ping; }
    
  • setevoy

    Trying build in FreeBSD from source code and Stop with:

    hmap.c: In function ‘hmap_new':
    hmap.c:170: error: ‘HMap’ has no member named ‘compare’
    hmap.c:170: error: ‘comparison_fn_t’ undeclared (first use in this function)
    hmap.c:170: error: expected ‘;’ before ‘strcmp’
    hmap.c: In function ‘hmap_get_entry':
    hmap.c:198: error: ‘HMap’ has no member named ‘compare’
    hmap.c: In function ‘hmap_put':
    hmap.c:235: error: ‘HMap’ has no member named ‘compare’
    hmap.c: In function ‘hmap_remove':
    hmap.c:281: error: ‘HMap’ has no member named ‘compare’
    *** Error code 1

    Stop in /tmp/regex-markup-0.10.0/src/common.
    *** Error code 1

    Stop in /tmp/regex-markup-0.10.0/src.
    *** Error code 1

    Stop in /tmp/regex-markup-0.10.0.
    *** Error code 1

    Stop in /tmp/regex-markup-0.10.0.

    Печалька :-(

  • pulsar

    an alternative might be grc. It should be included in pretty much every distribution by now (its old, really old). Does not need piping either:

    pulsar@atomic:~$ grc ping google.com

    or

    pulsar@atomic:~$ sudo grc tail -f /var/log/syslog

    check out

    /etc/grc.conf

    for command -> highlighter mapping and

    /usr/share/grc/conf.${highlighter} 

    to customize your fancy regular expressions.

    here is an example I threw together to highlight log4j: http://pastie.org/4650086 – save that in /usr/share/grc/conf.log4j and make your life a bit easier by using an alias:

    alias log4jtail="grc -c conf.log4j tail -f"

    now tail any log4j logs with :

    atomic:/var/subsonic# log4jtail subsonic.log

    swatch (sw) is also worth mentioning, it is not a generic highlighter and meant just to monitor system logfiles.

Previous post:

Next post: