V

A simple colorize for the shell

The following notes provide the essential parts of a simple bash script which allows to colorize text in a terminal. One just states the color and a regular expression for matches which should become highlighted. See further below for a running sample implementation (the "script") and a screenshot.

The essential idea is to use tput to get the colors, and to use sed's substitution command to prepend the color to the regular expression(s) provided on the command line while the text to be highlighted is piped through sed.

For example, we can use tput to store the color code for bright red in the variable $red1). We also at least need some way to reset colors. E.g. tput sgr0 stored in $normal should do.

red=$(tput bold;tput setaf 1)
normal=$(tput sgr0)

Now we can already use sed to colorize e.g. the log of a procmail by running something like

tail -f procmail.log | sed -e "s/\(^From: .*\)/$red\1$normal/"

Here, the regular expression ^From: .* is enclosed in \(…\) so that we can reference it by means of \1. If one wants to highlight only the address part without the From: one could write "s/\(^From:\) \(.*\)/\1 $red\2$normal/" instead.

For a simple newmail like script we can add a beep to every line starting with From: (and make the Subject green in one go)

bell=$(tput bel)
green=$(tput setaf 2)
tail -f procmail.log | sed -e "s/\(^From:\) \(.*\)/\1 $bell$red\2$normal/; s/\(^ Subject:\) \(.*\)/\1 $green\2$normal/"

Finally, we can use some bash code to parse the command line for more color definitions, put them all together and run sed for us. The assumed usage of the script would be

Usage:   mycolorize {color} {regex} ... {color} {regex} ...
Example: tail -f procmail.log | mycolorize red '^From: .*' green '^ Subject: .*'
# compile all rules given at command line to 1 set of rules for SED
while [ "/$1/" != '//' ] ; do
  # reset variables
  color=''; regex=''; beep=''
  # assign parameters from command line to variables and shift the rest
  color=$1 ; regex="$2" ; shift 2
  # add the substitute command to the set of rules for SED
  sedrules="$sedrules;s/\($regex\)/${!color}\1$normal/g"
done
 
# call sed with the compiled sedrules to do the main job
sed -e "$sedrules"

That's it. Still relatively simple, however, there are limitations:

  • One has to be careful when specifying the regular expressions. They need to comply with sed. And one needs proper escaping. In the above example at least slashes (/) need to be escaped with a backslash (\).
  • One cannot highlight parts of a regular expression. Always the complete match is colored. (Though, one can do some tricks with definitions which overlap.)

Sample implementation

A more complete sample implementation is available for download: mycolorize.sh (See below for a more up-to-date implemenation.)

This script adds support for beeps. Slashes in regular expressions don't need to be escaped. And \/ can be used to split regular expressions like in procmailrc to only highlight the part after \/.

Usage syntax:

mycolorize {color} {regex} [bell] ... {color} {regex} [bell]

Usage examples:

cat htmlfile | mycolorize white '<[^>]*>[^<]*</[^>]*>' red '&[^;]\+;'
tail -f maillog | mycolorize white '^From: \/.*' bell green 'Folder: .*'

The following screenshot shows the output of a (customized) procmail log of spam messages colorized with

tail -F log_spam | mycolorize \
  gray '^From: .*$' \
  red 'AS_[0-9]*_[A-Z0-9_]*' beep \
  green '4CHARNAME' \
  white 'autoforwarded for \/.*$' \
  purple '<<<' \
  cyan 'AS_OLD_[A-Z0-9_]*' bell \
  fawn '^ Subject:\/.*'

Screenshot

Sample usage

Beep on overload

You can use the script to beep when your system load reaches some limit with something like

while true ; do cat /proc/loadavg ; sleep 3 ; done | mycolorize red '^[1-9]\+\.[0-9]\+' beep

This would beep when the load is 1 or higher. It's based on the content of /proc/loadavg which on Debian systems is something like 1.00 0.88 0.48 3/107 28593.

The following would beep every 5 seconds at a load >= 2, and it highlights also the average values for the past 5 and 15 minutes.

while true ; do cat /proc/loadavg ; sleep 5 ; done \
| mycolorize red '^[2-9]\+\.[0-9]\+' beep \
    pink '^[^ ]\+ \/[1-9]\+\.[0-9]\+' beep \
    blue '^[^ ]\+ [^ ]\+ \/[1-9]\+\.[0-9]\+' beep

You can see I am no friend of blue ;-)

Colorize logfiles

Here is an example which puts some color on syslog output and Apache error log files:

tail -f /var/log/apache2/error.log /var/log/{sys,auth.,kern.}log \
| mycolorize green '[-A-Za-z0-9/]\+\[[0-9]\+\]' \
    cyan '^... .. ..:..:.. \/[A-Za-z0-9.]\+ ' \
    blue '^\(.... \)\?... .. ..:..:..\( .....\)\? ' \
    blue '^.....-..-.. ..:..:...' \
    red '\(error\|[Ww]arning\|[Nn]ot found\|does not exist\|[Rr]efused\|[Uu]nknown\)' \
    purple '\([Ll]ogin\|accepted\)' \
    cyan '[0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+' \
    pink '[Kk]ernel' beep \
    gray '^==> .* <==$'

This should render dates in blue, host names and IP addresses are cyan, processes and their numbers should show up in green, some keywords are red or purple, and kernel messages beep. tail's file headers are gray.

GNU Sed -r

GNU Sed offers an option --regexp-extended. Quoting from the mangpage:

--regexp-extended
Use extended regular expressions rather than basic regular expressions. Extended regexps are those that egrep accepts; they can be clearer because they usually have less backslashes, but are a GNU extension and hence scripts that use them are not portable. See Extended regexps

I have only recently learned that this option exists. Almost all computers I use regularly come with GNU Sed, and even though I am much used to sed's weird regexes, I am happily switching to the much easier egrep style.

For those who also prefer sed -r and egrep there is a version of mycolorize with sed -r: mycolorize-r.sh (updated 2012-10-12 to fix a bug with (…) before \/)

mycolorize-r.sh

History

2015-04-14  Fixed odd number of arguments causing infinite loop
2012-10-12  Fixed bug with (...) before \/
2012-08-12  Published version using sed --regexp-extended
2008-04-13  Published version using plain sed

Examples

The 2 examples from above in mycolorize-r style (lots of backslashes removed):

while true ; do cat /proc/loadavg ; sleep 5 ; done \
| mycolorize red '^[2-9]+\.[0-9]+' beep \
    pink '^[^ ]+ \/[1-9]+\.[0-9]+' beep \
    blue '^[^ ]+ [^ ]+ \/[1-9]+\.[0-9]+' beep
tail -f /var/log/apache2/error.log /var/log/{sys,auth.,kern.}log \
| mycolorize green '[-A-Za-z0-9/]+\[[0-9]+\]' \
    cyan '^... .. ..:..:.. \/[A-Za-z0-9.]+ ' \
    blue '^(.... )?... .. ..:..:..( .....)? ' \
    blue '^.....-..-.. ..:..:...' \
    red '(error|[Ww]arning|[Nn]ot found|does not exist|[Rr]efused|[Uu]nknown)' \
    purple '([Ll]ogin|accepted)' \
    cyan '[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+' \
    pink '[Kk]ernel' beep \
    gray '^==> .* <==$'

Discussion

1)
Note that color codes depend on your terminfo, see terminfo(5)
 
mycolorize.txt · Last modified: 2015-04-14 13:06 by andreas