Awk Words: Using AWK in the real world


AWK is a script-based programming language written by Aho, Weinberger, and Kernighan. It makes it fairly easy to write scripts to allow more sophisticated searching than provided directly by GREP.

Scripts may be as short as one line (in which case they may be given on the command line), or as long as several pages. There are numerous AWK scripts which have been written for assisting in file searching and processing. Many of my documents are formatted using AWK scripts and a batch command files.

The man page for AWK contains more information about the actual language, and there are several books which describe the language and how to use it.

Because my primary development environment is embedded programming, cross-developed from a PC, the descriptions that follow are based on using a PC version of AWK on a DOS machine. Most of the descriptions should be easily portable to other platforms, though the command lines may look slightly different.

Using AWK at the command line

In general, command line AWK will take a form like:

AWK "expression" fileList


AWK "expression { action }" fileList

The 'expression' is usually a regular expression (generally surrounded by '/' characters).

If no 'action' is specified, then all lines which match the regular expression will simply be displayed. If an action is specified, it will be performed for all lines which match the 'expression'.

Note that, due to the way DOS parses command lines, you must use the sequence backslash-quote (\") to represent any quotes you want in the command line AWK program.

Using AWK to find multiple strings

To find all lines containing "util.h" OR "std.h" and display the entire line, prefixed by the name of the file, in all .C files:

AWK "/util.h/ || /std.h/ { print FILENAME \": \" $0 }" *.c

expression: /util.h/ || /std.h/ contains "util.h" OR "std.h"

action: print FILENAME ": " $0 print the name of the file, a colon, then the line

Adding a blank line after lines in a list

To add a blank line after all lines containing "util.h":

AWK "{ print $0; if ( $0 ~ /util.h/) print \"\" }" TMP.TMP

expression: none do this for all lines

action: print $0; if ( $0 ~ /util.h/ ) print "" print the line, then if the line contains "util.h" print a blank line

Listing all lines except those containing "Frame overlap"

To list all the lines of a file (TMP.LOG), except those containing the string "Frame overlap", you can use the command:

AWK "!/Frame overlap/" TMP.LOG

expression: !/Frame overlap/ lines not containing "Frame overlap"

action: none, just output the lines

or, using GREP:

grep -v "Frame overlap" tmp.log

expression: Frame overlap

option: -v (only show non-matching lines)

Listing a file, but removing all the initial blank lines

To list a file, but skip over all the blank lines at the start of the file, use the command:

AWK "/[^ ]/ { copy=1 }; copy { print }" filename.ext

default: copy=0 (because all variables initialize to zero or ""

expression: /[^ ]/ lines containing any characters other than ' '

action: copy=1 turn on the copy flag

expression: copy 'copy' flag has been set by the time of this line

action: print send this line to the output

This simple script will suppress all lines until it finds a line containing a non-space character. Then it will start copying lines, and will copy all the rest of the lines to the output. (Note that the flag 'copy' never gets turned off and that we rely on the AWK initialization of copy=0 to cause the initial lines to be skipped)

Are you using AWK in an interesting way? Have you found something else you can do with AWK that might be useful to others? Write to Rob to let me know.
You can download the source for AWK (DOS version) from SimTel or from IBM. There are several other DOS utilities (many with source) also available at SimTel's site.
- Look at some Awk scripts.
- What functions can I use?
- Back to the front gate.
- Visit the other wagons.

Do not meddle in the affairs of wizards, for they are subtle and quick to anger

Do not meddle in the affairs of dragons, for you are crispy and good with ketchup.

Page maintained by Rob.