gawk (gnu awk)

Find and Replace text, database sort/validate/index

Syntax


      awk <options> 'Program' Input-File1 Input-File2 ...

      awk -f PROGRAM-FILE <options> Input-File1 Input-File2 ...

Key
`-F FS'
`--field-separator FS'
     Use FS for the input field separator (the value of the `FS'
     predefined variable).

`-f PROGRAM-FILE'
`--file PROGRAM-FILE'
     Read the `awk' program source from the file PROGRAM-FILE, instead
     of from the first command line argument.

`-mf NNN'
`-mr NNN'
     The `f' flag sets the maximum number of fields, and the `r' flag
     sets the maximum record size.  These options are ignored by
     `gawk', since `gawk' has no predefined limits; they are only for
     compatibility with the Bell Labs research version of Unix `awk'.

`-v VAR=VAL'
`--assign VAR=VAL'
     Assign the variable VAR the value VAL before program execution
     begins.

`-W traditional'
`-W compat'
`--traditional'
`--compat'
     Use compatibility mode, in which `gawk' extensions are turned off.

`-W lint'
`--lint'
     Give warnings about dubious or non-portable `awk' constructs.

`-W lint-old'
`--lint-old'
     Warn about constructs that are not available in the original
     Version 7 Unix version of `awk'.

`-W posix'
`--posix'
     Use POSIX compatibility mode, in which `gawk' extensions are
     turned off and additional restrictions apply.

`-W re-interval'
`--re-interval'
     Allow interval expressions, in regexps.

`-W source=PROGRAM-TEXT'
`--source PROGRAM-TEXT'
     Use PROGRAM-TEXT as `awk' program source code.  This option allows
     mixing command line source code with source code from files, and is
     particularly useful for mixing command line programs with library
     functions.

`--'
     Signal the end of options.  This is useful to allow further
     arguments to the `awk' program itself to start with a `-'.  This
     is mainly for consistency with POSIX argument parsing conventions.

'Program'
     A series of patterns and actions: see below

Input-File
     If no Input-File is specified then `awk' applies the Program to 
     "standard input", (piped output of some other command or the terminal.
     Typed input will continue until end-of-file (typing `Control-d')

Basic functions

The basic function of awk is to search files for lines (or other units of text) that contain a pattern. When a line matches, awk performs a specific action on that line.

The Program statement that tells `awk' what to do; consists of a series of "rules". Each rule specifies one pattern to search for, and one action to perform when that pattern is found.

For ease of reading, each line in an `awk' program is normally a separate Program statement , like this:

     pattern { action }
     pattern { action }
     ...

e.g. Display lines from my_file containing the string "123" or "abc" or "some text":

awk '/123/ { print $0 } 
     /abc/ { print $0 }
     /some text/ { print $0 }' my_file

A regular expression enclosed in slashes (`/') is an `awk' pattern that matches every input record whose text belongs to that set. e.g. the pattern /foo/ matches any input record containing the three characters `foo', *anywhere* in the record.

`awk' patterns may be one of the following:

/Regular Expression/        - Match =
Pattern && Pattern          - AND
Pattern || Pattern          - OR
! Pattern                   - NOT
Pattern ? Pattern : Pattern - If, Then, Else
Pattern1, Pattern2          - Range Start - end
BEGIN                       - Perform action BEFORE input file is read
END                         - Perform action AFTER input file is read

In addition to simple pattern matching `awk' has a huge range of text and arithmetic Functions, Variables and Operators.

`gawk' will ignore newlines after any of the following:

    , { ? : || && do else

Comments - start with a `#', and continue to the end of the line:

 # This program prints a nice friendly message

Examples

This program prints the length of the longest input line:

 awk '{ if (length($0) > max) max = length($0) }
      END { print max }' data

This program prints every line that has at least one field. This is an easy way to delete blank lines from a file (or rather, to
create a new file similar to the old file but from which the blank lines have been deleted)

 awk 'NF > 0' data

This program prints seven random numbers from zero to 100, inclusive.

 awk 'BEGIN { for (i = 1; i <= 7; i++)
                print int(101 * rand()) }'

This program prints the total number of bytes used by FILES.

 ls -lg FILES | awk '{ x += $5 } ; END { print "total bytes: " x }'

This program prints a sorted list of the login names of all users.

 awk -F: '{ print $1 }' /etc/passwd | sort

This program counts lines in a file.

 awk 'END { print NR }' data

This program prints the even numbered lines in the data file. If you were to use the expression `NR % 2 == 1' instead, it would print the odd numbered lines.

 awk 'NR % 2 == 0' data

"Justice is such a fine thing that we cannot pay too dearly for it - Alain-Rene Lesage

Related Linux Bash commands:

GNU Awk User Guide - more examples
`awk', `oawk', and `nawk' - Alternative, older and newer versions of awk
egrep - egrep foo FILES ...is essentially the same as awk '/foo/' FILES ...
expr - Evaluate expressions
eval - Evaluate several commands/arguments
for - Expand words, and execute commands
grep - search file(s) for lines that match a given pattern
m4 - Macro processor
tr - Translate, squeeze, and/or delete characters

Equivalent Windows XP commands:

FOR - Conditionally perform a command several times
SET - Display, set, or remove Windows NT environment variables