The awk command in Linux
awk is a small programming language for columnar text. It splits each line into fields ($1, $2, …, whole line $0) and runs your program on every line — which makes "give me column 2 of this output" a five-character program: {print $2}. Add patterns, conditions and arithmetic and it replaces many small scripts.
How awk works
awk's worldview: input is a sequence of records (lines, by default) which split into fields on a separator (whitespace, by default — collapsing runs of it, which is why awk handles aligned command output that defeats cut). A program is a list of pattern-action pairs; for each record, every pattern is tested and its action runs on match. A missing pattern matches everything; a missing action prints the record. BEGIN and END are patterns for before-any-input and after-all-input — the natural home of headers and totals.
What lifts awk from extractor to language is its data structure: associative arrays, indexed by arbitrary strings, created on first touch. count[$1]++ builds a frequency table of a column in four characters; !seen[$0]++ de-duplicates while preserving order. Variables need no declaration and convert between string and number by context. Half the "impossible one-liners" you will meet are just an associative array and the record-field model doing what they were built for.
Syntax
awk [-F SEP] 'PROGRAM' [FILE...]Common options
| Option | What it does |
|---|---|
'{print $1}' | Print the first field of every line. |
-F"," | Set the field separator (default is any whitespace). |
'/pat/ {…}' | Run the action only on lines matching a pattern. |
'$3 > 100' | Conditions on fields select lines (default action: print). |
NR / NF | Built-ins: current line number / number of fields. |
END {…} | Run after all input — where totals get printed. |
How to use awk: examples
$ ps aux | awk '{print $2, $11}'Just the PID and command columns from ps.
$ awk -F',' '{print $3}' data.csvThird column of a CSV (for real-world CSVs with quoted commas, use a proper parser).
$ df -h | awk '$5+0 > 80 {print $6, $5}'Mount points over 80% full — the +0 coerces "85%" to a number.
$ awk '{sum += $1} END {print sum}' numbers.txtSum a column. The moment awk starts replacing spreadsheets.
$ awk 'NR % 2 == 1' file.txtEvery odd-numbered line.
$ awk -F: '{print $1}' /etc/passwd | sortAll user names on the system: /etc/passwd is colon-separated, field 1 is the name.
Real-world use cases for awk
Instant log analytics
Which endpoints are slow? awk '$10 > 1000 {print $7}' access.log | sort | uniq -c | sort -nr — filter requests over 1000 ms, extract the URL field, count by frequency. Four piped commands doing what would otherwise mean importing gigabytes into a database.
Ad-hoc reports from CSV exports
Finance sends a CSV; you need the total of column 4 for rows marked "paid": awk -F, '$2=="paid" {s+=$4} END {printf "%.2f\n", s}' export.csv. No spreadsheet opened, answer in one line, trivially re-runnable next month.
Pro tips and common mistakes
- $NF is the last field, $(NF-1) the second-to-last — indispensable when column counts vary.
- awk collapses runs of whitespace by default — exactly why it beats cut on aligned command output.
- Sum, count, average in one: {s+=$1; n++} END {print s, n, s/n}.
- De-duplicate preserving order (unlike sort -u): awk '!seen[$0]++' file — a famous one-liner worth memorizing.
Frequently asked questions about awk
When should I use awk instead of cut?
cut is fine for fixed single-character delimiters. awk wins whenever fields are separated by variable whitespace (like most command output), or when you need conditions, arithmetic or reordering.
How do I print the last field?
awk '{print $NF}' — NF holds the number of fields, so $NF is the last one and $(NF-1) the one before it.
Why doesn't $1 mean the same as in bash?
Inside single quotes the shell leaves $1 alone and awk interprets it as field one. That is exactly why awk programs are wrapped in single quotes — double quotes would let the shell eat the dollar signs.
Related commands
Find lines matching a pattern in files or piped input — the workhorse of text search.
sedStream-edit text: substitute, delete lines, and edit files in place with -i.
cutSlice fields or character ranges out of each line — quick column extraction for delimited data.