AWK is a general purpose computer language that is designed for processing text-based data, either in files or data streams. The name AWK is derived from the surnames of its authors — Alfred Aho, Peter Weinberger, and Brian Kernighan; however, it is commonly pronounced "awk" and not as a string of separate letters. awk, when written in all lowercase letters, refers to the Unix or Plan 9 program that runs other programs written in the AWK programming language. A programming paradigm is a paradigmatic style of programming (compare with a methodology, which is a paradigmatic style of doing software engineering). ... Scripting languages (commonly called scripting programming languages or script languages) are computer programming languages initially used only for simple, repeated actions. ... This article or section does not cite its references or sources. ... Event-driven programming is a computer programming paradigm. ... Dr. Alfred V. Aho is a computer scientist. ... Peter J. Weinberger is a computer scientist who worked at AT&T Bell Labs and contributed to the design of the pioneering AWK programming language (he is the W in AWK). ... Brian W. Kernighan (pronounced Ker-ni-han; the g is silent; born 1942) is a computer scientist who worked at the Bell Labs and contributed to the design of the pioneering AWK and AMPL programming languages. ... In computer science, a type system defines how a programming language classifies values and variables into types, how it can manipulate those types and how they interact. ... The C Programming Language, Brian Kernighan and Dennis Ritchie, the original edition that served for many years as an informal specification of the language The C programming language (often, just C) is a general-purpose, procedural, imperative computer programming language developed in the early 1970s by Dennis Ritchie for use... To meet Wikipedias quality standards, this article or section may require cleanup. ... The Bourne shell, or sh, was the default Unix shell of Unix Version 7, and replaced the Thompson shell, whose executable file had the same name, sh. ... Perl, also Practical Extraction and Report Language (a backronym, see below) is a dynamic procedural programming language designed by Larry Wall and first released in 1987. ... Korn shell logo. ... The syntax of JavaScript is a set of rules that defines how a JavaScript program will be written and interpreted. ... A computer language is a language used by, or in association with, computers. ... A family name, or surname, is that part of a persons name that indicates to what family he or she belongs. ... Dr. Alfred V. Aho is a computer scientist. ... Peter J. Weinberger is a computer scientist who worked at AT&T Bell Labs and contributed to the design of the pioneering AWK programming language (he is the W in AWK). ... Brian W. Kernighan (pronounced Ker-ni-han; the g is silent; born 1942) is a computer scientist who worked at the Bell Labs and contributed to the design of the pioneering AWK and AMPL programming languages. ... Unix or UNIX is a computer operating system originally developed in the 1960s and 1970s by a group of AT&T Bell Labs employees including Ken Thompson, Dennis Ritchie, and Douglas McIlroy. ... To meet Wikipedias quality standards, this article or section may require cleanup. ...

AWK is an example of a programming language that extensively uses the string datatype, associative arrays (that is, arrays indexed by key strings), and regular expressions. The power, terseness, and limitations of AWK programs and sed scripts inspired Larry Wall to write Perl. Because of their dense notation, all these languages are often used for writing one-liner programs. Computer code (HTML with JavaScript) in a tool that uses syntax highlighting (colors) to help the developer see the purpose of each piece of code. ... In computer programming and some branches of mathematics, strings are sequences of various simple objects. ... In computer science, a datatype or data type (often simply a type) is a name or label for a set of values and some operations which one can perform on that set of values. ... An associative array (also dictionary, finite map, map, lookup table, and in query-processing an index or index file) is an abstract data type composed of a collection of keys and a collection of values, where each key is associated with one value. ... A regular expression (abbreviated as regexp or regex, with plural forms regexps, regexes, or regexen) is a string that describes or matches a set of strings, according to certain syntax rules. ... The correct title of this article is sed. ... Larry Wall (b. ... Perl, also Practical Extraction and Report Language (a backronym, see below) is a dynamic procedural programming language designed by Larry Wall and first released in 1987. ... A one-liner is a computer program or expression that takes no more than a single line. ...

AWK is one of the early tools to appear in Version 7 Unix and gained popularity as a way to add computational features to a Unix pipeline. A version of the AWK language is a standard feature of nearly every modern Unix-like operating system available today. AWK is mentioned in the Single UNIX Specification as one of the mandatory utilities of a Unix operating system. Besides the Bourne shell, AWK is the only other scripting language available in a standard Unix environment. Implementations of AWK exist as installed software for almost all other operating systems. The Version 7 Unix, the Seventh Edition Unix, was an important early release of the Unix operating system. ... A Unix-like operating system is one that behaves in a manner similar to a Unix system, while not necessarily conforming to or being certified to any version of the Single UNIX Specification. ... An operating system (OS) is an essential software program that manages the hardware and software resources of a computer. ... The Single UNIX Specification (SUS) is the collective name of a family of standards for computer operating systems to qualify for the name Unix. The SUS is developed and maintained by the Austin Group, based on earlier work by the IEEE and The Open Group. ... Unix or UNIX is a computer operating system originally developed in the 1960s and 1970s by a group of AT&T Bell Labs employees including Ken Thompson, Dennis Ritchie, and Douglas McIlroy. ... An operating system (OS) is an essential software program that manages the hardware and software resources of a computer. ... The Bourne shell, or sh, was the default Unix shell of Unix Version 7, and replaced the Thompson shell, whose executable file had the same name, sh. ...

Structure of AWK programs

Generally speaking, two pieces of data are given to AWK: a command file and a primary input file. A command file (which can be an actual file, or can be included in the command line invocation of awk) contains a series of commands which tell AWK how to process the input file. The primary input file is typically text that is formatted in some way; it can be an actual file, or it can be read by awk from the standard input. A typical AWK program consists of a series of lines, each of the form A command line interface or CLI is a method of interacting with a computer by giving it lines of textual commands (that is, a sequence of characters) either from keyboard input or from a script. ...

 /pattern/ { action } 

where pattern is a regular expression and action is a command. Most implementations of AWK use extended regular expressions by default. AWK looks through the input file; when it finds a line that matches pattern, it executes the command(s) specified in action. Alternate line forms include: A regular expression (abbreviated as regexp or regex, with plural forms regexps, regexes, or regexen) is a string that describes or matches a set of strings, according to certain syntax rules. ...

BEGIN { action }

Executes action commands at the beginning of the script execution, i.e. before any of the lines are processed.

END { action }

Similar to the previous form, but executes action after the end of input.

/pattern/

Prints any lines matching pattern.

{ action }

Executes action for each line in the input.

Each of these forms can be included multiple times in the command file. Lines in the command file are executed in order, so if there are two "BEGIN" statements, the first is executed, then the second, and then the rest of the lines. BEGIN and END statements do not have to be located before and after (respectively) the other lines in the command file.

AWK was created as a broadbased replacement to C algorithmic approaches developed to integrate text parsing methods. The C Programming Language, Brian Kernighan and Dennis Ritchie, the original edition that served for many years as an informal specification of the language The C programming language (often, just C) is a general-purpose, procedural, imperative computer programming language developed in the early 1970s by Dennis Ritchie for use...

AWK commands

AWK commands are the statement that is substituted for action in the examples above. AWK commands can include function calls, variable assignments, calculations, or any combination thereof. AWK contains built-in support for many functions; many more are provided by the various flavors of AWK. Also, some flavors support the inclusion of dynamically linked libraries, which can also provide more functions. In computer science, a library is a collection of subprograms used to develop software. ...

For brevity, the enclosing curly braces ( { } ) will be omitted from these examples.

The print command

The print command is used to output text. The simplest form of this command is

 print 

This displays the contents of the current line. In AWK, lines are broken down into fields, and these can be displayed separately:

print $1

Displays the first field of the current line

print $1, $3

Displays the first and third fields of the current line, separated by a predefined string called the output field separator (OFS) whose default value is a single space character

Although these fields ($X) may bear resemblance to variables (the $ symbol indicates variables in perl), they actually refer to the fields of the current line. A special case, $0, refers to the entire line. In fact, the commands "print" and "print $0" are identical in functionality.

The print command can also display the results of calculations and/or function calls:

 print 3+2 print foobar(3) print foobar(variable) print sin(3-2) 

Output may be sent to a file:

 print "expression" > "file name" 

Variables, et cetera

Variable names can use any of the characters [A-Za-z0-9_], with the exception of language keywords. The operators + - * / are addition, subtraction, multiplication, and division, respectively. For string concatenation, simply place two variables (or string constants) next to each other, optionally with a space in between. String constants are delimited by double quotes. Statements need not end with semicolons. Finally, comments can be added to programs by using # as the first character on a line. Delimited data uses specific characters (delimiters) to separate its values. ...

User-defined functions

In a format similar to C, function definitions consist of the keyword function, the function name, argument names and the function body. Here is an example function: The C Programming Language, Brian Kernighan and Dennis Ritchie, the original edition that served for many years as an informal specification of the language The C programming language (often, just C) is a general-purpose, procedural, imperative computer programming language developed in the early 1970s by Dennis Ritchie for use...

 function add_three (number, temp) { temp = number + 3 return temp } 

This statement can be invoked as follows:

 print add_three(36) # prints 39 

Functions can have variables that are in the local scope. The names of these are added to the end of the argument list, though values for these should be omitted when calling the function. It is convention to add some whitespace in the argument list before the local variables, in order to indicate where the parameters end and the local variables begin. For information on the programming language Whitespace, see Whitespace programming language. ...

Sample applications

Hello World

Here is the ubiquitous "Hello world program" program written in AWK: To meet Wikipedias quality standards, this article or section may require cleanup. ...

 BEGIN { print "Hello, world!"; exit } 

Print lines longer than 80 characters

Print all lines longer than 80 characters. Note that the default action is to print the current line.

 length > 80 

Print a count of words

Count words in the input, and print lines, words, and characters (like wc) wc (short for word count) is a command in Unix-like operating systems. ...

 { w += NF; c += length} END { print NR, w, c } 

Sum first column

Sum first column of input

 { s += $1 } END { print s } 

Calculate word frequencies

Word frequency, (uses associative arrays) An associative array (also dictionary, finite map, map, lookup table, and in query-processing an index or index file) is an abstract data type composed of a collection of keys and a collection of values, where each key is associated with one value. ...

 BEGIN { FS="[^a-zA-Z]+"} { for (i=1; i<=NF; i++) words[tolower($i)]++ } END { for (i in words) print i, words[i] } 

Self-contained AWK scripts

As with many other programming languages, self-contained AWK script can be constructed using the so-called "shebang" syntax. In computing, a shebang is a specific pair of characters used in a special line that begins a text file (commonly called a script) causing Unix-like operating systems to execute the commands in the text file using a specified interpreter (program) when executed. ...

For example, a UNIX command called hello.awk that prints the string "Hello, world!" may be built by going first creating a file named hello.awk containing the following lines:

 #!/usr/bin/awk -f BEGIN { print "Hello, world!"; exit } 

AWK versions and implementations

AWK was originally written in 1977, and distributed with Version 7 Unix. For the album by Ash, see 1977 (album). ... The Version 7 Unix, the Seventh Edition Unix, was an important early release of the Unix operating system. ...

In 1985 its authors started expanding the language, most significantly by adding user-defined functions. The language is described in the book The AWK Programming Language, published 1988, and its implementation was made available in releases of UNIX System V. To avoid confusion with the incompatible older version, this version was sometimes known as "new awk" or nawk. This implementation was released under a free software license in 1996, and is still maintained by Brian Kernighan. (see external links below) This article is about the year. ... 1988 (MCMLXXXVIII) was a leap year starting on Friday of the Gregorian calendar. ... It has been suggested that Traditional Unix be merged into this article or section. ... Generally speaking, free software license is a phrase used by the free software movement to mean any software license that meets the free software definition of the Free Software Foundation (FSF). ... 1996 (MCMXCVI) was a leap year starting on Monday of the Gregorian calendar, and was designated the International Year for the Eradication of Poverty. ...

GNU awk, or gawk, is another free software implementation. It was written before the original implementation became freely available, and is still widely used. Almost every Linux distribution comes with a recent version of gawk and gawk is widely recognized as the de-facto standard implementation in the Linux world. GNU (pronounced ) is a free software operating system consisting of a kernel, libraries, system tools, compilers and many end-user applications. ... A Linux distribution is a Unix-like operating system comprising the Linux kernel and other assorted free software/open-source software, and possibly proprietary software. ... Linux (also known as GNU/Linux) is a computer operating system. ...

xgawk is a SourceForge project based on gawk. It extends gawk with dynamically loadable libraries.

mawk is a very fast AWK implementation by Mike Brennan based on a byte code interpreter. Byte-code is a sort of intermediate code that is more abstract than machine code. ...

Downloads and further information about these versions are available from the sites listed below.

Thompson AWK or TAWK is an AWK compiler for DOS and Windows, previously sold by Thompson Automation Software (which has ceased its activities). A diagram of the operation of a typical multi-language compiler. ... Microsofts disk operating system, MS-DOS, was Microsofts implementation of DOS, which was the first popular operating system for the IBM PC, and until recently, was widely used on the PC compatible platform. ... As of 2006, Microsoft Windows is the worlds most common operating system for use on personal computers. ...

Criticism

Three kinds of criticism can be distinguished:

1. Language specific
   1. AWK cannot handle NULL characters in the input and output streams. The GNU implementation has resolved this problem, but AWK in general has not.
   2. AWK looks like C, but it isn't C. Local variables (inside functions) are only emulated. Multidimensional arrays are only emulated. The switch statement is missing (only available in GNU implementation).
   3. Checking strings for numeric contents is difficult.
2. Implementation specific
   1. Sun's implementation is notoriously different from standards and other implementations. Solaris comes with three different implementations of AWK: oawk (aka old awk), nawk (aka new awk), pawk (aka POSIX awk) which all behave differently in some situations.
   2. Implementations targeting MS-DOS and MS Windows are notorious for several problems
      1. Memory allocation is sometimes still limited to 640 KB.
      2. The most common problem with AWK on Microsoft-platforms is the quoting problem: When using AWK source code on the command line, users often overlook the details of quoting. While the ' and " characters have a well-defined meaning in Unix environments, their semantics in the command line interpreter of Microsoft is different.
      3. Text lines are terminated with a carriage return and an additional line feed character. Platform like Cygwin have solutions for this.

1. Myths and Misconceptions
   1. Line length is limited to 4096 or 8192 characters. This myth originates from early versions.
   2. AWK has no functions or subroutines. This is only true for the oawk implementation from 1977.
   3. AWK has no hashes. AWK predates the invention of the term hash. AWK and its predecessor SNOBOL were actually the first languages which offered hashes, but under the associative array moniker.
   4. i18n (internationalization) breaks old scripts. With the introduction of Unicode and international locales, the order of characters depends on the locale; therefore regular expressions like [A-z] change their semantics in different locales. In some European locales, the decimal point is replaced by a comma when printing numbers. So, the output of all scripts printing numbers depends on the locale. Reading back this printed data may truncate numbers without notice.
   5. AWK has been replaced by other scripting languages like Perl. AWK will not disappear or be replaced. Since the introduction of Unix standards, AWK has always been mentioned in documents like the current Single UNIX Specification as one of the mandatory utilities of a Unix operating system

The POSIX standard for AWK requires that "The value of an integer constant beginning with 0 shall be taken in decimal rather than octal". (In C and other languages derived from it, integer literals beginning with 0 are read as octal.) Thenull character (also null terminator) is a character with the value zero, present in the ASCII and Unicode character sets, and available in nearly all mainstream programming languages. ... Originally, carriage return was the term for the key, lever, or mechanism on a typewriter that would cause the cylinder on which the paper was held (the carriage) to return to the left side of the paper after a line of text had been typed, and would often move it... In computing, line feed (LF) is a control character indicating that one line should be fed out. ... Cygwin is a collection of free software tools originally developed by Cygnus Solutions to allow various versions of Microsoft Windows to act somewhat like a Unix system. ... In computer science, a hash table is a data structure that speeds up searching for information by a particular aspect of that information, called a key. ... Internationalization and localization are means of adapting products such as publications or software for non-native environments, especially other nations and cultures. ... Due to technical limitations, some web browsers may not display some special characters in this article. ... Perl, also Practical Extraction and Report Language (a backronym, see below) is a dynamic procedural programming language designed by Larry Wall and first released in 1987. ... The Single UNIX Specification (SUS) is the collective name of a family of standards for computer operating systems to qualify for the name Unix. The SUS is developed and maintained by the Austin Group, based on earlier work by the IEEE and The Open Group. ... Unix or UNIX is a computer operating system originally developed in the 1960s and 1970s by a group of AT&T Bell Labs employees including Ken Thompson, Dennis Ritchie, and Douglas McIlroy. ... An operating system (OS) is an essential software program that manages the hardware and software resources of a computer. ...

This is the behaviour of most versions of AWK, the GNU version, gawk, being the exception.

In gawk:

 $ gawk 'BEGIN { print "06612"; print 6612; print 06612;}' 06612 6612 3466 $ 

In other AWKs (pre-1987 awk, new awk, Brian Kernighan's One True Awk, mawk) the POSIX behaviour is standard. For example:

 $ mawk 'BEGIN { print "06612"; print 6612; print 06612;}' 06612 6612 6612 $ nawk 'BEGIN { print "06612"; print 6612; print 06612;}' 06612 6612 6612 

The POSIX behaviour is available with gawk by setting the environment variable POSIXLY_CORRECT, or by calling gawk with the --posix option:

 $ gawk --posix 'BEGIN { print "06612"; print 6612; print 06612;}' 06612 6612 6612 

Digression

• The bird emblematic of AWK (a.o. on The AWK Programming Language book cover) is the Auk.

Genera Alle Uria Alca Pinguinus Cepphus Brachyramphus Synthliboramphus Ptychoramphus Cyclorrhynchus Aethia Cerorhinca Fratercula This article is about a family of birds. ...

Books

• Alfred V. Aho, Brian W. Kernighan, and Peter J. Weinberger (1988). The AWK Programming Language. Addison-Wesley. ISBN 0-201-07981-X. The book's webpage includes downloads of the current implementation of Awk and links to others.
• Arnold Robbins. Effective awk Programming, Edition 3. Arnold Robbins maintains the GNU Awk implementation of AWK for more than 10 years. The free GNU Awk manual was also published by O'Reilly in May 2001. Free download of this manual is possible through the following book references.
• Arnold Robbins. GAWK: Effective AWK Programming: A User's Guide for GNU Awk, Edition 3.
• Dale Dougherty, Arnold Robbins (March 1997). sed & awk, Second Edition, Second Edition, O'Reilly Media. ISBN 1-56592-225-5.

Dr. Alfred V. Aho is a computer scientist. ... Brian Kernighan¹ (born 1942) is a computer scientist who worked at the Bell Labs and contributed to the design of the pioneering AWK and AMPL programming languages. ... Peter J. Weinberger is a computer scientist who worked at AT&T Bell Labs and contributed to the design of the pioneering AWK programming language (he is the W in AWK). ... Dale Dougherty is one of the co-founders (with Tim OReilly) of OReilly Media. ... Programming Perl is a classic OReilly book. ...

See also

• sed
• List of Unix programs

The correct title of this article is sed. ... This is a list of Unix programs. ...

External links

• awk maintained by Brian Kernighan.
• comp.lang.awk is a USENET newsgroup dedicated to AWK.
• GAWK (GNU awk) webpage
• mawk download site
• DJGPP port of Gawk 3.11b as a downloadable 768KB zipfile
• xgawk download site
• Awka Open Source, AWK to C Conversion Tool
• TAWK Compiler