GAWK: Effective AWK Programming

A User's Guide for GNU Awk

Edition 3

March, 2001

• Foreword
• Preface
  • History of @command{awk} and @command{gawk}
  • 1.0 A Rose by Any Other Name
  • 1.1 Using This Book
  • 1.2 Typographical Conventions
  • The GNU Project and This Book
  • How to Contribute
  • Acknowledgments
• 2. Getting Started with @command{awk}
  • 2.1 How to Run @command{awk} Programs
    • 2.1.1 One-Shot Throw-Away @command{awk} Programs
    • 2.1.2 Running @command{awk} Without Input Files
    • 2.1.3 Running Long Programs
    • 2.1.4 Executable @command{awk} Programs
    • 2.1.6 Comments in @command{awk} Programs
    • 2.1.7 Shell Quoting Issues
  • 2.2 Data Files for the Examples
  • 2.3 Some Simple Examples
  • 2.4 An Example with Two Rules
  • 2.5 A More Complex Example
  • 2.6 @command{awk} Statements Versus Lines
  • 2.7 Other Features of @command{awk}
  • 2.8 When to Use @command{awk}
• 3. Regular Expressions
  • 3.1 How to Use Regular Expressions
  • 3.2 Escape Sequences
  • 3.3 Regular Expression Operators
  • 3.4 Using Character Lists
  • 3.5 @command{gawk}-Specific Regexp Operators
  • 3.6 Case Sensitivity in Matching
  • 3.7 How Much Text Matches?
  • 3.8 Using Dynamic Regexps
• 4. Reading Input Files
  • 4.1 How Input Is Split into Records
  • 4.2 Examining Fields
  • 4.3 Non-Constant Field Numbers
  • 4.4 Changing the Contents of a Field
  • 4.5 Specifying How Fields Are Separated
    • 4.5.1 Using Regular Expressions to Separate Fields
    • 4.5.2 Making Each Character a Separate Field
    • 4.5.3 Setting FS from the Command Line
    • 4.5.4 Field Splitting Summary
  • 4.6 Reading Fixed-Width Data
  • 4.7 Multiple-Line Records
  • 4.8 Explicit Input with getline
    • 4.8.1 Using getline with No Arguments
    • 4.8.2 Using getline into a Variable
    • 4.8.3 Using getline from a File
    • 4.8.4 Using getline into a Variable from a File
    • 4.8.5 Using getline from a Pipe
    • 4.8.6 Using getline into a Variable from a Pipe
    • 4.8.7 Using getline from a Coprocess
    • 4.8.8 Using getline into a Variable from a Coprocess
    • 4.8.9 Points About getline to Remember
    • 4.8.10 Summary of getline Variants
• 5. Printing Output
  • 5.1 The print Statement
  • 5.2 Examples of print Statements
  • 5.3 Output Separators
  • 5.4 Controlling Numeric Output with print
  • 5.5 Using printf Statements for Fancier Printing
    • 5.5.1 Introduction to the printf Statement
    • 5.5.2 Format-Control Letters
    • 5.5.3 Modifiers for printf Formats
    • 5.5.4 Examples Using printf
  • 5.6 Redirecting Output of print and printf
  • 5.7 Special File Names in @command{gawk}
    • 5.7.1 Special Files for Standard Descriptors
    • 5.7.2 Special Files for Process-Related Information
    • 5.7.3 Special Files for Network Communications
    • 5.7.4 Special File Name Caveats
  • 5.8 Closing Input and Output Redirections
• 6. Expressions
  • 6.1 Constant Expressions
    • 6.1.1 Numeric and String Constants
    • 6.1.2 Octal and Hexadecimal Numbers
    • 6.1.4 Regular Expression Constants
  • 6.2 Using Regular Expression Constants
  • 6.3 Variables
    • 6.3.1 Using Variables in a Program
    • 6.3.2 Assigning Variables on the Command Line
  • 6.4 Conversion of Strings and Numbers
  • 6.5 Arithmetic Operators
  • 6.6 String Concatenation
  • 6.7 Assignment Expressions
  • 6.8 Increment and Decrement Operators
  • 6.9 True and False in @command{awk}
  • 6.10 Variable Typing and Comparison Expressions
  • 6.11 Boolean Expressions
  • 6.12 Conditional Expressions
  • 6.13 Function Calls
  • 6.14 Operator Precedence (How Operators Nest)
• 7. Patterns, Actions, and Variables
  • 7.1 Pattern Elements
    • 7.1.1 Regular Expressions as Patterns
    • 7.1.2 Expressions as Patterns
    • 7.1.3 Specifying Record Ranges with Patterns
    • 7.1.4 The BEGIN and END Special Patterns
      • 7.1.4.1 Startup and Cleanup Actions
      • 7.1.4.2 Input/Output from BEGIN and END Rules
    • 7.1.5 The Empty Pattern
  • 7.2 Using Shell Variables in Programs
  • 7.3 Actions
  • 7.4 Control Statements in Actions
    • 7.4.1 The if-else Statement
    • 7.4.2 The while Statement
    • 7.4.3 The do-while Statement
    • 7.4.4 The for Statement
    • 7.4.5 The break Statement
    • 7.4.6 The continue Statement
    • 7.4.7 The next Statement
    • 7.4.8 Using @command{gawk}'s nextfile Statement
    • 7.4.9 The exit Statement
  • 7.5 Built-in Variables
    • 7.5.1 Built-in Variables That Control @command{awk}
    • 7.5.2 Built-in Variables That Convey Information
    • 7.5.4 Using ARGC and ARGV
• 8. Arrays in @command{awk}
  • 8.1 Introduction to Arrays
  • 8.2 Referring to an Array Element
  • 8.3 Assigning Array Elements
  • 8.4 Basic Array Example
  • 8.5 Scanning All Elements of an Array
  • 8.6 The delete Statement
  • 8.7 Using Numbers to Subscript Arrays
  • 8.8 Using Uninitialized Variables as Subscripts
  • 8.9 Multidimensional Arrays
  • 8.10 Scanning Multidimensional Arrays
  • 8.11 Sorting Array Values and Indices with @command{gawk}
• 9. Functions
  • 9.1 Built-in Functions
    • 9.1.1 Calling Built-in Functions
    • 9.1.2 Numeric Functions
    • 9.1.3 String Manipulation Functions
      • 9.1.3.1 More About `\' and `&' with sub, gsub, and gensub
    • 9.1.5 Input/Output Functions
    • 9.1.8 Using @command{gawk}'s Timestamp Functions
    • 9.1.9 Using @command{gawk}'s Bit Manipulation Functions
    • 9.1.10 Using @command{gawk}'s String Translation Functions
  • 9.2 User-Defined Functions
    • 9.2.1 Function Definition Syntax
    • 9.2.2 Function Definition Examples
    • 9.2.3 Calling User-Defined Functions
    • 9.2.4 The return Statement
    • 9.2.5 Functions and Their Effect on Variable Typing
• 10. Internationalization with @command{gawk}
  • 10.1 Internationalization and Localization
  • 10.2 GNU gettext
  • 10.3 Internationalizing @command{awk} Programs
  • 10.4 Translating @command{awk} Programs
    • 10.4.1 Extracting Marked Strings
    • 10.4.2 Rearranging printf Arguments
    • 10.4.3 @command{awk} Portability Issues
  • 10.5 A Simple Internationalization Example
  • 10.6 @command{gawk} Can Speak Your Language
• 11. Advanced Features of @command{gawk}
  • 11.1 Allowing Non-Decimal Input Data
  • 11.2 Two-Way Communications with Another Process
  • 11.3 Using @command{gawk} for Network Programming
  • 11.4 Using @command{gawk} with BSD Portals
  • 11.5 Profiling Your @command{awk} Programs
• 12. Running @command{awk} and @command{gawk}
  • 12.1 Invoking @command{awk}
  • 12.2 Command-Line Options
  • 12.3 Other Command-Line Arguments
  • 12.4 The @env{AWKPATH} Environment Variable
  • 12.5 Obsolete Options and/or Features
  • 12.6 Undocumented Options and Features
  • 12.7 Known Bugs in @command{gawk}
• 13. A Library of @command{awk} Functions
  • 13.1 Naming Library Function Global Variables
  • 13.2 General Programming
    • 13.2.1 Implementing nextfile as a Function
    • 13.2.2 Assertions
    • 13.2.3 Rounding Numbers
    • 13.2.4 The Cliff Random Number Generator
    • 13.2.5 Translating Between Characters and Numbers
    • 13.2.6 Merging an Array into a String
    • 13.2.7 Managing the Time of Day
  • 13.3 Data File Management
    • 13.3.1 Noting Data File Boundaries
    • 13.3.2 Rereading the Current File
    • 13.3.3 Checking for Readable Data Files
    • 13.3.4 Treating Assignments as File Names
  • 13.4 Processing Command-Line Options
  • 13.5 Reading the User Database
  • 13.6 Reading the Group Database
• 14. Practical @command{awk} Programs
  • 14.1 Running the Example Programs
  • 14.2 Reinventing Wheels for Fun and Profit
    • 14.2.1 Cutting out Fields and Columns
    • 14.2.2 Searching for Regular Expressions in Files
    • 14.2.3 Printing out User Information
    • 14.2.4 Splitting a Large File into Pieces
    • 14.2.5 Duplicating Output into Multiple Files
    • 14.2.6 Printing Non-Duplicated Lines of Text
    • 14.2.7 Counting Things
  • 14.3 A Grab Bag of @command{awk} Programs
    • 14.3.1 Finding Duplicated Words in a Document
    • 14.3.2 An Alarm Clock Program
    • 14.3.3 Transliterating Characters
    • 14.3.4 Printing Mailing Labels
    • 14.3.5 Generating Word Usage Counts
    • 14.3.6 Removing Duplicates from Unsorted Text
    • 14.3.7 Extracting Programs from Texinfo Source Files
    • 14.3.8 A Simple Stream Editor
    • 14.3.9 An Easy Way to Use Library Functions
• A. The Evolution of the @command{awk} Language
  • A.1 Major Changes Between V7 and SVR3.1
  • A.2 Changes Between SVR3.1 and SVR4
  • A.3 Changes Between SVR4 and POSIX @command{awk}
  • A.4 Extensions in the Bell Laboratories @command{awk}
  • A.5 Extensions in @command{gawk} Not in POSIX @command{awk}
  • A.6 Major Contributors to @command{gawk}
• B. Installing @command{gawk}
  • B.1 The @command{gawk} Distribution
    • B.1.1 Getting the @command{gawk} Distribution
    • B.1.2 Extracting the Distribution
    • B.1.3 Contents of the @command{gawk} Distribution
  • B.2 Compiling and Installing @command{gawk} on Unix
    • B.2.1 Compiling @command{gawk} for Unix
    • B.2.2 Additional Configuration Options
    • B.2.3 The Configuration Process
  • B.3 Installation on Other Operating Systems
    • B.3.1 Installing @command{gawk} on an Amiga
    • B.3.2 Installing @command{gawk} on BeOS
    • B.3.3 Installation on PC Operating Systems
      • B.3.3.1 Installing a Prepared Distribution for PC Systems
      • B.3.3.2 Compiling @command{gawk} for PC Operating Systems
      • B.3.3.3 Using @command{gawk} on PC Operating Systems
    • B.3.4 How to Compile and Install @command{gawk} on VMS
      • B.3.4.1 Compiling @command{gawk} on VMS
      • B.3.4.2 Installing @command{gawk} on VMS
      • B.3.4.3 Running @command{gawk} on VMS
      • B.3.4.4 Building and Using @command{gawk} on VMS POSIX
  • B.4 Unsupported Operating System Ports
    • B.4.1 Installing @command{gawk} on the Atari ST
      • B.4.1.1 Compiling @command{gawk} on the Atari ST
      • B.4.1.2 Running @command{gawk} on the Atari ST
    • B.4.2 Installing @command{gawk} on a Tandem
  • B.5 Reporting Problems and Bugs
  • B.6 Other Freely Available @command{awk} Implementations
• C. Implementation Notes
  • C.1 Downward Compatibility and Debugging
  • C.2 Making Additions to @command{gawk}
    • C.2.1 Adding New Features
    • C.2.2 Porting @command{gawk} to a New Operating System
  • C.3 Adding New Built-in Functions to @command{gawk}
    • C.3.1 A Minimal Introduction to @command{gawk} Internals
    • C.3.2 Directory and File Operation Built-ins
      • C.3.2.1 Using chdir and stat
      • C.3.2.2 C Code for chdir and stat
      • C.3.2.3 Integrating the Extensions
  • C.4 Probable Future Extensions
• D. Basic Programming Concepts
  • D.1 What a Program Does
  • D.2 Data Values in a Computer
  • D.3 Floating-Point Number Caveats
• Glossary
• GNU General Public License
  • Preamble
  • Terms and Conditions for Copying, Distribution and Modification
  • How to Apply These Terms to Your New Programs
• GNU Free Documentation License
  • ADDENDUM: How to use this License for your documents
• Index

This document was generated on 13 July 2001 using texi2html 1.56k.