AWK Command Examples

This directory contains comprehensive examples for the yupsh awk command, organized from simplest to most sophisticated. Each example is in its own test file and demonstrates a specific feature or pattern.

Organization

Examples are organized by complexity level to help you learn progressively:

Basic Field Operations

Simple examples that demonstrate basic field printing and manipulation.

1. printfield_test.go - Print a specific field ($2)

   # Equivalent: awk '{print $2}'

2. printmultiplefields_test.go - Print multiple fields with separator

   # Equivalent: awk '{print $1, $3}'

3. lastfield_test.go - Print the last field using $NF

   # Equivalent: awk '{print $NF}'

Built-in Variables

Examples using awk's built-in variables like NR (line number) and NF (field count).

4. linenumbers_test.go - Use NR for line numbers

   # Equivalent: awk '{print NR": "$0}'

5. fieldcount_test.go - Use NF for field count

   # Equivalent: awk '{print NF" fields"}'

6. countlines_test.go - Count total lines using END block

   # Equivalent: awk 'END{print NR}'

Field Separators

Examples demonstrating custom input and output field separators.

7. customfieldseparator_test.go - Custom input separator (:)

   # Equivalent: awk -F: '{print $2}'

8. customoutputseparator_test.go - Custom output separator (,)

   # Equivalent: awk 'BEGIN{OFS=","} {print $1,$2,$3}'

9. csvprocessing_test.go - Process CSV data with formatting

   # Equivalent: awk -F, '{print $1": "$2" years old"}'

Text Transformation

Examples showing how to transform and reorder text.

10. uppercase_test.go - Convert text to uppercase

    # Equivalent: awk '{print toupper($0)}'

11. reversefields_test.go - Reverse field order

    # Equivalent: awk '{for(i=NF;i>=1;i--)print $i}'

12. fieldmodification_test.go - Modify fields in-place

    # Equivalent: awk '{$2="MODIFIED"; print}'

Program Flow Control

Examples using BEGIN and END blocks for initialization and finalization.

13. begin_test.go - Initialize with BEGIN block

    # Equivalent: awk 'BEGIN{print "Starting..."} {print $0}'

14. end_test.go - Finalize with END block

    # Equivalent: awk '{print $0} END{print "Done"}'

15. sum_test.go - Accumulate values using BEGIN/ACTION/END

    # Equivalent: awk 'BEGIN{sum=0} {sum+=$1} END{print "Sum:",sum}'

16. average_test.go - Compute statistics (average)

    # Equivalent: awk '{sum+=$1;count++} END{print sum/count}'

Conditional Processing

Advanced examples using conditions to filter and process data.

17. condition_test.go - Pattern matching with Condition method

    # Equivalent: awk '/ap/'

18. conditionallinenumber_test.go - Filter by line number range

    # Equivalent: awk 'NR>1 && NR<=3'

19. variablethreshold_test.go - Use initialized variables for filtering

    # Equivalent: awk -v threshold=20 '$1>threshold'

20. uniquelines_test.go - Deduplication using state and variables

    # Equivalent: awk '!seen[$0]++'

File-Based Examples

These examples demonstrate reading from files instead of strings.NewReader(), using the testdata/ directory. All the patterns above are also available as file-based examples with the fromfile_ prefix.

• fromfile_printfield_test.go - Print field from file
• fromfile_sum_test.go - Calculate sum from numbers file
• fromfile_average_test.go - Calculate average from file
• fromfile_csv_test.go - Process CSV data from file
• fromfile_condition_test.go - Pattern matching from file
• fromfile_unique_test.go - Remove duplicates from file
• fromfile_linenumbers_test.go - Add line numbers to file
• fromfile_logerrors_test.go - Filter log entries from file
• fromfile_pricethreshold_test.go - Filter by numeric threshold
• fromfile_multifield_test.go - Calculate averages across fields

Test Data Files

The testdata/ directory contains sample files for use in examples and testing:

• simple_fields.txt - Space-separated fields for basic operations
• numbers.txt - One number per line for calculations
• people.csv - CSV data with name,age,city format
• fruits.txt - List of items for pattern matching
• duplicates.txt - Lines with duplicates for deduplication tests
• scores.txt - Student scores for multi-field operations
• log_entries.txt - Log file entries with timestamps and levels
• prices.txt - Product names and prices
• tab_separated.tsv - Tab-separated employee data
• mixed_text.txt - Various text for pattern matching

See testdata/README.md for detailed information about each file.

Running Examples

Run all examples:

go test -v

Run a specific example:

go test -v -run ExampleAwk_printField

Writing Style Guide

Each test file follows this structure:

1. Package declaration: package awk_test
2. Imports: Only import what's needed for that specific example
3. Program type: Define a custom type that embeds SimpleProgram
4. Methods: Implement only the methods needed (Begin/Condition/Action/End)
5. Example function: Named ExampleAwk_<feature> with:
   • Comment showing equivalent traditional awk command
   • Call to yup.MustRun() with the awk program
   • // Output: comment with expected output

Template

package awk_test

import (
	"strings"

	. "github.com/yupsh/awk"
	gloo "github.com/yupsh/framework"
)

// myProgram demonstrates [what it does]
type myProgram struct {
	SimpleProgram
}

func (p myProgram) Action(ctx *Context) (string, bool) {
	// Implementation
	return ctx.Field(0), true
}

// Example using strings.NewReader (for inline data)
func ExampleAwk_myFeature() {
	// echo "input" | awk '{traditional awk command}'
	gloo.MustRun(
		Awk(
			myProgram{},
			strings.NewReader("input"),
		),
	)
	// Output:
	// expected output
}

// Example using file (for testdata files)
func ExampleAwk_fromFile_myFeature() {
	// cat testdata/myfile.txt | awk '{traditional awk command}'
	gloo.MustRun(
		Awk(
			myProgram{},
			gloo.File("testdata/myfile.txt"),
		),
	)
	// Output:
	// expected output
}

Key Patterns

Program Interface

The Program interface has four optional methods:

• Begin(ctx *Context) error - Initialize before processing
• Condition(ctx *Context) bool - Filter which lines to process
• Action(ctx *Context) (string, bool) - Process each line
• End(ctx *Context) (string, error) - Finalize after processing

Embed SimpleProgram to get default implementations, then override only what you need.

Context Access

• ctx.Field(n) - Get field by index (0 = whole line, 1 = first field)
• ctx.NR - Current line number (1-based)
• ctx.NF - Number of fields in current line
• ctx.Var(name) / ctx.SetVar(name, value) - User variables
• ctx.Print(values...) - Format values with OFS separator

Flags

• FieldSeparator(":") - Set input field separator
• OutputFieldSeparator(",") - Set output field separator
• Variable{Name: "x", Value: 10} - Initialize variables

Learning Path

1. Start with Basic Field Operations (1-3) to understand field access
2. Move to Built-in Variables (4-6) to learn about NR and NF
3. Explore Field Separators (7-9) for parsing different formats
4. Practice Text Transformation (10-12) for data manipulation
5. Master Program Flow Control (13-16) for stateful processing
6. Apply Conditional Processing (17-20) for advanced filtering

See Also

• AWK Command Documentation
• yupsh Framework Documentation
• Go Examples Documentation