Thoughts on Programming

August 8, 2011

Unix commands: sed & awk

Filed under: Linux — shadiyya @ 5:20 pm

sed

It is a stream editor is used to perform basic text transformations on an input stream. The sed utility is a stream editor that reads one or more text files, makes editing changes according to a script of editing commands, and writes the results to standard output.

The pattern to match is typically included between a pair of slashes // and quoted. For example, to print lines containing the string “1024”, we may use:

cat filename | sed -n ‘/1024/p’

Here, sed filters the output from the cat command. The option “-n” tells sed to block all the incoming lines but those explicitly matching the expression.
Here is another example, this time for deleting selected lines:

cat filename | sed ‘/.*o$/d’ > new_file

In this example, lines ending with an “o” will be deleted. It uses a regular expression for matching any string followed by an “o” and the end of the line. The output (i.e., all lines but those ending with “o”) is directed to new_file.

To search and replace, use the sed ‘s’ action, which comes in front of two expressions:

sed ‘s/string_old/string_new/’ filename > newfile

awk

The awk is mostly used for pattern scanning and processing. It searches one or more files to see if they contain lines that matches with the specified patterns and then perform associated actions. Awk breaks each line of input passed to it into fields. By default, a field is a string of consecutive characters delimited by whitespace, though there are options for changing this.

Awk runs through a text file by reading and processing one record at time. Its commands are written with the intention that they act repetitively on each record as it is read in to awk. A record that has been read by awk is broken into separate fields, and actions can be performed on the separate fields as well as on the whole record.

As awk processes each line of the input file, each word on the line is assigned to variables named $1 (the first word), $2 (the second word), and so on.

Let’s start with a file, words.txt, that contains these lines:

nail hammer wood
pedal foot car
clown pie circus

Now we’ll use the print function in awk to plug the words from each input line into a template, like this:

awk ‘{print “Hit the”,$1,”with your”,$2}’ words.txt


Then the output will be:

Hit the nail with your hammer
Hit the pedal with your foot
Hit the clown with your pie

Advertisements

Leave a Comment »

No comments yet.

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: