Using the ‘sed’ Command in Bash/Linux, With Examples

The sed (Stream EDitor) command in Bash/Linux reads text from a stream or file and performs line-by-line operations on it based on a set of supplied criteria.  Here’s how to use it.

sed is an automated way to process text.

One simple example of where sed can be used is doing a simple find-and-replace for a word in a text document – the text with the words replaced can then be saved to a new file or overwrite the original.

Why?

Being able to modify text – either from a file or as is it output from an application – is useful. Log files are often jam-packed full of entries that you don’t want to see when you’re chasing an issue – sed is one way you can filter the output from a giant log file to only include the entries you are looking for.

sed Syntax

sed is run from the Linux command line/shell and has the following syntax:

sed [OPTIONS] COMMANDS [INPUT-FILE]

Note that:

• OPTIONS will be a list of options from the below table
• COMMANDS are a list of commands/operations from the below table, provided as a string
  • These commands are what you use to tell sed what to do with the incoming text
  • These commands can also be defined in a file using the -f option
• INPUT-FILE is the path to the text file you want to perform the operations on
  • It is optional as sed can read from the text being piped to it as a stream
• The processed text is output as standard output – which can continue to be used as a stream or can be sent to a file
  • Standard output (also known as stdout) is the output of a command that is usually printed to the shell/console
  • Standard output can also be redirected into another command/application or to a file, rather than being printed

Options

Here is a list of the options available for sed, straight from the user manual:

Commands

There are a lot of sed commands, so I won’t reproduce them all here. The link below outlines the commands available, while the examples below will cover common usage:

https://www.gnu.org/software/sed/manual/html_node/sed-commands-list.html

You can view the full user manual for the sed command on your system by running:

man sed

Example File

The examples in this article will refer to a plain-text file called test.txt which will contain the following:

test.txt

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed feugiat felis quis convallis tempor. 
Suspendisse potenti. Integer cursus pulvinar efficitur. 
Nullam at orci laoreet, scelerisque lacus in, mattis mauris. Aenean laoreet tortor vel arcu luctus, sed viverra sapien ornare. Nunc quis lacus vitae magna tristique lobortis venenatis in risus. Praesent tempor dui sit amet metus consectetur aliquet.

Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. 
In pharetra pulvinar lectus, quis tincidunt sem ultricies sit amet. Mauris id felis sem. 
Cras lorem ex, pretium ac dapibus non, vestibulum eget nulla. Proin condimentum, quam sit amet ultricies consectetur, est justo accumsan nulla, sit amet semper ex lectus vel mauris. 
Vivamus eget nibh quis mauris porta consequat ut sed ligula. Pellentesque augue est, suscipit sit amet sodales nec, consectetur non ante. 
Suspendisse ligula ante, finibus eget hendrerit nec, varius a ante. 
Aenean feugiat ipsum lectus, eu maximus risus viverra in. Praesent aliquam nunc neque, id sagittis lectus posuere at. 
Donec viverra feugiat odio molestie fermentum. Nullam id ligula vitae nibh ornare ullamcorper. Vivamus vel pellentesque lacus. 
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec urna risus, commodo a tempus sed, dapibus sed mi.

To simply output the contents of this file using sed:

sed '' test.txt

No processing operations are done on the text as no commands are supplied (as the commands string is empty).

Pattern Matching

sed uses Regular Expressions (regex) to perform pattern matching in the text – allowing you to define the pattern used to find elements in the input text.

Regular Expressions are a topic on their own, so they won’t be covered in depth here. Regex is also notoriously finicky and annoying to write. The below examples will cover some basic usage, but there’s no shame in using an online regex generator to build your search patterns rather than trying to craft your own.

Finding Text Using the sed Command

The p command prints out matching lines.

The below example uses the regex /Lorem/ to find text containing the case-sensitive text Lorem, followed by the p command to print matching lines:

sed -n '/Lorem/p' test.txt

The -n option is being used here to disable the input text’s automatic output so that only matching lines are output.

The output of the above command is only the two lines containing the string Lorem (case sensitive):

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed feugiat felis quis convallis tempor. 
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec urna risus, commodo a tempus sed, dapibus sed mi.

Address Ranges

Address ranges are defined by simply entering the line number or range.

Print the 3rd line of the file:

sed -n '3p' test.txt

Print lines 3-6 – commas are used to define ranges:

sed -n '3,6p' test.text

You can also do things like print lines at a certain interval. This example prints only every 5th line, starting at line 1 using the ~, which sets the interval for lines to match.

sed -n '1~5p' test.txt

sed is a pretty versatile tool; you can chain commands and perform some complex operations on your text if you choose. Additional commands for working with ranges and replacement are available in the user manual.

Replacing Text Using the sed Command

This example replaces ‘sit‘ with ‘foo‘ in the example text:

sed 's/sit/foo/g' test.txt

This command contains:

• The s command, which instructs sed to substitute
• Followed by the text pattern to match (sit)
• Followed by the text pattern to replace (foo)
• The /g command tells sed that it is global – text should be replaced in the entire line.
  • Without this, only the first match in each line is replaced

Removing Text Using the sed Command

The d command deletes text from the matching pattern.

Here it is used to delete a range – the first 12 lines of test.txt

sed '1,11d' test.txt

This will output only the text file’s last line (as it has 12 lines total).

Here the delete command is used to delete text lines containing a matching a pattern – it searches for the word sit:

sed  '/sit/d' test.txt

Only lines which do not contain sit will be output.

You can also remove the text between two patterns:

sed '/Cras lorem ex,vestibulum eget nulla/d' test.txt

This will output the full text, with only the text between ‘Cras lorem ex‘ and ‘vestibulum eget nulla‘ removed.

If you wanted to keep the lines containing the word and only remove the word itself:

sed 's/sit//g' test.txt

This command contains:

• The s command, which instructs sed to substitute
• Followed by the text pattern to match (sit)
• Followed by the text pattern to replace (in this case NOTHING – hence two adjacent forward slashes with nothing between them)
• The /g command tells sed that it is global – text should be replaced in the entire line.
  • Without this, only the first match in each line is replaced

Combining sed Commands – Range and Substitution

This example replaces sit with foo only on lines 11-12:

sed '11,12s/sit/foo/g' test.txt

The full contents of the example file, with the substitutions made to lines 11-12, will be output

Pipe/Stream Input to the sed Command

You can pipe input to the sed command using the | (pipe) command:

echo "Lorem ipsum dolor" | sed ''

The echo command output is piped into the sed command using the | (pipe) command – no source text file is specified as the input text is read from the piped input.

Using the Output of the sed Command

Writing a File

The output of the sed command can be sent to a file by using the > command:

sed -n '' test.txt > newfile.txt

If the output file name is the same as an existing file (including the input file), it will be overwritten.

Overwriting the Input File

If an input file is specified, sed can be instructed to overwrite it with its processed output using the -i option:

sed -i -n '' test.txt

This will overwrite the input file, and NO BACKUP WILL BE MADE OF THE ORIGINAL – the changes will not be undo-able, so please be careful.

Pipe/Stream Output

Just as you can pipe input into sed, you can pipe its output to another command.

echo "Lorem ipsum dolor" | sed '' | cat

As in the pipe/stream example above, the output from the echo command is piped to sed – and then here, the output of sed is piped to the cat command, which simply prints it again. The cat command here, of course, could be any other command which accepts input in this manner so that the text can be further processed or transmitted.

sed itself is not a complex command. Regex is the source of the complexity, and it’s impossible to cover every usage scenario – every situation is going to be unique.

I hope this article helps get you started with the sed bit. As for regex…

You are definitely going to find yourself doing a lot of Googling to build the specific expressions you need.