Manpages - sed.1p

This manual page is part of the POSIX Programmer’s Manual. The Linux implementation of this interface may differ (consult the corresponding Linux manual page for details of Linux behavior), or the interface may not be implemented on Linux.

sed — stream editor

  sed [-n] script [file...]
  sed [-n] -e script [-e script]... [-f script_file]... [file...]
  sed [-n] [-e script]... -f script_file [-f script_file]... [file...]

The sed utility is a stream editor that shall read one or more text files, make editing changes according to a script of editing commands, and write the results to standard output. The script shall be obtained from either the script operand string or a combination of the option-arguments from the -e script and -f script_file options.

The sed utility shall conform to the Base Definitions volume of POSIX.1‐2017, Section 12.2, Utility Syntax Guidelines, except that the order of presentation of the -e and -f options is significant.

The following options shall be supported:

-e script

Add the editing commands specified by the script option-argument to the end of the script of editing commands.

-f script_file

Add the editing commands in the file script_file to the end of the script of editing commands.

-n

Suppress the default output (in which each line, after it is examined for editing, is written to standard output). Only lines explicitly selected for output are written.

If any -e or -f options are specified, the script of editing commands shall initially be empty. The commands specified by each -e or -f option shall be added to the script in the order specified. When each addition is made, if the previous addition (if any) was from a -e option, a <newline> shall be inserted before the new addition. The resulting script shall have the same properties as the script operand, described in the OPERANDS section.

The following operands shall be supported:

file

A pathname of a file whose contents are read and edited. If multiple file operands are specified, the named files shall be read in the order specified and the concatenation shall be edited. If no file operands are specified, the standard input shall be used.

script

A string to be used as the script of editing commands. The application shall not present a script that violates the restrictions of a text file except that the final character need not be a <newline>.

The standard input shall be used if no file operands are specified, and shall be used if a file operand is ’-’ and the implementation treats the ’-’ as meaning standard input. Otherwise, the standard input shall not be used. See the INPUT FILES section.

The input files shall be text files. The /script_file/s named by the -f option shall consist of editing commands.

The following environment variables shall affect the execution of sed:

LANG

Provide a default value for the internationalization variables that are unset or null. (See the Base Definitions volume of POSIX.1‐2017, Section 8.2, Internationalization Variables for the precedence of internationalization variables used to determine the values of locale categories.)

LC_ALL

If set to a non-empty string value, override the values of all the other internationalization variables.

LC_COLLATE

Determine the locale for the behavior of ranges, equivalence classes, and multi-character collating elements within regular expressions.

LC_CTYPE

Determine the locale for the interpretation of sequences of bytes of text data as characters (for example, single-byte as opposed to multi-byte characters in arguments and input files), and the behavior of character classes within regular expressions.

LC_MESSAGES

Determine the locale that should be used to affect the format and contents of diagnostic messages written to standard error.

NLSPATH

Determine the location of message catalogs for the processing of LC_MESSAGES.

Default.

The input files shall be written to standard output, with the editing commands specified in the script applied. If the -n option is specified, only those input lines selected by the script shall be written to standard output.

The standard error shall be used only for diagnostic and warning messages.

The output files shall be text files whose formats are dependent on the editing commands given.

The script shall consist of editing commands of the following form:

    [address[,address]]function

where function represents a single-character command verb from the list in Editing Commands in sed, followed by any applicable arguments.

The command can be preceded by <blank> characters and/or <semicolon> characters. The function can be preceded by <blank> characters. These optional characters shall have no effect.

In default operation, sed cyclically shall append a line of input, less its terminating <newline> character, into the pattern space. Reading from input shall be skipped if a <newline> was in the pattern space prior to a D command ending the previous cycle. The sed utility shall then apply in sequence all commands whose addresses select that pattern space, until a command starts the next cycle or quits. If no commands explicitly started a new cycle, then at the end of the script the pattern space shall be copied to standard output (except when -n is specified) and the pattern space shall be deleted. Whenever the pattern space is written to standard output or a named file, sed shall immediately follow it with a <newline>.

Some of the editing commands use a hold space to save all or part of the pattern space for subsequent retrieval. The pattern and hold spaces shall each be able to hold at least 8 192 bytes.

The following exit values shall be returned:

 0

Successful completion.

>0

An error occurred.

Default.

The following sections are informative.

Regular expressions match entire strings, not just individual lines, but a <newline> is matched by ’\n’ in a sed RE; a <newline> is not allowed by the general definition of regular expression in POSIX.1‐2008. Also note that ’\n’ cannot be used to match a <newline> at the end of an arbitrary input line; <newline> characters appear in the pattern space as a result of the N editing command.

When using sed to process pathnames, it is recommended that LC_ALL, or at least LC_CTYPE and LC_COLLATE, are set to POSIX or C in the environment, since pathnames can contain byte sequences that do not form valid characters in some locales, in which case the utility’s behavior would be undefined. In the POSIX locale each byte is a valid single-byte character, and therefore this problem is avoided.

This sed script simulates the BSD cat -s command, squeezing excess empty lines from standard input.

    sed -n '
    # Write non-empty lines.
    /./ {
        p
        d
        }
    # Write a single empty line, then look for more empty lines.
    /^$/    p
    # Get next line, discard the held <newline> (empty line),
    # and look for more empty lines.
    :Empty
    /^$/    {
        N
        s/.//
        b Empty
        }
    # Write the non-empty line before going back to search
    # for the first in a set of empty lines.
        p
    '

The following sed command is a much simpler method of squeezing empty lines, although it is not quite the same as cat -s since it removes any initial empty lines:

    sed -n '/./,/^$/p'

This volume of POSIX.1‐2017 requires implementations to support at least ten distinct /wfile/s, matching historical practice on many implementations. Implementations are encouraged to support more, but conforming applications should not exceed this limit.

The exit status codes specified here are different from those in System

1. System V returns 2 for garbled sed commands, but returns zero with

its usage message or if the input file could not be opened. The standard developers considered this to be a bug.

The manner in which the l command writes non-printable characters was changed to avoid the historical backspace-overstrike method, and other requirements to achieve unambiguous output were added. See the RATIONALE for /ed / for details of the format chosen, which is the same as that chosen for sed.

This volume of POSIX.1‐2017 requires implementations to provide pattern and hold spaces of at least 8 192 bytes, larger than the 4 000 bytes spaces used by some historical implementations, but less than the 20 480 bytes limit used in an early proposal. Implementations are encouraged to allocate dynamically larger pattern and hold spaces as needed.

The requirements for acceptance of <blank> and <space> characters in command lines has been made more explicit than in early proposals to describe clearly the historical practice and to remove confusion about the phrase ``protect initial blanks [/sic/] and tabs from the stripping that is done on every script line’’ that appears in much of the historical documentation of the sed utility description of text. (Not all implementations are known to have stripped <blank> characters from text lines, although they all have allowed leading <blank> characters preceding the address on a command line.)

The treatment of ’#’ comments differs from the SVID which only allows a comment as the first line of the script, but matches BSD-derived implementations. The comment character is treated as a command, and it has the same properties in terms of being accepted with leading <blank> characters; the BSD implementation has historically supported this.

Early proposals required that a script_file have at least one non-comment line. Some historical implementations have behaved in unexpected ways if this were not the case. The standard developers considered that this was incorrect behavior and that application developers should not have to avoid this feature. A correct implementation of this volume of POSIX.1‐2017 shall permit /script_file/s that consist only of comment lines.

Early proposals indicated that if -e and -f options were intermixed, all -e options were processed before any -f options. This has been changed to process them in the order presented because it matches historical practice and is more intuitive.

The treatment of the p flag to the s command differs between System V and BSD-based systems when the default output is suppressed. In the two examples:

    echo a | sed    's/a/A/p'
    echo a | sed -n 's/a/A/p'

this volume of POSIX.1‐2017, BSD, System V documentation, and the SVID indicate that the first example should write two lines with A, whereas the second should write one. Some System V systems write the A only once in both examples because the p flag is ignored if the -n option is not specified.

This is a case of a diametrical difference between systems that could not be reconciled through the compromise of declaring the behavior to be unspecified. The SVID/BSD/System V documentation behavior was adopted for this volume of POSIX.1‐2017 because:

*

No known documentation for any historic system describes the interaction between the p flag and the -n option.

*

The selected behavior is more correct as there is no technical justification for any interaction between the p flag and the -n option. A relationship between -n and the p flag might imply that they are only used together, but this ignores valid scripts that interrupt the cyclical nature of the processing through the use of the D, d, q, or branching commands. Such scripts rely on the p suffix to write the pattern space because they do not make use of the default output at the ``bottom’’ of the script.

*

Because the -n option makes the p flag unnecessary, any interaction would only be useful if sed scripts were written to run both with and without the -n option. This is believed to be unlikely. It is even more unlikely that programmers have coded the p flag expecting it to be unnecessary. Because the interaction was not documented, the likelihood of a programmer discovering the interaction and depending on it is further decreased.

*

Finally, scripts that break under the specified behavior produce too much output instead of too little, which is easier to diagnose and correct.

The form of the substitute command that uses the n suffix was limited to the first 512 matches in an early proposal. This limit has been removed because there is no reason an editor processing lines of {LINE_MAX} length should have this restriction. The command s/a/A/2047 should be able to substitute the 2 047th occurrence of a on a line.

The b, t, and : commands are documented to ignore leading white space, but no mention is made of trailing white space. Historical implementations of sed assigned different locations to the labels ’x’ and “x ”. This is not useful, and leads to subtle programming errors, but it is historical practice, and changing it could theoretically break working scripts. Implementors are encouraged to provide warning messages about labels that are never referenced by a b or t command, jumps to labels that do not exist, and label arguments that are subject to truncation.

Earlier versions of this standard allowed for implementations with bytes other than eight bits, but this has been modified in this version.

None.

/awk , //ed , //grep /

The Base Definitions volume of POSIX.1‐2017, Table 5-1, Escape Sequences and Associated Actions, Chapter 8, Environment Variables, Section 9.3, Basic Regular Expressions, Section 12.2, Utility Syntax Guidelines

Portions of this text are reprinted and reproduced in electronic form from IEEE Std 1003.1-2017, Standard for Information Technology – Portable Operating System Interface (POSIX), The Open Group Base Specifications Issue 7, 2018 Edition, Copyright (C) 2018 by the Institute of Electrical and Electronics Engineers, Inc and The Open Group. In the event of any discrepancy between this version and the original IEEE and The Open Group Standard, the original IEEE and The Open Group Standard is the referee document. The original Standard can be obtained online at http://www.opengroup.org/unix/online.html .

Any typographical or formatting errors that appear in this page are most likely to have been introduced during the conversion of the source files to man page format. To report such errors, see https://www.kernel.org/doc/man-pages/reporting_bugs.html .