Manpages - libpipeline.3
is a C library for setting up and running pipelines of processes, without needing to involve shell command-line parsing which is often error-prone and insecure. This relieves programmers of the need to laboriously construct pipelines using lower-level primitives such as
and
The general way to use
involves constructing a
structure and adding one or more
structures to it. A
represents a subprocess (or
while a
represents a sequence of subprocesses each of whose outputs is connected to the next one’s input, as in the example
The calling program may adjust certain properties of each command independently, such as its environment and
priority, as well as properties of the entire pipeline such as its input and output and the way signals are handled while executing it. The calling program may then start the pipeline, read output from it, wait for it to complete, and gather its exit status.
Strings passed as
function arguments will be copied by the library.
Construct a new command representing execution of a program called
Convenience constructors wrapping
and
Construct a new command representing execution of a program called
with arguments. Terminate arguments with
Split
on whitespace to construct a command and arguments, honouring shell-style single-quoting, double-quoting, and backslashes, but not other shell evilness like wildcards, semicolons, or backquotes. This is included only to support situations where command arguments are encoded into configuration files and the like. While it is safer than
it still involves significant string parsing which is inherently riskier than avoiding it altogether. Please try to avoid using it in new code.
Construct a new command that calls a given function rather than executing a process.
The data argument is passed as the function’s only argument, and will be freed before returning using free_func (if
functions that deal with arguments cannot be used with the command returned by this function.
Construct a new command that itself runs a sequence of commands, supplied as
arguments following
and terminated by
The commands will be executed in forked children; if any exits non-zero then it will terminate the sequence, as with “&&” in shell.
functions that deal with arguments cannot be used with the command returned by this function.
Return a new command that just passes data from its input to its output.
Return a duplicate of a command.
Add an argument to a command.
Convenience function to add an argument with printf substitutions.
Convenience functions wrapping
to add multiple arguments at once. Terminate arguments with
Split
on whitespace to add a list of arguments, honouring shell-style single-quoting, double-quoting, and backslashes, but not other shell evilness like wildcards, semicolons, or backquotes. This is included only to support situations where command arguments are encoded into configuration files and the like. While it is safer than
it still involves significant string parsing which is inherently riskier than avoiding it altogether. Please try to avoid using it in new code.
Return the number of arguments to this command. Note that this includes the command name as the first argument, so the command
is counted as having three arguments.
Set the
value for this command. Defaults to 0. Errors while attempting to set the nice value are ignored, aside from emitting a debug message.
If
is non-zero, redirect this command’s standard error to
Otherwise, and by default, pass it through. This is usually a bad idea.
Change the working directory to
while running this command.
Change the working directory to the directory given by the open file descriptor
while running this command.
Set environment variable
to
while running this command.
Unset environment variable
while running this command.
Clear the environment while running this command. (Note that environment operations work in sequence; pipecmd_clearenv followed by pipecmd_setenv causes the command to have just a single environment variable set.) Beware that this may cause unexpected failures, for example if some of the contents of the environment are necessary to execute programs at all (say,
Install a pre-exec handler. This will be run immediately before executing the command’s payload (process or function). Pass NULL to clear any existing pre-exec handler. The data argument is passed as the function’s only argument, and will be freed before returning using free_func (if non-NULL).
This is similar to pipeline_install_post_fork, except that is specific to a single command rather than installing a global handler, and it runs slightly later (immediately before exec rather than immediately after fork).
Add a command to a sequence created using
Dump a string representation of a command to stream.
Return a string representation of a command. The caller should free the result.
Execute a single command, replacing the current process. Never returns, instead exiting non-zero on failure.
Destroy a command. Safely does nothing if
is
Construct a new pipeline.
Convenience constructors wrapping
and
Construct a new pipeline consisting of the given list of commands. Terminate commands with
Construct a new pipeline and add a single command to it.
Joins two pipelines, neither of which are allowed to be started. Discards
and
from
and
and
from
Connect the input of one or more sink pipelines to the output of a source pipeline. The source pipeline may be started, but in that case
must have been called with a negative
otherwise, calls
In any event, calls
on all sinks, none of which are allowed to be started. Terminate arguments with
This is an application-level connection; data may be intercepted between the pipelines by the program before calling
which sets data flowing from the source to the sinks. It is primarily useful when more than one sink pipeline is involved, in which case the pipelines cannot simply be concatenated into one.
The result is similar to
except that output can be sent to more than two places and can easily be sent to multiple processes.
Add a command to a pipeline.
Construct a new command and add it to a pipeline in one go.
Construct a new command from a shell-quoted string and add it to a pipeline in one go. See the comment against
above if you’re tempted to use this function.
Convenience functions wrapping
to add multiple commands at once. Terminate arguments with
Set file descriptors to use as the input and output of the whole pipeline. If non-negative,
is used directly as a file descriptor. If negative,
will create pipes and store the input writing half and the output reading half in the pipeline’s
or
field as appropriate. The default is to leave input and output as stdin and stdout unless
or
respectively has been called.
Calling these functions supersedes any previous call to
or
respectively.
Set file names to open and use as the input and output of the whole pipeline. This may be more convenient than supplying file descriptors, and guarantees that the files are opened with the same privileges under which the pipeline is run.
Calling these functions (even with
which returns to the default of leaving input and output as stdin and stdout) supersedes any previous call to
or
respectively.
The given files will be opened when the pipeline is started. If an output file does not already exist, it is created (with mode 0666 modified in the usual way by umask); if it does exist, then it is truncated.
If
is non-zero, ignore
and
in the calling process while the pipeline is running, like
Otherwise, and by default, leave their dispositions unchanged.
Return the number of commands in this pipeline.
Return command number
from this pipeline, counting from zero, or
if
is out of range.
Set command number
in this pipeline, counting from zero, to
and return the previous command in that position. Do nothing and return
if
is out of range.
Return the process ID of command number
from this pipeline, counting from zero. The pipeline must be started. Return
if
is out of range or if the command has already exited and been reaped.
Get streams corresponding to
and
respectively. The pipeline must be started.
Dump a string representation of
to stream.
Return a string representation of
The caller should free the result.
Destroy a pipeline and all its commands. Safely does nothing if
is
May wait for the pipeline to complete if it has not already done so.
Install a post-fork handler. This will be run in any child process immediately after it is forked. For instance, this may be used for cleaning up application-specific signal handlers. Pass
to clear any existing post-fork handler.
See pipecmd_pre_exec for a similar facility limited to a single command rather than global to the calling process.
Start the processes in a pipeline. Installs this library’s
handler if not already installed. Calls
on error.
The standard file descriptors (0, 1, and 2) must be open before calling this function.
Wait for a pipeline to complete. Set
to a newly-allocated array of wait statuses, as returned by
and
to the length of that array. The return value is similar to the exit status that a shell would return, with some modifications. If the last command exits with a signal (other than
which is considered equivalent to exiting zero), then the return value is 128 plus the signal number; if the last command exits normally but non-zero, then the return value is its exit status; if any other command exits non-zero, then the return value is 127; otherwise, the return value is 0. This means that the return value is only 0 if all commands in the pipeline exit successfully.
Wait for a pipeline to complete and return its combined exit status, calculated as for
Start a pipeline, wait for it to complete, and free it, all in one go.
Pump data among one or more pipelines connected using
until all source pipelines have reached end-of-file and all data has been written to all sinks (or failed). All relevant pipelines must be supplied: that is, no pipeline that has been connected to a source pipeline may be supplied unless that source pipeline is also supplied. Automatically starts all pipelines if they are not already started, but does not wait for them. Terminate arguments with
In general, output is returned as a pointer into a buffer owned by the pipeline, which is automatically freed when
is called. This saves the caller from having to explicitly free individual blocks of output data.
Read
bytes of data from the pipeline, returning the data block.
is updated with the number of bytes read.
Look ahead in the pipeline’s output for
bytes of data, returning the data block.
is updated with the number of bytes read. The starting position of the next read or peek is not affected by this call.
Return the number of bytes of data that can be read using
or
solely from the peek cache, without having to read from the pipeline itself (and thus potentially block).
Skip over and discard
bytes of data from the peek cache. Asserts that enough data is available to skip, so you may want to check using
first.
Read a line of data from the pipeline, returning it.
Look ahead in the pipeline’s output for a line of data, returning it. The starting position of the next read or peek is not affected by this call.
installs a signal handler for
and collects the exit status of child processes in
Applications using this library must either refrain from changing the disposition of
(in other words, must rely on
for all child process handling) or else must make sure to restore
handler before calling any of its functions.
If the
flag is set in a pipeline (which is the default), then the
and
signals will be ignored in the parent process while child processes are running. This mirrors the behaviour of
leaves child processes with the default disposition of
namely to terminate the process. It ignores
in the parent process while running
installs a
handler that will attempt to reap child processes which have exited. This calls
with
so it will reap any child process, not merely those created by way of this library. At present, this means that if the calling program forks other child processes which may exit while a pipeline is running, the program is not guaranteed to be able to collect exit statuses of those processes.
You should not rely on this behaviour, and in future it may be modified either to reap only child processes created by this library or to provide a way to return foreign statuses to the application. Please contact the author if you have an example application and would like to help design such an interface.
If the
environment variable is set to
then
will emit debugging messages on standard error.
If the
environment variable is set to any value, then
will refrain from printing an error message when a subprocess is terminated by a signal.
In the following examples, function names starting with
or
are real
functions, while any other function names are pseudocode.
The simplest case is simple. To run a single command, such as
pipeline *p = pipeline_new_command_args (“mv”, source, dest, NULL); int status = pipeline_run (p);
is often used to mimic shell pipelines, such as the following example:
The code to construct this would be:
pipeline *p; int status;
p = pipeline_new (); pipeline_want_infile (p, “input-file”); pipeline_command_args (p, “zsoelim”, NULL); pipeline_command_args (p, “tbl”, NULL); pipeline_command_args (p, “nroff”, “-mandoc”, “-Tutf8”, NULL); status = pipeline_run (p);
You might want to construct a command more dynamically:
pipecmd *manconv = pipecmd_new_args (“manconv”, “-f”, from_code, “-t”, “UTF-8”, NULL); if (quiet) pipecmd_arg (manconv, “-q”); pipeline_command (p, manconv);
Perhaps you want an environment variable set only while running a certain command:
pipecmd *less = pipecmd_new (“less”); pipecmd_setenv (less, “LESSCHARSET”, lesscharset);
You might find yourself needing to pass the output of one pipeline to several other pipelines, in a
arrangement:
pipeline *source, *sink1, *sink2;
source = make_source (); sink1 = make_sink1 (); sink2 = make_sink2 (); pipeline_connect (source, sink1, sink2, NULL); * Pump data among these pipelines until there’s nothing left. * pipeline_pump (source, sink1, sink2, NULL); pipeline_free (sink2); pipeline_free (sink1); pipeline_free (source);
Maybe one of your commands is actually an in-process function, rather than an external program:
pipecmd *inproc = pipecmd_new_function (“in-process”, &func, NULL, NULL); pipeline_command (p, inproc);
Sometimes your program needs to consume the output of a pipeline, rather than sending it all to some other subprocess:
pipeline *p = make_pipeline (); const char *line;
pipeline_want_out (p, -1); pipeline_start (p); line = pipeline_peekline (p); if (!strstr (line, “coding: UTF-8”)) printf (“Unicode text follows:); while (line = pipeline_readline (p)) printf (” %s“, line); pipeline_free (p);
Most of
was written by
originally for use in man-db. The initial version was based very loosely on the
function in GNU groff, written by
It also contains library code by
and by various contributors to Gnulib.
is licensed under the GNU General Public License, version 3 or later. See the README file for full details.
Using this library in a program which runs any other child processes and/or installs its own
handler is unlikely to work.