This module provides a mechanism for composing and running Unix shell pipelines. It defines a typed algebraic term language using operators for piping and redirections while checking that the type of data passing through the standard input and output streams of each subprocess match with those of connected processes. The language is only capable of describing simple, linear pipelines, where each process can have one or zero input streams and one or zero output streams. The type of a process is denoted by a term X>>Y, where X and Y are stream types and can be 0 for no stream, or $T for a stream of type T, where T is an arbitrary term describing what sort of data is in the stream, eg, plain text or XML. The typing judgements are as follows:

P >> Q :: X>>Z :- P :: X>>Y,    Q::Y>>Z.
F :> Q :: 0>>Z :- F :: file(Y), Q::Y>>Z.
P >: F :: X>>0 :- F :: file(Y), P::X>>Y.
P * Q  :: T    :- P :: T1, Q :: T2, seq_types(T1,T2,T).
P + Q  :: T    :- P :: T1, Q :: T2, par_types(T1,T2,T).
P :: T :- swipe:def(P,Q), Q :: T.
sh(T,Fmt,Args) :: T.
sh(T,Cmd) :: T.

in(D,P) :: T   :- P::T. % execute P in directory D

Filename^T :: file(T).

The rules for combining types with the * operator (shell &&, sequential execution) and + operator (shell &, concurrent execution) are encoded in the predicates seq_types and par_types. The rules for sequential excution are:

1. A process with no input (output) (type 0) can combine with a process with any input (output) type, and the compound inherits that input (output) type.
2. If both processes have nonzero input (output) types, then those types must unify, and the compound inherits that output type.

The rules for concurrent execution are

1. A process with no input (output) (type 0) can combine with a process with any input (output) type, and the compound inherits that input (output) type.
2. If both processes have nonzero input types, then they cannot be run concurrently.
3. If both processes have nonzero output types, then those types must unify, and the compound inherits that output type.

If the type requirements are not met, then the system throws a helpful type_mismatch exception.

The primitive processes are expressed as shell commands. A term sh(T,Cmd), where T is an explicitly given type, corresponds to a shell command Cmd, written, including arguments, as you would type it into the Unix shell. Arguments can be handling using the form sh(T,Fmt,Args), where Fmt is a format string as used by format/2, and Args is a list of arguments of type:

shell_args ---> spec+access % A file spec and access mode, format with ~s
              ; @ground     % any term, is written and escaped, format with ~s
              ; \_.         % Any other kind of argument, passed through
              ; T<(_>>T)    % bash process redirection with pipeline of output type T
              ; T>(T>>_)    % bash process redirection with pipeline of input type T
access ---> read ; write ; append ; execute.

In process redirection, a command expecting to read to or write from a named file can be redirected to a bash pipeline. In this case, one end of the pipeline is attached to the command, but the other end is left free. The input/output type of that free end interacts with the type of overall command being constructed in the same way as parallel processes interact.

File names

File names should passed to sh/3 as Spec+Access. If Spec is atomic, it is treated as an explicit absolute or relative path in the file system and formatted quoted and escaped so that any special characters in the path are properly handled.

If Spec is a compound term, the system uses absolute_file_name/3 with the access(Access) option to expand Spec. This must succeed exactly once, otherwise an exception is thrown. The resulting path is quoted and escaped.

In both cases, the result is captured by '~s' in the format string. There is a subtlety in the handling of compound file specifier terms: the file must exist with the correct access at pipeline composition time---if the file is only created when the pipeline is run, then the path expansion will fail. In these cases, you must use an atomic file specifier, or the (@)/1 operator. This also applies to files used with the redirection operators (:>)/2 and (>:)/2.

Declaring new processes

New compound pipelines can be declared using the multifile predicate def/2. The commands cat/0, cat/1 and echo/1 are already defined.

cat       :: $T >> $T. % any stream type to the same stream type
cat(F^T)  :: 0 >> $T.   % output contents of file F
echo(S^T) :: 0 >> $T.   % output literal text S as type T

Running

A pipeline expression can be used in one of three ways:

1. With command/{2,3}, which produce a string which can be passed to shell/1 or used with open(pipe(Cmd), ...).
2. With run/1, which calls the formatted command directly using shell/1.
3. Using with_pipe_output/3 or with_pipe_output/3, which runs the pipeline concurrently with the current thread, making either its standard input or standard output available on a Prolog stream.

To be done

-

• Use of parenthesis for grouping might not work in some cases
• Decide on best quoting/escaping mechanism

 command(Pipe:X>>Y, -Type:pipe_type, Cmd:string) is det

 command(Pipe:X>>Y, Cmd:string) is det

Formats the shell command for a pipeline expression. Three argument version unifies Type with the inferred type of the pipeline. Throws an exception if the types do not unify.

 run(Pipe:X>>Y) is det

Runs a pipeline. Standard input and output of the process are inherited directly from Prolog process.

Undocumented predicates

The following predicates are exported, but not or incorrectly documented.

 command(Arg1, Arg2)

 with_pipe_output(Arg1, Arg2, Arg3)

 with_pipe_input(Arg1, Arg2, Arg3)

 with_pipe_io(Arg1, Arg2, Arg3)

 shell_quote(Arg1, Arg2, Arg3)