xref: /inferno-os/man/1/alphabet-grid (revision 46439007cf417cbd9ac8049bb4122c890097a0fa)
ALPHABET-GRID 1
NAME
grid - peer-to-peer data distribution
SYNOPSIS
.EX load alphabet typeset /grid type /grid/endpoint
DESCRIPTION
Grid is a typeset for alphabet (see sh-alphabet (1)) which enables allows direct interconnection of remote and local processing components. It defines one new type, endpoint , which represents a place in the network to which two parties can connect and exchange data.

In the following descriptions, if a type is not endpoint or a type defined in the root typeset (see alphabet-main (1)), it is assumed to be of type /string .

Modules currently provided within the /grid typeset include:

10 \f5farm [\f5-lnkavA] endpoint addr tasktype [arg...] -> endpoint Farm connects to a grid labour exchange (see scheduler (4)) at addr , starts a new job of type workflow , and passes all the data read from endpoint to be processed by the currently available labour. The data is split into records, each one of which will be processed on a worker node by tasktype , with its associated arg uments. Other than -A , which specifies unauthenticated access to the scheduler, the various options are all passed verbatim to workflow : -l causes it to split its input on newline-separated records; -n specifies that no record separation is necessary on output; -k specifies that intermediate data for failed tasks should be kept around; -a specifies that intermediate data for all tasks should be kept around, and -v specifies that workflow should produce a wordy description of what it is doing.

\f5local endpoint -> /fd Local reads everything from endpoint , and writes it to fd .

\f5remote [-a addr] fd -> endpoint Remote is the inverse of local : it reads data from fd and writes it to a newly created endpoint. If -a is given, addr specifies the network address of an endpoint server on which to create the new endpoint.

\f5rexec [\f5-A] endpoint addr cmd -> endpoint Rexec connects to a remote execution server at addr (unauthenticated if -A is specified), and arranges to execute the alphabet expression cmd there. The expression should be compatible with usage "fd -> fd" . Data from the argument endpoint will be piped through this expression, and made available as the resulting endpoint endpoint.

EXAMPLES
The examples below that a local endpoint is available, and the following alphabet declarations: .EX load alphabet typeset /grid type /string /endpoint /fd import /grid/local /grid/remote autoconvert fd endpoint remote Set up a rendering pipeline: .EX -{/read /tmp/somedata | remote | rexec tcp!node1!rexec "{(/fd); /filter $1 "{os render_stage1}} | rexec tcp!node2!rexec "{(/fd); /filter $1 "{os render_stage2}} | /create /tmp/somedata.result }
SOURCE
/appl/alphabet/grid.b , /appl/alphabet/gridtypes.b

/appl/cmd/grid/*.b

SEE ALSO
sh-alphabet (1), alphabet-main (1), sh (1)