123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309 |
- @c -*-texinfo-*-
- @c This file is part of Guile-SSH Reference Manual.
- @c Copyright (C) 2015, 2016 Artyom V. Poptsov
- @c See the file guile-ssh.texi for copying conditions.
- @node Distributed Forms
- @section Distributed Forms
- @cindex secure distributed computing
- The @code{(ssh dist)} module provides the spirit of distributed computing for
- Guile.
- To make use of the procedures listed in this section you will need an SSH
- daemon and a GNU Guile REPL server both running on the remote host.
- Also note that currently there may be cases in which distributed procedures
- may fail to (de)serialise data; namely @code{make-vector} is one of such
- procedures which output may be troublesome for @code{with-ssh}. To overcome
- this specific case one could pass the @code{fill} argument to
- @code{make-vector} to fill the newly created vector with the specified filling
- instead of @code{#<unspecified>}.
- Node management procedures:
- @deffn {Scheme Procedure} make-node session [repl-port=37146] [#:start-repl-server?=#t] [#:stop-repl-server?=#f]
- Make a new node that uses an SSH @var{session} to connect to a @var{repl-port}
- number on the remote side. Return a new node.
- If @var{start-repl-server?} is set to @code{#t} (which is by default) then
- start a REPL server on a remote host automatically in case when it is not
- started yet.
- If @var{stop-repl-server?} is set to #t then a REPL server will be stopped as
- soon as an evaluation is done. Alternatively you could use
- @code{node-stop-server} procedure from @code{(ssh dist node)} to stop the
- server when it is not needed anymore.
- Note that when either @var{start-repl-server?} or @var{stop-repl-server?} are
- specified, Guile-SSH tries to use @command{pgrep}/@command{pkill} from
- @url{https://gitlab.com/procps-ng/procps, procps} package first to start/stop
- a remote Guile process, but if the package is missing then the Guile-SSH falls
- back to its own implementation of the commands (for more details see
- @pxref{Shell, Guile-SSH shell API})
- @end deffn
- @deffn {Scheme Procedure} node? x
- Return @code{#t} if @var{x} is a node object, @code{#f} otherwise.
- @end deffn
- @deffn {Scheme Procedure} node-session node
- Get underlying SSH session from @var{node}.
- @end deffn
- @deffn {Scheme Procedure} node-repl-port node
- Get REPL port number from a @var{node}.
- @end deffn
- @deffn {Scheme Procedure} node-loadavg node
- Get average load of a @var{node}. Return multiple values. The 1st value is
- an alist of five elements as described in proc(5) man page. The rest of
- values are as described in documentation for @code{node-eval} procedure.
- For example:
- @lisp
- (use-modules (ssh auth)
- (ssh session)
- (ssh dist node))
- (let ((s (make-session #:host "example.org")))
- (connect! s)
- (userauth-agent! s)
- (let ((n (make-node s)))
- (node-loadavg n)))
- @result{} ((one . 0.15) (five . 0.14) (fifteen . 0.16) (scheduling-entities 1 189) (last-pid . 15629))
- @result{} 1
- @result{} "(guile-user)"
- @result{} "scheme"
- @end lisp
- @end deffn
- Interaction with remote REPLs:
- @deffn {Scheme Procedure} distribute nodes expr ...
- Evaluate each @var{expr} in parallel, using distributed computation. Split
- the job to nearly equal parts and hand out each of resulting sub-jobs to
- @var{nodes} list. Return the results of N expressions as a set of N multiple
- values (@pxref{Multiple Values,,, guile, The GNU Guile Reference Manual}).
- @end deffn
- @deffn {Scheme Procedure} dist-map nodes proc lst
- Do list mapping using distributed computation. Split the work into nearly
- equal parts and hand out the resulting jobs to @var{nodes} list. Return the
- result of computation.
- If for some reason a job could not be executed on a node (for example, if
- connection to a remote REPL fails), @code{dist-map} transfers the job to
- another node from the @var{nodes} list. When job execution failed on all
- nodes, an error is reported.
- In a case when an error that occured during job execution is considered
- non-recoverable (eg. when evaluation of @var{proc} on a node failed due to an
- unbound variable) then execution of a job stops immediately.
- @end deffn
- @deffn {Scheme Procedure} with-ssh node exp ...
- Evaluate expressions on a remote REPL using a @var{node}, return four values:
- an evaluation result, a number of the evaluation, a module name and a language
- name. Throw @code{node-error} or @code{node-repl-error} on an error.
- Example:
- @lisp
- (use-modules (ssh session)
- (ssh auth)
- (ssh dist))
- (let ((session (make-session #:user "alice" #:host "www.example.org")))
- (connect! session)
- (userauth-agent! session)
- (display (with-ssh (make-node session)
- (gethostname)))
- (newline))
- @end lisp
- If an expression is evaluated to multiple values then the 1st value returned
- by @code{with-ssh} will be a vector of the evaluated values and the 2nd value
- will be a vector of evaluation numbers. In this case the 2nd value can be
- used to check whether @code{with-ssh} body evaluated to multiple values or
- not. For example:
- @lisp
- (use-modules (ssh session)
- (ssh auth)
- (ssh dist))
- (let ((session (make-session #:user "alice" #:host "www.example.org")))
- (connect! session)
- (userauth-agent! session)
- (with-ssh (make-node session)
- (values 1 2)))
- => #(1 2)
- => #(39 40)
- => "(guile-user)"
- => "scheme"
- @end lisp
- @end deffn
- @deffn {Scheme Procedure} rrepl node
- Start an interactive remote REPL (RREPL) session using @var{node}.
- @end deffn
- @subsection Low-level API
- @subsubsection Nodes
- @tindex node
- The module @code{(ssh dist node)} provides low-level API for node management.
- Here's the description of the format of node type printed representation:
- @example
- #<node avp@@127.0.0.1:22/37146 a0dbdc0>
- A A A A A
- | | | | |
- ,---' | ,-' '---. '-----------.
- | | | | |
- user host port REPL port object address
- @end example
- There are two types of node errors: recoverable and non-recoverable. The
- first group is represented by @code{node-error} exceptions. If an exception
- of this kind is occured then there is a chance that a job can be executed on
- another node. That's because such an exception occures in cases when a node
- is unreachable, for example. The second group is represented by
- @code{node-repl-error} exceptions. Such exceptions mean that an error is
- occured during execution of a job on a node's REPL -- eg. due to the malformed
- job. Those errors are non-recoverable because if the job is broken it will
- likely fail on another nodes as well.
- In addition to @code{make-node}, @code{node?}, @code{node-session} and
- @code{node-repl-port} the module provides:
- @deffn {Scheme Procedure} node-eval node quoted-exp
- Evaluate a @var{quoted-exp} on a @var{node} and return four values: an
- evaluation result, a number of the evaluation, a module name and a language
- name. Throw @code{node-repl-error} if a non-recoverable error occured, or
- @code{node-error} if the evaluation potentially could be succesfully evaluated
- on another node.
- @strong{Note} that @url{https://gitlab.com/procps-ng/procps, procps} version
- 3.3.12 or later is needed on the server side in case of either
- @code{start-repl-server?} or @code{stop-repl-server?} was set to @code{#t} for
- a @var{NODE} (see the documentation for @code{make-node}.)
- @end deffn
- @deffn {Scheme Procedure} node-open-rrepl node
- Open a remote REPL (RREPL). Return a new RREPL channel.
- @end deffn
- @deffn {Scheme Procedure} node-run-server node
- Run a REPL server on a @var{node}.
- @end deffn
- @deffn {Scheme Procedure} node-stop-server node
- Stop a RREPL server on a @var{node}.
- @end deffn
- @deffn {Scheme Procedure} node-guile-version node
- Get Guile version installed on a @var{node}, return the version string.
- Return @code{#f} if Guile is not installed.
- @end deffn
- @deffn {Scheme Procedure} node-server-running? node
- Check if a REPL server is running on a @var{node}, return @code{#t} if it is
- running and listens on an expected port, return @code{#f} otherwise.
- @end deffn
- @deffn {Scheme Procedure} rrepl-eval rrepl-channel expr
- Evaluate expression @var{expr} using @var{rrepl-channel}, return four values:
- an evaluation result, a number of the evaluation, a module name and a language
- name. Throw @code{node-repl-error} on an error.
- @end deffn
- @deffn {Scheme Procedure} rrepl-skip-to-prompt rrepl-channel
- Read from @var{rrepl-channel} until REPL is observed. Throw @code{node-error}
- on an error.
- @end deffn
- @subsubsection Jobs
- @tindex job
- The module @code{(ssh dist job)} provides low-level API for job management.
- Here's the description of the format of node type printed representation:
- @example
- #<job map #<node avp@@127.0.0.1:22/37147 a0dbdc0> a1345a0>
- A A A
- | | |
- | '----------. |
- | | |
- job type node (see above) job object address
- @end example
- @deffn {Scheme Procedure} split lst count
- Split a list @var{lst} into @var{count} chunks. Return a list of chunks.
- Example:
- @lisp
- (split '(a b c d) 2)
- @result{} '((a b) (c d))
- @end lisp
- @end deffn
- @deffn {Scheme Procedure} make-job type node data proc
- Make a new job of @var{type} using @var{node}.
- @end deffn
- @deffn {Scheme Procedure} job? x
- Return @code{#t} if @var{x} is a job object, @code{#f} otherwise.
- @end deffn
- @deffn {Scheme Procedure} job-type job
- Get type of a @var{job}.
- @end deffn
- @deffn {Scheme Procedure} job-node job
- Get a @var{job} node.
- @end deffn
- @deffn {Scheme Procedure} set-job-node job node
- Transfer @var{job} to a new @var{node}. Return a new job object.
- @end deffn
- @deffn {Scheme Procedure} job-data job
- Get a @var{job} data.
- @end deffn
- @deffn {Scheme Procedure} job-proc job
- Get a @var{job} procedure.
- @end deffn
- @deffn {Scheme Procedure} assign-eval nodes expressions
- Split an @var{expressions} list to nearly equal parts according to the length
- of a @var{nodes} list and assign each evaluation job to a node. Return a list
- of assigned jobs.
- @end deffn
- @deffn {Scheme Procedure} assign-map nodes lst proc
- Split the work to nearly equal parts according to length of @var{nodes} list
- and assign each part of work to a node. Return list of assigned jobs.
- @end deffn
- @deffn {Scheme Procedure} hand-out-job job
- Hand out @var{job} to the assigned node and return the result of computation.
- @end deffn
- @deffn {Scheme Procedure} job->sexp job
- Convert a @var{job} to an equivalent symbolic expression.
- @end deffn
- @c Local Variables:
- @c TeX-master: "guile-ssh.texi"
- @c End:
|