This is octave.info, produced by makeinfo version 6.5 from octave.texi.

START-INFO-DIR-ENTRY
* Octave: (octave).           Interactive language for numerical computations.

END-INFO-DIR-ENTRY

   Copyright © 1996-2018 John W. Eaton.

   Permission is granted to make and distribute verbatim copies of this
manual provided the copyright notice and this permission notice are
preserved on all copies.

   Permission is granted to copy and distribute modified versions of
this manual under the conditions for verbatim copying, provided that the
entire resulting derived work is distributed under the terms of a
permission notice identical to this one.

   Permission is granted to copy and distribute translations of this
manual into another language, under the above conditions for modified
versions.


File: octave.info,  Node: Functions of a Matrix,  Next: Specialized Solvers,  Prev: Matrix Factorizations,  Up: Linear Algebra

18.4 Functions of a Matrix
==========================

 -- : expm (A)
     Return the exponential of a matrix.

     The matrix exponential is defined as the infinite Taylor series

          expm (A) = I + A + A^2/2! + A^3/3! + ...

     However, the Taylor series is _not_ the way to compute the matrix
     exponential; see Moler and Van Loan, ‘Nineteen Dubious Ways to
     Compute the Exponential of a Matrix’, SIAM Review, 1978.  This
     routine uses Ward’s diagonal Padé approximation method with three
     step preconditioning (SIAM Journal on Numerical Analysis, 1977).
     Diagonal Padé approximations are rational polynomials of matrices

               -1
          D (A)   N (A)

     whose Taylor series matches the first ‘2q+1’ terms of the Taylor
     series above; direct evaluation of the Taylor series (with the same
     preconditioning steps) may be desirable in lieu of the Padé
     approximation when ‘Dq(A)’ is ill-conditioned.

     See also: *note logm: XREFlogm, *note sqrtm: XREFsqrtm.

 -- : S = logm (A)
 -- : S = logm (A, OPT_ITERS)
 -- : [S, ITERS] = logm (...)
     Compute the matrix logarithm of the square matrix A.

     The implementation utilizes a Padé approximant and the identity

          logm (A) = 2^k * logm (A^(1 / 2^k))

     The optional input OPT_ITERS is the maximum number of square roots
     to compute and defaults to 100.

     The optional output ITERS is the number of square roots actually
     computed.

     See also: *note expm: XREFexpm, *note sqrtm: XREFsqrtm.

 -- : S = sqrtm (A)
 -- : [S, ERROR_ESTIMATE] = sqrtm (A)
     Compute the matrix square root of the square matrix A.

     Ref: N.J. Higham.  ‘A New sqrtm for MATLAB’.  Numerical Analysis
     Report No.  336, Manchester Centre for Computational Mathematics,
     Manchester, England, January 1999.

     See also: *note expm: XREFexpm, *note logm: XREFlogm.

 -- : kron (A, B)
 -- : kron (A1, A2, ...)
     Form the Kronecker product of two or more matrices.

     This is defined block by block as

          x = [ a(i,j)*b ]

     For example:

          kron (1:4, ones (3, 1))
               ⇒  1  2  3  4
                   1  2  3  4
                   1  2  3  4

     If there are more than two input arguments A1, A2, ..., AN the
     Kronecker product is computed as

          kron (kron (A1, A2), ..., AN)

     Since the Kronecker product is associative, this is well-defined.

 -- : blkmm (A, B)
     Compute products of matrix blocks.

     The blocks are given as 2-dimensional subarrays of the arrays A, B.
     The size of A must have the form ‘[m,k,...]’ and size of B must be
     ‘[k,n,...]’.  The result is then of size ‘[m,n,...]’ and is
     computed as follows:

          for i = 1:prod (size (A)(3:end))
            C(:,:,i) = A(:,:,i) * B(:,:,i)
          endfor

 -- : X = sylvester (A, B, C)
     Solve the Sylvester equation

          A X + X B = C

     using standard LAPACK subroutines.

     For example:

          sylvester ([1, 2; 3, 4], [5, 6; 7, 8], [9, 10; 11, 12])
             ⇒ [ 0.50000, 0.66667; 0.66667, 0.50000 ]


File: octave.info,  Node: Specialized Solvers,  Prev: Functions of a Matrix,  Up: Linear Algebra

18.5 Specialized Solvers
========================

 -- : X = bicg (A, B, RTOL, MAXIT, M1, M2, X0)
 -- : X = bicg (A, B, RTOL, MAXIT, P)
 -- : [X, FLAG, RELRES, ITER, RESVEC] = bicg (A, B, ...)
     Solve ‘A x = b’ using the Bi-conjugate gradient iterative method.

        − RTOL is the relative tolerance, if not given or set to [] the
          default value 1e-6 is used.

        − MAXIT the maximum number of outer iterations, if not given or
          set to [] the default value ‘min (20, numel (b))’ is used.

        − X0 the initial guess, if not given or set to [] the default
          value ‘zeros (size (b))’ is used.

     A can be passed as a matrix or as a function handle or inline
     function ‘f’ such that ‘f(x, "notransp") = A*x’ and ‘f(x, "transp")
     = A'*x’.

     The preconditioner P is given as ‘P = M1 * M2’.  Both M1 and M2 can
     be passed as a matrix or as a function handle or inline function
     ‘g’ such that ‘g(x, "notransp") = M1 \ x’ or ‘g(x, "notransp") = M2
     \ x’ and ‘g(x, "transp") = M1' \ x’ or ‘g(x, "transp") = M2' \ x’.

     If called with more than one output parameter

        − FLAG indicates the exit status:

             − 0: iteration converged to the within the chosen tolerance

             − 1: the maximum number of iterations was reached before
               convergence

             − 3: the algorithm reached stagnation

          (the value 2 is unused but skipped for compatibility).

        − RELRES is the final value of the relative residual.

        − ITER is the number of iterations performed.

        − RESVEC is a vector containing the relative residual at each
          iteration.

     See also: *note bicgstab: XREFbicgstab, *note cgs: XREFcgs, *note
     gmres: XREFgmres, *note pcg: XREFpcg, *note qmr: XREFqmr.

 -- : X = bicgstab (A, B, RTOL, MAXIT, M1, M2, X0)
 -- : X = bicgstab (A, B, RTOL, MAXIT, P)
 -- : [X, FLAG, RELRES, ITER, RESVEC] = bicgstab (A, B, ...)
     Solve ‘A x = b’ using the stabilizied Bi-conjugate gradient
     iterative method.

        − RTOL is the relative tolerance, if not given or set to [] the
          default value 1e-6 is used.

        − MAXIT the maximum number of outer iterations, if not given or
          set to [] the default value ‘min (20, numel (b))’ is used.

        − X0 the initial guess, if not given or set to [] the default
          value ‘zeros (size (b))’ is used.

     A can be passed as a matrix or as a function handle or inline
     function ‘f’ such that ‘f(x) = A*x’.

     The preconditioner P is given as ‘P = M1 * M2’.  Both M1 and M2 can
     be passed as a matrix or as a function handle or inline function
     ‘g’ such that ‘g(x) = M1 \ x’ or ‘g(x) = M2 \ x’.

     If called with more than one output parameter

        − FLAG indicates the exit status:

             − 0: iteration converged to the within the chosen tolerance

             − 1: the maximum number of iterations was reached before
               convergence

             − 3: the algorithm reached stagnation

          (the value 2 is unused but skipped for compatibility).

        − RELRES is the final value of the relative residual.

        − ITER is the number of iterations performed.

        − RESVEC is a vector containing the relative residual at each
          iteration.

     See also: *note bicg: XREFbicg, *note cgs: XREFcgs, *note gmres:
     XREFgmres, *note pcg: XREFpcg, *note qmr: XREFqmr.

 -- : X = cgs (A, B, RTOL, MAXIT, M1, M2, X0)
 -- : X = cgs (A, B, RTOL, MAXIT, P)
 -- : [X, FLAG, RELRES, ITER, RESVEC] = cgs (A, B, ...)
     Solve ‘A x = b’, where A is a square matrix, using the Conjugate
     Gradients Squared method.

        − RTOL is the relative tolerance, if not given or set to [] the
          default value 1e-6 is used.

        − MAXIT the maximum number of outer iterations, if not given or
          set to [] the default value ‘min (20, numel (b))’ is used.

        − X0 the initial guess, if not given or set to [] the default
          value ‘zeros (size (b))’ is used.

     A can be passed as a matrix or as a function handle or inline
     function ‘f’ such that ‘f(x) = A*x’.

     The preconditioner P is given as ‘P = M1 * M2’.  Both M1 and M2 can
     be passed as a matrix or as a function handle or inline function
     ‘g’ such that ‘g(x) = M1 \ x’ or ‘g(x) = M2 \ x’.

     If called with more than one output parameter

        − FLAG indicates the exit status:

             − 0: iteration converged to the within the chosen tolerance

             − 1: the maximum number of iterations was reached before
               convergence

             − 3: the algorithm reached stagnation

          (the value 2 is unused but skipped for compatibility).

        − RELRES is the final value of the relative residual.

        − ITER is the number of iterations performed.

        − RESVEC is a vector containing the relative residual at each
          iteration.

     See also: *note pcg: XREFpcg, *note bicgstab: XREFbicgstab, *note
     bicg: XREFbicg, *note gmres: XREFgmres, *note qmr: XREFqmr.

 -- : X = gmres (A, B, M, RTOL, MAXIT, M1, M2, X0)
 -- : X = gmres (A, B, M, RTOL, MAXIT, P)
 -- : [X, FLAG, RELRES, ITER, RESVEC] = gmres (...)
     Solve ‘A x = b’ using the Preconditioned GMRES iterative method
     with restart, a.k.a.  PGMRES(m).

        − RTOL is the relative tolerance, if not given or set to [] the
          default value 1e-6 is used.

        − MAXIT is the maximum number of outer iterations, if not given
          or set to [] the default value ‘min (10, numel (b) / restart)’
          is used.

        − X0 is the initial guess, if not given or set to [] the default
          value ‘zeros (size (b))’ is used.

        − M is the restart parameter, if not given or set to [] the
          default value ‘numel (b)’ is used.

     Argument A can be passed as a matrix, function handle, or inline
     function ‘f’ such that ‘f(x) = A*x’.

     The preconditioner P is given as ‘P = M1 * M2’.  Both M1 and M2 can
     be passed as a matrix, function handle, or inline function ‘g’ such
     that ‘g(x) = M1\x’ or ‘g(x) = M2\x’.

     Besides the vector X, additional outputs are:

        − FLAG indicates the exit status:

          0 : iteration converged to within the specified tolerance

          1 : maximum number of iterations exceeded

          2 : unused, but skipped for compatibility

          3 : algorithm reached stagnation (no change between iterations)

        − RELRES is the final value of the relative residual.

        − ITER is a vector containing the number of outer iterations and
          total iterations performed.

        − RESVEC is a vector containing the relative residual at each
          iteration.

     See also: *note bicg: XREFbicg, *note bicgstab: XREFbicgstab, *note
     cgs: XREFcgs, *note pcg: XREFpcg, *note pcr: XREFpcr, *note qmr:
     XREFqmr.

 -- : X = qmr (A, B, RTOL, MAXIT, M1, M2, X0)
 -- : X = qmr (A, B, RTOL, MAXIT, P)
 -- : [X, FLAG, RELRES, ITER, RESVEC] = qmr (A, B, ...)
     Solve ‘A x = b’ using the Quasi-Minimal Residual iterative method
     (without look-ahead).

        − RTOL is the relative tolerance, if not given or set to [] the
          default value 1e-6 is used.

        − MAXIT the maximum number of outer iterations, if not given or
          set to [] the default value ‘min (20, numel (b))’ is used.

        − X0 the initial guess, if not given or set to [] the default
          value ‘zeros (size (b))’ is used.

     A can be passed as a matrix or as a function handle or inline
     function ‘f’ such that ‘f(x, "notransp") = A*x’ and ‘f(x, "transp")
     = A'*x’.

     The preconditioner P is given as ‘P = M1 * M2’.  Both M1 and M2 can
     be passed as a matrix or as a function handle or inline function
     ‘g’ such that ‘g(x, "notransp") = M1 \ x’ or ‘g(x, "notransp") = M2
     \ x’ and ‘g(x, "transp") = M1' \ x’ or ‘g(x, "transp") = M2' \ x’.

     If called with more than one output parameter

        − FLAG indicates the exit status:

             − 0: iteration converged to the within the chosen tolerance

             − 1: the maximum number of iterations was reached before
               convergence

             − 3: the algorithm reached stagnation

          (the value 2 is unused but skipped for compatibility).

        − RELRES is the final value of the relative residual.

        − ITER is the number of iterations performed.

        − RESVEC is a vector containing the residual norms at each
          iteration.

     References:

       1. R. Freund and N. Nachtigal, ‘QMR: a quasi-minimal residual
          method for non-Hermitian linear systems’, Numerische
          Mathematik, 1991, 60, pp.  315-339.

       2. R. Barrett, M. Berry, T. Chan, J. Demmel, J. Donato, J.
          Dongarra, V. Eijkhour, R. Pozo, C. Romine, and H. van der
          Vorst, ‘Templates for the solution of linear systems: Building
          blocks for iterative methods’, SIAM, 2nd ed., 1994.

     See also: *note bicg: XREFbicg, *note bicgstab: XREFbicgstab, *note
     cgs: XREFcgs, *note gmres: XREFgmres, *note pcg: XREFpcg.


File: octave.info,  Node: Vectorization and Faster Code Execution,  Next: Nonlinear Equations,  Prev: Linear Algebra,  Up: Top

19 Vectorization and Faster Code Execution
******************************************

Vectorization is a programming technique that uses vector operations
instead of element-by-element loop-based operations.  Besides frequently
producing more succinct Octave code, vectorization also allows for
better optimization in the subsequent implementation.  The optimizations
may occur either in Octave’s own Fortran, C, or C++ internal
implementation, or even at a lower level depending on the compiler and
external numerical libraries used to build Octave.  The ultimate goal is
to make use of your hardware’s vector instructions if possible or to
perform other optimizations in software.

   Vectorization is not a concept unique to Octave, but it is
particularly important because Octave is a matrix-oriented language.
Vectorized Octave code will see a dramatic speed up (10X–100X) in most
cases.

   This chapter discusses vectorization and other techniques for writing
faster code.

* Menu:

* Basic Vectorization::        Basic techniques for code optimization
* Broadcasting::               Broadcasting operations
* Function Application::       Applying functions to arrays, cells, and structs
* Accumulation::               Accumulation functions
* JIT Compiler::               Just-In-Time Compiler for loops
* Miscellaneous Techniques::   Other techniques for speeding up code
* Examples::


File: octave.info,  Node: Basic Vectorization,  Next: Broadcasting,  Up: Vectorization and Faster Code Execution

19.1 Basic Vectorization
========================

To a very good first approximation, the goal in vectorization is to
write code that avoids loops and uses whole-array operations.  As a
trivial example, consider

     for i = 1:n
       for j = 1:m
         c(i,j) = a(i,j) + b(i,j);
       endfor
     endfor

compared to the much simpler

     c = a + b;

This isn’t merely easier to write; it is also internally much easier to
optimize.  Octave delegates this operation to an underlying
implementation which, among other optimizations, may use special vector
hardware instructions or could conceivably even perform the additions in
parallel.  In general, if the code is vectorized, the underlying
implementation has more freedom about the assumptions it can make in
order to achieve faster execution.

   This is especially important for loops with "cheap" bodies.  Often it
suffices to vectorize just the innermost loop to get acceptable
performance.  A general rule of thumb is that the "order" of the
vectorized body should be greater or equal to the "order" of the
enclosing loop.

   As a less trivial example, instead of

     for i = 1:n-1
       a(i) = b(i+1) - b(i);
     endfor

write

     a = b(2:n) - b(1:n-1);

   This shows an important general concept about using arrays for
indexing instead of looping over an index variable.  *Note Index
Expressions::.  Also use boolean indexing generously.  If a condition
needs to be tested, this condition can also be written as a boolean
index.  For instance, instead of

     for i = 1:n
       if (a(i) > 5)
         a(i) -= 20
       endif
     endfor

write

     a(a>5) -= 20;

which exploits the fact that ‘a > 5’ produces a boolean index.

   Use elementwise vector operators whenever possible to avoid looping
(operators like ‘.*’ and ‘.^’).  *Note Arithmetic Ops::.  For simple
inline functions, the ‘vectorize’ function can do this automatically.

 -- : vectorize (FUN)
     Create a vectorized version of the inline function FUN by replacing
     all occurrences of ‘*’, ‘/’, etc., with ‘.*’, ‘./’, etc.

     This may be useful, for example, when using inline functions with
     numerical integration or optimization where a vector-valued
     function is expected.

          fcn = vectorize (inline ("x^2 - 1"))
             ⇒ fcn = f(x) = x.^2 - 1
          quadv (fcn, 0, 3)
             ⇒ 6

     See also: *note inline: XREFinline, *note formula: XREFformula,
     *note argnames: XREFargnames.

   Also exploit broadcasting in these elementwise operators both to
avoid looping and unnecessary intermediate memory allocations.  *Note
Broadcasting::.

   Use built-in and library functions if possible.  Built-in and
compiled functions are very fast.  Even with an m-file library function,
chances are good that it is already optimized, or will be optimized more
in a future release.

   For instance, even better than

     a = b(2:n) - b(1:n-1);

is

     a = diff (b);

   Most Octave functions are written with vector and array arguments in
mind.  If you find yourself writing a loop with a very simple operation,
chances are that such a function already exists.  The following
functions occur frequently in vectorized code:

   • Index manipulation

        • find

        • sub2ind

        • ind2sub

        • sort

        • unique

        • lookup

        • ifelse / merge

   • Repetition

        • repmat

        • repelems

   • Vectorized arithmetic

        • sum

        • prod

        • cumsum

        • cumprod

        • sumsq

        • diff

        • dot

        • cummax

        • cummin

   • Shape of higher dimensional arrays

        • reshape

        • resize

        • permute

        • squeeze

        • deal


File: octave.info,  Node: Broadcasting,  Next: Function Application,  Prev: Basic Vectorization,  Up: Vectorization and Faster Code Execution

19.2 Broadcasting
=================

Broadcasting refers to how Octave binary operators and functions behave
when their matrix or array operands or arguments differ in size.  Since
version 3.6.0, Octave now automatically broadcasts vectors, matrices,
and arrays when using elementwise binary operators and functions.
Broadly speaking, smaller arrays are “broadcast” across the larger one,
until they have a compatible shape.  The rule is that corresponding
array dimensions must either

  1. be equal, or

  2. one of them must be 1.

In case all dimensions are equal, no broadcasting occurs and ordinary
element-by-element arithmetic takes place.  For arrays of higher
dimensions, if the number of dimensions isn’t the same, then missing
trailing dimensions are treated as 1.  When one of the dimensions is 1,
the array with that singleton dimension gets copied along that dimension
until it matches the dimension of the other array.  For example,
consider

     x = [1 2 3;
          4 5 6;
          7 8 9];

     y = [10 20 30];

     x + y

Without broadcasting, ‘x + y’ would be an error because the dimensions
do not agree.  However, with broadcasting it is as if the following
operation were performed:

     x = [1 2 3
          4 5 6
          7 8 9];

     y = [10 20 30
          10 20 30
          10 20 30];

     x + y
     ⇒    11   22   33
           14   25   36
           17   28   39

That is, the smaller array of size ‘[1 3]’ gets copied along the
singleton dimension (the number of rows) until it is ‘[3 3]’.  No actual
copying takes place, however.  The internal implementation reuses
elements along the necessary dimension in order to achieve the desired
effect without copying in memory.

   Both arrays can be broadcast across each other, for example, all
pairwise differences of the elements of a vector with itself:

     y - y'
     ⇒    0   10   20
         -10    0   10
         -20  -10    0

Here the vectors of size ‘[1 3]’ and ‘[3 1]’ both get broadcast into
matrices of size ‘[3 3]’ before ordinary matrix subtraction takes place.

   A special case of broadcasting that may be familiar is when all
dimensions of the array being broadcast are 1, i.e., the array is a
scalar.  Thus for example, operations like ‘x - 42’ and ‘max (x, 2)’ are
basic examples of broadcasting.

   For a higher-dimensional example, suppose ‘img’ is an RGB image of
size ‘[m n 3]’ and we wish to multiply each color by a different scalar.
The following code accomplishes this with broadcasting,

     img .*= permute ([0.8, 0.9, 1.2], [1, 3, 2]);

Note the usage of permute to match the dimensions of the ‘[0.8, 0.9,
1.2]’ vector with ‘img’.

   For functions that are not written with broadcasting semantics,
‘bsxfun’ can be useful for coercing them to broadcast.

 -- : bsxfun (F, A, B)
     The binary singleton expansion function performs broadcasting, that
     is, it applies a binary function F element-by-element to two array
     arguments A and B, and expands as necessary singleton dimensions in
     either input argument.

     F is a function handle, inline function, or string containing the
     name of the function to evaluate.  The function F must be capable
     of accepting two column-vector arguments of equal length, or one
     column vector argument and a scalar.

     The dimensions of A and B must be equal or singleton.  The
     singleton dimensions of the arrays will be expanded to the same
     dimensionality as the other array.

     See also: *note arrayfun: XREFarrayfun, *note cellfun: XREFcellfun.

   Broadcasting is only applied if either of the two broadcasting
conditions hold.  As usual, however, broadcasting does not apply when
two dimensions differ and neither is 1:

     x = [1 2 3
          4 5 6];
     y = [10 20
          30 40];
     x + y

This will produce an error about nonconformant arguments.

   Besides common arithmetic operations, several functions of two
arguments also broadcast.  The full list of functions and operators that
broadcast is

           plus      +  .+
           minus     -  .-
           times     .*
           rdivide   ./
           ldivide   .\
           power     .^  .**
           lt        <
           le        <=
           eq        ==
           gt        >
           ge        >=
           ne        !=  ~=
           and       &
           or        |
           atan2
           hypot
           max
           min
           mod
           rem
           xor

           +=  -=  .+=  .-=  .*=  ./=  .\=  .^=  .**=  &=  |=

   Beware of resorting to broadcasting if a simpler operation will
suffice.  For matrices A and B, consider the following:

     C = sum (permute (A, [1, 3, 2]) .* permute (B, [3, 2, 1]), 3);

This operation broadcasts the two matrices with permuted dimensions
across each other during elementwise multiplication in order to obtain a
larger 3-D array, and this array is then summed along the third
dimension.  A moment of thought will prove that this operation is simply
the much faster ordinary matrix multiplication, ‘C = A*B;’.

   A note on terminology: “broadcasting” is the term popularized by the
Numpy numerical environment in the Python programming language.  In
other programming languages and environments, broadcasting may also be
known as _binary singleton expansion_ (BSX, in MATLAB, and the origin of
the name of the ‘bsxfun’ function), _recycling_ (R programming
language), _single-instruction multiple data_ (SIMD), or _replication_.

19.2.1 Broadcasting and Legacy Code
-----------------------------------

The new broadcasting semantics almost never affect code that worked in
previous versions of Octave.  Consequently, all code inherited from
MATLAB that worked in previous versions of Octave should still work
without change in Octave.  The only exception is code such as

     try
       c = a.*b;
     catch
       c = a.*a;
     end_try_catch

that may have relied on matrices of different size producing an error.
Because such operation is now valid Octave syntax, this will no longer
produce an error.  Instead, the following code should be used:

     if (isequal (size (a), size (b)))
       c = a .* b;
     else
       c = a .* a;
     endif


File: octave.info,  Node: Function Application,  Next: Accumulation,  Prev: Broadcasting,  Up: Vectorization and Faster Code Execution

19.3 Function Application
=========================

As a general rule, functions should already be written with matrix
arguments in mind and should consider whole matrix operations in a
vectorized manner.  Sometimes, writing functions in this way appears
difficult or impossible for various reasons.  For those situations,
Octave provides facilities for applying a function to each element of an
array, cell, or struct.

 -- : arrayfun (FUNC, A)
 -- : X = arrayfun (FUNC, A)
 -- : X = arrayfun (FUNC, A, B, ...)
 -- : [X, Y, ...] = arrayfun (FUNC, A, ...)
 -- : arrayfun (..., "UniformOutput", VAL)
 -- : arrayfun (..., "ErrorHandler", ERRFUNC)

     Execute a function on each element of an array.

     This is useful for functions that do not accept array arguments.
     If the function does accept array arguments it is better to call
     the function directly.

     The first input argument FUNC can be a string, a function handle,
     an inline function, or an anonymous function.  The input argument A
     can be a logic array, a numeric array, a string array, a structure
     array, or a cell array.  By a call of the function ‘arrayfun’ all
     elements of A are passed on to the named function FUNC
     individually.

     The named function can also take more than two input arguments,
     with the input arguments given as third input argument B, fourth
     input argument C, ... If given more than one array input argument
     then all input arguments must have the same sizes, for example:

          arrayfun (@atan2, [1, 0], [0, 1])
               ⇒ [ 1.57080   0.00000 ]

     If the parameter VAL after a further string input argument
     "UniformOutput" is set ‘true’ (the default), then the named
     function FUNC must return a single element which then will be
     concatenated into the return value and is of type matrix.
     Otherwise, if that parameter is set to ‘false’, then the outputs
     are concatenated in a cell array.  For example:

          arrayfun (@(x,y) x:y, "abc", "def", "UniformOutput", false)
          ⇒
             {
               [1,1] = abcd
               [1,2] = bcde
               [1,3] = cdef
             }

     If more than one output arguments are given then the named function
     must return the number of return values that also are expected, for
     example:

          [A, B, C] = arrayfun (@find, [10; 0], "UniformOutput", false)
          ⇒
          A =
          {
             [1,1] =  1
             [2,1] = [](0x0)
          }
          B =
          {
             [1,1] =  1
             [2,1] = [](0x0)
          }
          C =
          {
             [1,1] =  10
             [2,1] = [](0x0)
          }

     If the parameter ERRFUNC after a further string input argument
     "ErrorHandler" is another string, a function handle, an inline
     function, or an anonymous function, then ERRFUNC defines a function
     to call in the case that FUNC generates an error.  The definition
     of the function must be of the form

          function [...] = errfunc (S, ...)

     where there is an additional input argument to ERRFUNC relative to
     FUNC, given by S.  This is a structure with the elements
     "identifier", "message", and "index" giving, respectively, the
     error identifier, the error message, and the index of the array
     elements that caused the error.  The size of the output argument of
     ERRFUNC must have the same size as the output argument of FUNC,
     otherwise a real error is thrown.  For example:

          function y = ferr (s, x), y = "MyString"; endfunction
          arrayfun (@str2num, [1234],
                    "UniformOutput", false, "ErrorHandler", @ferr)
          ⇒
             {
               [1,1] = MyString
             }

     See also: *note spfun: XREFspfun, *note cellfun: XREFcellfun, *note
     structfun: XREFstructfun.

 -- : Y = spfun (F, S)
     Compute ‘f(S)’ for the nonzero values of S.

     This results in a sparse matrix with the same structure as S.  The
     function F can be passed as a string, a function handle, or an
     inline function.

     See also: *note arrayfun: XREFarrayfun, *note cellfun: XREFcellfun,
     *note structfun: XREFstructfun.

 -- : cellfun (NAME, C)
 -- : cellfun ("size", C, K)
 -- : cellfun ("isclass", C, CLASS)
 -- : cellfun (FUNC, C)
 -- : cellfun (FUNC, C, D)
 -- : [A, ...] = cellfun (...)
 -- : cellfun (..., "ErrorHandler", ERRFUNC)
 -- : cellfun (..., "UniformOutput", VAL)

     Evaluate the function named NAME on the elements of the cell array
     C.

     Elements in C are passed on to the named function individually.
     The function NAME can be one of the functions

     ‘isempty’
          Return 1 for empty elements.

     ‘islogical’
          Return 1 for logical elements.

     ‘isnumeric’
          Return 1 for numeric elements.

     ‘isreal’
          Return 1 for real elements.

     ‘length’
          Return a vector of the lengths of cell elements.

     ‘ndims’
          Return the number of dimensions of each element.

     ‘numel’
     ‘prodofsize’
          Return the number of elements contained within each cell
          element.  The number is the product of the dimensions of the
          object at each cell element.

     ‘size’
          Return the size along the K-th dimension.

     ‘isclass’
          Return 1 for elements of CLASS.

     Additionally, ‘cellfun’ accepts an arbitrary function FUNC in the
     form of an inline function, function handle, or the name of a
     function (in a character string).  The function can take one or
     more arguments, with the inputs arguments given by C, D, etc.
     Equally the function can return one or more output arguments.  For
     example:

          cellfun ("atan2", {1, 0}, {0, 1})
               ⇒ [ 1.57080   0.00000 ]

     The number of output arguments of ‘cellfun’ matches the number of
     output arguments of the function.  The outputs of the function will
     be collected into the output arguments of ‘cellfun’ like this:

          function [a, b] = twoouts (x)
            a = x;
            b = x*x;
          endfunction
          [aa, bb] = cellfun (@twoouts, {1, 2, 3})
               ⇒
                  aa =
                     1 2 3
                  bb =
                     1 4 9

     Note that per default the output argument(s) are arrays of the same
     size as the input arguments.  Input arguments that are singleton
     (1x1) cells will be automatically expanded to the size of the other
     arguments.

     If the parameter "UniformOutput" is set to true (the default), then
     the function must return scalars which will be concatenated into
     the return array(s).  If "UniformOutput" is false, the outputs are
     concatenated into a cell array (or cell arrays).  For example:

          cellfun ("tolower", {"Foo", "Bar", "FooBar"},
                   "UniformOutput", false)
          ⇒ {"foo", "bar", "foobar"}

     Given the parameter "ErrorHandler", then ERRFUNC defines a function
     to call in case FUNC generates an error.  The form of the function
     is

          function [...] = errfunc (S, ...)

     where there is an additional input argument to ERRFUNC relative to
     FUNC, given by S.  This is a structure with the elements
     "identifier", "message", and "index" giving respectively the error
     identifier, the error message, and the index into the input
     arguments of the element that caused the error.  For example:

          function y = foo (s, x), y = NaN; endfunction
          cellfun ("factorial", {-1,2}, "ErrorHandler", @foo)
          ⇒ [NaN 2]

     Use ‘cellfun’ intelligently.  The ‘cellfun’ function is a useful
     tool for avoiding loops.  It is often used with anonymous function
     handles; however, calling an anonymous function involves an
     overhead quite comparable to the overhead of an m-file function.
     Passing a handle to a built-in function is faster, because the
     interpreter is not involved in the internal loop.  For example:

          a = {...}
          v = cellfun (@(x) det (x), a); # compute determinants
          v = cellfun (@det, a); # faster

     See also: *note arrayfun: XREFarrayfun, *note structfun:
     XREFstructfun, *note spfun: XREFspfun.

 -- : structfun (FUNC, S)
 -- : [A, ...] = structfun (...)
 -- : structfun (..., "ErrorHandler", ERRFUNC)
 -- : structfun (..., "UniformOutput", VAL)

     Evaluate the function named NAME on the fields of the structure S.
     The fields of S are passed to the function FUNC individually.

     ‘structfun’ accepts an arbitrary function FUNC in the form of an
     inline function, function handle, or the name of a function (in a
     character string).  In the case of a character string argument, the
     function must accept a single argument named X, and it must return
     a string value.  If the function returns more than one argument,
     they are returned as separate output variables.

     If the parameter "UniformOutput" is set to true (the default), then
     the function must return a single element which will be
     concatenated into the return value.  If "UniformOutput" is false,
     the outputs are placed into a structure with the same fieldnames as
     the input structure.

          s.name1 = "John Smith";
          s.name2 = "Jill Jones";
          structfun (@(x) regexp (x, '(\w+)$', "matches"){1}, s,
                     "UniformOutput", false)
          ⇒
             {
               name1 = Smith
               name2 = Jones
             }

     Given the parameter "ErrorHandler", ERRFUNC defines a function to
     call in case FUNC generates an error.  The form of the function is

          function [...] = errfunc (SE, ...)

     where there is an additional input argument to ERRFUNC relative to
     FUNC, given by SE.  This is a structure with the elements
     "identifier", "message" and "index", giving respectively the error
     identifier, the error message, and the index into the input
     arguments of the element that caused the error.  For an example on
     how to use an error handler, *note cellfun: XREFcellfun.

     See also: *note cellfun: XREFcellfun, *note arrayfun: XREFarrayfun,
     *note spfun: XREFspfun.

   Consistent with earlier advice, seek to use Octave built-in functions
whenever possible for the best performance.  This advice applies
especially to the four functions above.  For example, when adding two
arrays together element-by-element one could use a handle to the
built-in addition function ‘@plus’ or define an anonymous function
‘@(x,y) x + y’.  But, the anonymous function is 60% slower than the
first method.  *Note Operator Overloading::, for a list of basic
functions which might be used in place of anonymous ones.


File: octave.info,  Node: Accumulation,  Next: JIT Compiler,  Prev: Function Application,  Up: Vectorization and Faster Code Execution

19.4 Accumulation
=================

Whenever it’s possible to categorize according to indices the elements
of an array when performing a computation, accumulation functions can be
useful.

 -- : accumarray (SUBS, VALS, SZ, FUNC, FILLVAL, ISSPARSE)
 -- : accumarray (SUBS, VALS, ...)

     Create an array by accumulating the elements of a vector into the
     positions defined by their subscripts.

     The subscripts are defined by the rows of the matrix SUBS and the
     values by VALS.  Each row of SUBS corresponds to one of the values
     in VALS.  If VALS is a scalar, it will be used for each of the row
     of SUBS.  If SUBS is a cell array of vectors, all vectors must be
     of the same length, and the subscripts in the Kth vector must
     correspond to the Kth dimension of the result.

     The size of the matrix will be determined by the subscripts
     themselves.  However, if SZ is defined it determines the matrix
     size.  The length of SZ must correspond to the number of columns in
     SUBS.  An exception is if SUBS has only one column, in which case
     SZ may be the dimensions of a vector and the subscripts of SUBS are
     taken as the indices into it.

     The default action of ‘accumarray’ is to sum the elements with the
     same subscripts.  This behavior can be modified by defining the
     FUNC function.  This should be a function or function handle that
     accepts a column vector and returns a scalar.  The result of the
     function should not depend on the order of the subscripts.

     The elements of the returned array that have no subscripts
     associated with them are set to zero.  Defining FILLVAL to some
     other value allows these values to be defined.  This behavior
     changes, however, for certain values of FUNC.  If FUNC is ‘@min’
     (respectively, ‘@max’) then the result will be filled with the
     minimum (respectively, maximum) integer if VALS is of integral
     type, logical false (respectively, logical true) if VALS is of
     logical type, zero if FILLVAL is zero and all values are
     non-positive (respectively, non-negative), and NaN otherwise.

     By default ‘accumarray’ returns a full matrix.  If ISSPARSE is
     logically true, then a sparse matrix is returned instead.

     The following ‘accumarray’ example constructs a frequency table
     that in the first column counts how many occurrences each number in
     the second column has, taken from the vector X.  Note the usage of
     ‘unique’ for assigning to all repeated elements of X the same index
     (*note unique: XREFunique.).

          X = [91, 92, 90, 92, 90, 89, 91, 89, 90, 100, 100, 100];
          [U, ~, J] = unique (X);
          [accumarray(J', 1), U']
            ⇒  2    89
                3    90
                2    91
                2    92
                3   100

     Another example, where the result is a multi-dimensional 3-D array
     and the default value (zero) appears in the output:

          accumarray ([1, 1, 1;
                       2, 1, 2;
                       2, 3, 2;
                       2, 1, 2;
                       2, 3, 2], 101:105)
          ⇒ ans(:,:,1) = [101, 0, 0; 0, 0, 0]
          ⇒ ans(:,:,2) = [0, 0, 0; 206, 0, 208]

     The sparse option can be used as an alternative to the ‘sparse’
     constructor (*note sparse: XREFsparse.).  Thus

          sparse (I, J, SV)

     can be written with ‘accumarray’ as

          accumarray ([I, J], SV', [], [], 0, true)

     For repeated indices, ‘sparse’ adds the corresponding value.  To
     take the minimum instead, use ‘min’ as an accumulator function:

          accumarray ([I, J], SV', [], @min, 0, true)

     The complexity of accumarray in general for the non-sparse case is
     generally O(M+N), where N is the number of subscripts and M is the
     maximum subscript (linearized in multi-dimensional case).  If FUNC
     is one of ‘@sum’ (default), ‘@max’, ‘@min’ or ‘@(x) {x}’, an
     optimized code path is used.  Note that for general reduction
     function the interpreter overhead can play a major part and it may
     be more efficient to do multiple accumarray calls and compute the
     results in a vectorized manner.

     See also: *note accumdim: XREFaccumdim, *note unique: XREFunique,
     *note sparse: XREFsparse.

 -- : accumdim (SUBS, VALS, DIM, N, FUNC, FILLVAL)
     Create an array by accumulating the slices of an array into the
     positions defined by their subscripts along a specified dimension.

     The subscripts are defined by the index vector SUBS.  The dimension
     is specified by DIM.  If not given, it defaults to the first
     non-singleton dimension.  The length of SUBS must be equal to ‘size
     (VALS, DIM)’.

     The extent of the result matrix in the working dimension will be
     determined by the subscripts themselves.  However, if N is defined
     it determines this extent.

     The default action of ‘accumdim’ is to sum the subarrays with the
     same subscripts.  This behavior can be modified by defining the
     FUNC function.  This should be a function or function handle that
     accepts an array and a dimension, and reduces the array along this
     dimension.  As a special exception, the built-in ‘min’ and ‘max’
     functions can be used directly, and ‘accumdim’ accounts for the
     middle empty argument that is used in their calling.

     The slices of the returned array that have no subscripts associated
     with them are set to zero.  Defining FILLVAL to some other value
     allows these values to be defined.

     An example of the use of ‘accumdim’ is:

          accumdim ([1, 2, 1, 2, 1], [ 7, -10,   4;
                                      -5, -12,   8;
                                     -12,   2,   8;
                                     -10,   9,  -3;
                                      -5,  -3, -13])
          ⇒ [-10,-11,-1;-15,-3,5]

     See also: *note accumarray: XREFaccumarray.


File: octave.info,  Node: JIT Compiler,  Next: Miscellaneous Techniques,  Prev: Accumulation,  Up: Vectorization and Faster Code Execution

19.5 JIT Compiler
=================

Vectorization is the preferred technique for eliminating loops and
speeding up code.  Nevertheless, it is not always possible to replace
every loop.  In such situations it may be worth trying Octave’s
*experimental* Just-In-Time (JIT) compiler.

   A JIT compiler works by analyzing the body of a loop, translating the
Octave statements into another language, compiling the new code segment
into an executable, and then running the executable and collecting any
results.  The process is not simple and there is a significant amount of
work to perform for each step.  It can still make sense, however, if the
number of loop iterations is large.  Because Octave is an interpreted
language every time through a loop Octave must parse the statements in
the loop body before executing them.  With a JIT compiler this is done
just once when the body is translated to another language.

   The JIT compiler is a very new feature in Octave and not all valid
Octave statements can currently be accelerated.  However, if no other
technique is available it may be worth benchmarking the code with JIT
enabled.  The function ‘jit_enable’ is used to turn compilation on or
off.  The function ‘jit_startcnt’ sets the threshold for acceleration.
Loops with iteration counts above ‘jit_startcnt’ will be accelerated.
The functions ‘jit_failcnt’ and ‘debug_jit’ are not likely to be of use
to anyone not working directly on the implementation of the JIT
compiler.

 -- : VAL = jit_enable ()
 -- : OLD_VAL = jit_enable (NEW_VAL)
 -- : jit_enable (NEW_VAL, "local")
     Query or set the internal variable that enables Octave’s JIT
     compiler.

     When called from inside a function with the "local" option, the
     variable is changed locally for the function and any subroutines it
     calls.  The original variable value is restored when exiting the
     function.

     See also: *note jit_startcnt: XREFjit_startcnt, *note debug_jit:
     XREFdebug_jit.

 -- : VAL = jit_startcnt ()
 -- : OLD_VAL = jit_startcnt (NEW_VAL)
 -- : jit_startcnt (NEW_VAL, "local")
     Query or set the internal variable that determines whether JIT
     compilation will take place for a specific loop.

     Because compilation is a costly operation it does not make sense to
     employ JIT when the loop count is low.  By default only loops with
     greater than 1000 iterations will be accelerated.

     When called from inside a function with the "local" option, the
     variable is changed locally for the function and any subroutines it
     calls.  The original variable value is restored when exiting the
     function.

     See also: *note jit_enable: XREFjit_enable, *note jit_failcnt:
     XREFjit_failcnt, *note debug_jit: XREFdebug_jit.

 -- : VAL = jit_failcnt ()
 -- : OLD_VAL = jit_failcnt (NEW_VAL)
 -- : jit_failcnt (NEW_VAL, "local")
     Query or set the internal variable that counts the number of JIT
     fail exceptions for Octave’s JIT compiler.

     When called from inside a function with the "local" option, the
     variable is changed locally for the function and any subroutines it
     calls.  The original variable value is restored when exiting the
     function.

     See also: *note jit_enable: XREFjit_enable, *note jit_startcnt:
     XREFjit_startcnt, *note debug_jit: XREFdebug_jit.

 -- : VAL = debug_jit ()
 -- : OLD_VAL = debug_jit (NEW_VAL)
 -- : debug_jit (NEW_VAL, "local")
     Query or set the internal variable that determines whether
     debugging/tracing is enabled for Octave’s JIT compiler.

     When called from inside a function with the "local" option, the
     variable is changed locally for the function and any subroutines it
     calls.  The original variable value is restored when exiting the
     function.

     See also: *note jit_enable: XREFjit_enable, *note jit_startcnt:
     XREFjit_startcnt.


File: octave.info,  Node: Miscellaneous Techniques,  Next: Examples,  Prev: JIT Compiler,  Up: Vectorization and Faster Code Execution

19.6 Miscellaneous Techniques
=============================

Here are some other ways of improving the execution speed of Octave
programs.

   • Avoid computing costly intermediate results multiple times.  Octave
     currently does not eliminate common subexpressions.  Also, certain
     internal computation results are cached for variables.  For
     instance, if a matrix variable is used multiple times as an index,
     checking the indices (and internal conversion to integers) is only
     done once.

   • Be aware of lazy copies (copy-on-write).  When a copy of an object
     is created, the data is not immediately copied, but rather shared.
     The actual copying is postponed until the copied data needs to be
     modified.  For example:

          a = zeros (1000); # create a 1000x1000 matrix
          b = a; # no copying done here
          b(1) = 1; # copying done here

     Lazy copying applies to whole Octave objects such as matrices,
     cells, struct, and also individual cell or struct elements (not
     array elements).

     Additionally, index expressions also use lazy copying when Octave
     can determine that the indexed portion is contiguous in memory.
     For example:

          a = zeros (1000); # create a 1000x1000 matrix
          b = a(:,10:100);  # no copying done here
          b = a(10:100,:);  # copying done here

     This applies to arrays (matrices), cell arrays, and structs indexed
     using ‘()’.  Index expressions generating comma-separated lists can
     also benefit from shallow copying in some cases.  In particular,
     when A is a struct array, expressions like ‘{a.x}, {a(:,2).x}’ will
     use lazy copying, so that data can be shared between a struct array
     and a cell array.

     Most indexing expressions do not live longer than their parent
     objects.  In rare cases, however, a lazily copied slice outlasts
     its parent, in which case it becomes orphaned, still occupying
     unnecessarily more memory than needed.  To provide a remedy working
     in most real cases, Octave checks for orphaned lazy slices at
     certain situations, when a value is stored into a "permanent"
     location, such as a named variable or cell or struct element, and
     possibly economizes them.  For example:

          a = zeros (1000); # create a 1000x1000 matrix
          b = a(:,10:100);  # lazy slice
          a = []; # the original "a" array is still allocated
          c{1} = b; # b is reallocated at this point

   • Avoid deep recursion.  Function calls to m-file functions carry a
     relatively significant overhead, so rewriting a recursion as a loop
     often helps.  Also, note that the maximum level of recursion is
     limited.

   • Avoid resizing matrices unnecessarily.  When building a single
     result matrix from a series of calculations, set the size of the
     result matrix first, then insert values into it.  Write

          result = zeros (big_n, big_m)
          for i = over:and_over
            ridx = ...
            cidx = ...
            result(ridx, cidx) = new_value ();
          endfor

     instead of

          result = [];
          for i = ever:and_ever
            result = [ result, new_value() ];
          endfor

     Sometimes the number of items can not be computed in advance, and
     stack-like operations are needed.  When elements are being
     repeatedly inserted or removed from the end of an array, Octave
     detects it as stack usage and attempts to use a smarter memory
     management strategy by pre-allocating the array in bigger chunks.
     This strategy is also applied to cell and struct arrays.

          a = [];
          while (condition)
            ...
            a(end+1) = value; # "push" operation
            ...
            a(end) = []; # "pop" operation
            ...
          endwhile

   • Avoid calling ‘eval’ or ‘feval’ excessively.  Parsing input or
     looking up the name of a function in the symbol table are
     relatively expensive operations.

     If you are using ‘eval’ merely as an exception handling mechanism,
     and not because you need to execute some arbitrary text, use the
     ‘try’ statement instead.  *Note The try Statement::.

   • Use ‘ignore_function_time_stamp’ when appropriate.  If you are
     calling lots of functions, and none of them will need to change
     during your run, set the variable ‘ignore_function_time_stamp’ to
     "all".  This will stop Octave from checking the time stamp of a
     function file to see if it has been updated while the program is
     being run.


File: octave.info,  Node: Examples,  Prev: Miscellaneous Techniques,  Up: Vectorization and Faster Code Execution

19.7 Examples
=============

The following are examples of vectorization questions asked by actual
users of Octave and their solutions.

   • For a vector ‘A’, the following loop

          n = length (A);
          B = zeros (n, 2);
          for i = 1:length (A)
            ## this will be two columns, the first is the difference and
            ## the second the mean of the two elements used for the diff.
            B(i,:) = [A(i+1)-A(i), (A(i+1) + A(i))/2];
          endfor

     can be turned into the following one-liner:

          B = [diff(A)(:), 0.5*(A(1:end-1)+A(2:end))(:)]

     Note the usage of colon indexing to flatten an intermediate result
     into a column vector.  This is a common vectorization trick.


File: octave.info,  Node: Nonlinear Equations,  Next: Diagonal and Permutation Matrices,  Prev: Vectorization and Faster Code Execution,  Up: Top

20 Nonlinear Equations
**********************

* Menu:

* Solvers::
* Minimizers::


File: octave.info,  Node: Solvers,  Next: Minimizers,  Up: Nonlinear Equations

20.1 Solvers
============

Octave can solve sets of nonlinear equations of the form

     F (x) = 0

using the function ‘fsolve’, which is based on the MINPACK subroutine
‘hybrd’.  This is an iterative technique so a starting point must be
provided.  This also has the consequence that convergence is not
guaranteed even if a solution exists.

 -- : fsolve (FCN, X0, OPTIONS)
 -- : [X, FVEC, INFO, OUTPUT, FJAC] = fsolve (FCN, ...)
     Solve a system of nonlinear equations defined by the function FCN.

     FCN should accept a vector (array) defining the unknown variables,
     and return a vector of left-hand sides of the equations.
     Right-hand sides are defined to be zeros.  In other words, this
     function attempts to determine a vector X such that ‘FCN (X)’ gives
     (approximately) all zeros.

     X0 determines a starting guess.  The shape of X0 is preserved in
     all calls to FCN, but otherwise it is treated as a column vector.

     OPTIONS is a structure specifying additional options.  Currently,
     ‘fsolve’ recognizes these options: "FunValCheck", "OutputFcn",
     "TolX", "TolFun", "MaxIter", "MaxFunEvals", "Jacobian", "Updating",
     "ComplexEqn" "TypicalX", "AutoScaling" and "FinDiffType".

     If "Jacobian" is "on", it specifies that FCN, called with 2 output
     arguments also returns the Jacobian matrix of right-hand sides at
     the requested point.  "TolX" specifies the termination tolerance in
     the unknown variables, while "TolFun" is a tolerance for equations.
     Default is ‘1e-7’ for both "TolX" and "TolFun".

     If "AutoScaling" is on, the variables will be automatically scaled
     according to the column norms of the (estimated) Jacobian.  As a
     result, TolF becomes scaling-independent.  By default, this option
     is off because it may sometimes deliver unexpected (though
     mathematically correct) results.

     If "Updating" is "on", the function will attempt to use Broyden
     updates to update the Jacobian, in order to reduce the amount of
     Jacobian calculations.  If your user function always calculates the
     Jacobian (regardless of number of output arguments) then this
     option provides no advantage and should be set to false.

     "ComplexEqn" is "on", ‘fsolve’ will attempt to solve complex
     equations in complex variables, assuming that the equations possess
     a complex derivative (i.e., are holomorphic).  If this is not what
     you want, you should unpack the real and imaginary parts of the
     system to get a real system.

     For description of the other options, see ‘optimset’.

     On return, FVAL contains the value of the function FCN evaluated at
     X.

     INFO may be one of the following values:

     1
          Converged to a solution point.  Relative residual error is
          less than specified by TolFun.

     2
          Last relative step size was less that TolX.

     3
          Last relative decrease in residual was less than TolF.

     0
          Iteration limit exceeded.

     -3
          The trust region radius became excessively small.

     Note: If you only have a single nonlinear equation of one variable,
     using ‘fzero’ is usually a much better idea.

     Note about user-supplied Jacobians: As an inherent property of the
     algorithm, a Jacobian is always requested for a solution vector
     whose residual vector is already known, and it is the last accepted
     successful step.  Often this will be one of the last two calls, but
     not always.  If the savings by reusing intermediate results from
     residual calculation in Jacobian calculation are significant, the
     best strategy is to employ OutputFcn: After a vector is evaluated
     for residuals, if OutputFcn is called with that vector, then the
     intermediate results should be saved for future Jacobian
     evaluation, and should be kept until a Jacobian evaluation is
     requested or until OutputFcn is called with a different vector, in
     which case they should be dropped in favor of this most recent
     vector.  A short example how this can be achieved follows:

          function [fvec, fjac] = user_func (x, optimvalues, state)
          persistent sav = [], sav0 = [];
          if (nargin == 1)
            ## evaluation call
            if (nargout == 1)
              sav0.x = x; # mark saved vector
              ## calculate fvec, save results to sav0.
            elseif (nargout == 2)
              ## calculate fjac using sav.
            endif
          else
            ## outputfcn call.
            if (all (x == sav0.x))
              sav = sav0;
            endif
            ## maybe output iteration status, etc.
          endif
          endfunction

          ## ...

          fsolve (@user_func, x0, optimset ("OutputFcn", @user_func, ...))

     See also: *note fzero: XREFfzero, *note optimset: XREFoptimset.

   The following is a complete example.  To solve the set of equations

     -2x^2 + 3xy   + 4 sin(y) = 6
      3x^2 - 2xy^2 + 3 cos(x) = -4

you first need to write a function to compute the value of the given
function.  For example:

     function y = f (x)
       y = zeros (2, 1);
       y(1) = -2*x(1)^2 + 3*x(1)*x(2)   + 4*sin(x(2)) - 6;
       y(2) =  3*x(1)^2 - 2*x(1)*x(2)^2 + 3*cos(x(1)) + 4;
     endfunction

   Then, call ‘fsolve’ with a specified initial condition to find the
roots of the system of equations.  For example, given the function ‘f’
defined above,

     [x, fval, info] = fsolve (@f, [1; 2])

results in the solution

     x =

       0.57983
       2.54621

     fval =

       -5.7184e-10
        5.5460e-10

     info = 1

A value of ‘info = 1’ indicates that the solution has converged.

   When no Jacobian is supplied (as in the example above) it is
approximated numerically.  This requires more function evaluations, and
hence is less efficient.  In the example above we could compute the
Jacobian analytically as

     function [y, jac] = f (x)
       y = zeros (2, 1);
       y(1) = -2*x(1)^2 + 3*x(1)*x(2)   + 4*sin(x(2)) - 6;
       y(2) =  3*x(1)^2 - 2*x(1)*x(2)^2 + 3*cos(x(1)) + 4;
       if (nargout == 2)
         jac = zeros (2, 2);
         jac(1,1) =  3*x(2) - 4*x(1);
         jac(1,2) =  4*cos(x(2)) + 3*x(1);
         jac(2,1) = -2*x(2)^2 - 3*sin(x(1)) + 6*x(1);
         jac(2,2) = -4*x(1)*x(2);
       endif
     endfunction

The Jacobian can then be used with the following call to ‘fsolve’:

     [x, fval, info] = fsolve (@f, [1; 2], optimset ("jacobian", "on"));

which gives the same solution as before.

 -- : fzero (FUN, X0)
 -- : fzero (FUN, X0, OPTIONS)
 -- : [X, FVAL, INFO, OUTPUT] = fzero (...)
     Find a zero of a univariate function.

     FUN is a function handle, inline function, or string containing the
     name of the function to evaluate.

     X0 should be a two-element vector specifying two points which
     bracket a zero.  In other words, there must be a change in sign of
     the function between X0(1) and X0(2).  More mathematically, the
     following must hold

          sign (FUN(X0(1))) * sign (FUN(X0(2))) <= 0

     If X0 is a single scalar then several nearby and distant values are
     probed in an attempt to obtain a valid bracketing.  If this is not
     successful, the function fails.

     OPTIONS is a structure specifying additional options.  Currently,
     ‘fzero’ recognizes these options: "FunValCheck", "OutputFcn",
     "TolX", "MaxIter", "MaxFunEvals".  For a description of these
     options, see *note optimset: XREFoptimset.

     On exit, the function returns X, the approximate zero point and
     FVAL, the function value thereof.

     INFO is an exit flag that can have these values:

        • 1 The algorithm converged to a solution.

        • 0 Maximum number of iterations or function evaluations has
          been reached.

        • -1 The algorithm has been terminated from user output
          function.

        • -5 The algorithm may have converged to a singular point.

     OUTPUT is a structure containing runtime information about the
     ‘fzero’ algorithm.  Fields in the structure are:

        • iterations Number of iterations through loop.

        • nfev Number of function evaluations.

        • bracketx A two-element vector with the final bracketing of the
          zero along the x-axis.

        • brackety A two-element vector with the final bracketing of the
          zero along the y-axis.

     See also: *note optimset: XREFoptimset, *note fsolve: XREFfsolve.


File: octave.info,  Node: Minimizers,  Prev: Solvers,  Up: Nonlinear Equations

20.2 Minimizers
===============

Often it is useful to find the minimum value of a function rather than
just the zeroes where it crosses the x-axis.  ‘fminbnd’ is designed for
the simpler, but very common, case of a univariate function where the
interval to search is bounded.  For unbounded minimization of a function
with potentially many variables use ‘fminunc’ or ‘fminsearch’.  The two
functions use different internal algorithms and some knowledge of the
objective function is required.  For functions which can be
differentiated, ‘fminunc’ is appropriate.  For functions with
discontinuities, or for which a gradient search would fail, use
‘fminsearch’.  *Note Optimization::, for minimization with the presence
of constraint functions.  Note that searches can be made for maxima by
simply inverting the objective function (‘Fto_max = -Fto_min’).

 -- : [X, FVAL, INFO, OUTPUT] = fminbnd (FUN, A, B, OPTIONS)
     Find a minimum point of a univariate function.

     FUN should be a function handle or name.  A, B specify a starting
     interval.  OPTIONS is a structure specifying additional options.
     Currently, ‘fminbnd’ recognizes these options: "FunValCheck",
     "OutputFcn", "TolX", "MaxIter", "MaxFunEvals".  For a description
     of these options, see *note optimset: XREFoptimset.

     On exit, the function returns X, the approximate minimum point and
     FVAL, the function value thereof.

     INFO is an exit flag that can have these values:

        • 1 The algorithm converged to a solution.

        • 0 Maximum number of iterations or function evaluations has
          been exhausted.

        • -1 The algorithm has been terminated from user output
          function.

     Notes: The search for a minimum is restricted to be in the interval
     bound by A and B.  If you only have an initial point to begin
     searching from you will need to use an unconstrained minimization
     algorithm such as ‘fminunc’ or ‘fminsearch’.  ‘fminbnd’ internally
     uses a Golden Section search strategy.

     See also: *note fzero: XREFfzero, *note fminunc: XREFfminunc, *note
     fminsearch: XREFfminsearch, *note optimset: XREFoptimset.

 -- : fminunc (FCN, X0)
 -- : fminunc (FCN, X0, OPTIONS)
 -- : [X, FVAL, INFO, OUTPUT, GRAD, HESS] = fminunc (FCN, ...)
     Solve an unconstrained optimization problem defined by the function
     FCN.

     FCN should accept a vector (array) defining the unknown variables,
     and return the objective function value, optionally with gradient.
     ‘fminunc’ attempts to determine a vector X such that ‘FCN (X)’ is a
     local minimum.

     X0 determines a starting guess.  The shape of X0 is preserved in
     all calls to FCN, but otherwise is treated as a column vector.

     OPTIONS is a structure specifying additional options.  Currently,
     ‘fminunc’ recognizes these options: "FunValCheck", "OutputFcn",
     "TolX", "TolFun", "MaxIter", "MaxFunEvals", "GradObj",
     "FinDiffType", "TypicalX", "AutoScaling".

     If "GradObj" is "on", it specifies that FCN, when called with two
     output arguments, also returns the Jacobian matrix of partial first
     derivatives at the requested point.  ‘TolX’ specifies the
     termination tolerance for the unknown variables X, while ‘TolFun’
     is a tolerance for the objective function value FVAL.  The default
     is ‘1e-7’ for both options.

     For a description of the other options, see ‘optimset’.

     On return, X is the location of the minimum and FVAL contains the
     value of the objective function at X.

     INFO may be one of the following values:

     1
          Converged to a solution point.  Relative gradient error is
          less than specified by ‘TolFun’.

     2
          Last relative step size was less than ‘TolX’.

     3
          Last relative change in function value was less than ‘TolFun’.

     0
          Iteration limit exceeded—either maximum number of algorithm
          iterations ‘MaxIter’ or maximum number of function evaluations
          ‘MaxFunEvals’.

     -1
          Algorithm terminated by ‘OutputFcn’.

     -3
          The trust region radius became excessively small.

     Optionally, ‘fminunc’ can return a structure with convergence
     statistics (OUTPUT), the output gradient (GRAD) at the solution X,
     and approximate Hessian (HESS) at the solution X.

     Application Notes: If the objective function is a single nonlinear
     equation of one variable then using ‘fminbnd’ is usually a better
     choice.

     The algorithm used by ‘fminunc’ is a gradient search which depends
     on the objective function being differentiable.  If the function
     has discontinuities it may be better to use a derivative-free
     algorithm such as ‘fminsearch’.

     See also: *note fminbnd: XREFfminbnd, *note fminsearch:
     XREFfminsearch, *note optimset: XREFoptimset.

 -- : X = fminsearch (FUN, X0)
 -- : X = fminsearch (FUN, X0, OPTIONS)
 -- : [X, FVAL] = fminsearch (...)

     Find a value of X which minimizes the function FUN.

     The search begins at the point X0 and iterates using the Nelder &
     Mead Simplex algorithm (a derivative-free method).  This algorithm
     is better-suited to functions which have discontinuities or for
     which a gradient-based search such as ‘fminunc’ fails.

     Options for the search are provided in the parameter OPTIONS using
     the function ‘optimset’.  Currently, ‘fminsearch’ accepts the
     options: "TolX", "MaxFunEvals", "MaxIter", "Display".  For a
     description of these options, see ‘optimset’.

     On exit, the function returns X, the minimum point, and FVAL, the
     function value thereof.

     Example usages:

          fminsearch (@(x) (x(1)-5).^2+(x(2)-8).^4, [0;0])

          fminsearch (inline ("(x(1)-5).^2+(x(2)-8).^4", "x"), [0;0])

     See also: *note fminbnd: XREFfminbnd, *note fminunc: XREFfminunc,
     *note optimset: XREFoptimset.


File: octave.info,  Node: Diagonal and Permutation Matrices,  Next: Sparse Matrices,  Prev: Nonlinear Equations,  Up: Top

21 Diagonal and Permutation Matrices
************************************

* Menu:

* Basic Usage::          Creation and Manipulation of Diagonal/Permutation Matrices
* Matrix Algebra::       Linear Algebra with Diagonal/Permutation Matrices
* Function Support::     Functions That Are Aware of These Matrices
* Example Code::         Examples of Usage
* Zeros Treatment::      Differences in Treatment of Zero Elements


File: octave.info,  Node: Basic Usage,  Next: Matrix Algebra,  Up: Diagonal and Permutation Matrices

21.1 Creating and Manipulating Diagonal/Permutation Matrices
============================================================

A diagonal matrix is defined as a matrix that has zero entries outside
the main diagonal; that is, ‘D(i,j) == 0’ if ‘i != j’.  Most often,
square diagonal matrices are considered; however, the definition can
equally be applied to non-square matrices, in which case we usually
speak of a rectangular diagonal matrix.

   A permutation matrix is defined as a square matrix that has a single
element equal to unity in each row and each column; all other elements
are zero.  That is, there exists a permutation (vector) ‘p’ such that
‘P(i,j) == 1’ if ‘j == p(i)’ and ‘P(i,j) == 0’ otherwise.

   Octave provides special treatment of real and complex rectangular
diagonal matrices, as well as permutation matrices.  They are stored as
special objects, using efficient storage and algorithms, facilitating
writing both readable and efficient matrix algebra expressions in the
Octave language.  The special treatment may be disabled by using the
functions “disable_diagonal_matrix” and “disable_permutation_matrix”.

 -- : VAL = disable_diagonal_matrix ()
 -- : OLD_VAL = disable_diagonal_matrix (NEW_VAL)
 -- : disable_diagonal_matrix (NEW_VAL, "local")
     Query or set the internal variable that controls whether diagonal
     matrices are stored in a special space-efficient format.

     The default value is true.  If this option is disabled Octave will
     store diagonal matrices as full matrices.

     When called from inside a function with the "local" option, the
     variable is changed locally for the function and any subroutines it
     calls.  The original variable value is restored when exiting the
     function.

     See also: *note disable_range: XREFdisable_range, *note
     disable_permutation_matrix: XREFdisable_permutation_matrix.

 -- : VAL = disable_permutation_matrix ()
 -- : OLD_VAL = disable_permutation_matrix (NEW_VAL)
 -- : disable_permutation_matrix (NEW_VAL, "local")
     Query or set the internal variable that controls whether
     permutation matrices are stored in a special space-efficient
     format.

     The default value is true.  If this option is disabled Octave will
     store permutation matrices as full matrices.

     When called from inside a function with the "local" option, the
     variable is changed locally for the function and any subroutines it
     calls.  The original variable value is restored when exiting the
     function.

     See also: *note disable_range: XREFdisable_range, *note
     disable_diagonal_matrix: XREFdisable_diagonal_matrix.

   The space savings are significant as demonstrated by the following
code.

     x = diag (rand (10, 1));
     xf = full (x);
     sizeof (x)
     ⇒ 80
     sizeof (xf)
     ⇒ 800

* Menu:

* Creating Diagonal Matrices::
* Creating Permutation Matrices::
* Explicit and Implicit Conversions::


File: octave.info,  Node: Creating Diagonal Matrices,  Next: Creating Permutation Matrices,  Up: Basic Usage

21.1.1 Creating Diagonal Matrices
---------------------------------

The most common and easiest way to create a diagonal matrix is using the
built-in function “diag”.  The expression ‘diag (v)’, with V a vector,
will create a square diagonal matrix with elements on the main diagonal
given by the elements of V, and size equal to the length of V.  ‘diag
(v, m, n)’ can be used to construct a rectangular diagonal matrix.  The
result of these expressions will be a special diagonal matrix object,
rather than a general matrix object.

   Diagonal matrix with unit elements can be created using “eye”.  Some
other built-in functions can also return diagonal matrices.  Examples
include “balance” or “inv”.

   Example:

       diag (1:4)
     ⇒
     Diagonal Matrix

        1   0   0   0
        0   2   0   0
        0   0   3   0
        0   0   0   4

       diag (1:3,5,3)

     ⇒
     Diagonal Matrix

        1   0   0
        0   2   0
        0   0   3
        0   0   0
        0   0   0


File: octave.info,  Node: Creating Permutation Matrices,  Next: Explicit and Implicit Conversions,  Prev: Creating Diagonal Matrices,  Up: Basic Usage

21.1.2 Creating Permutation Matrices
------------------------------------

For creating permutation matrices, Octave does not introduce a new
function, but rather overrides an existing syntax: permutation matrices
can be conveniently created by indexing an identity matrix by
permutation vectors.  That is, if Q is a permutation vector of length N,
the expression

       P = eye (n) (:, q);

will create a permutation matrix - a special matrix object.

     eye (n) (q, :)

will also work (and create a row permutation matrix), as well as

     eye (n) (q1, q2).

   For example:

       eye (4) ([1,3,2,4],:)
     ⇒
     Permutation Matrix

        1   0   0   0
        0   0   1   0
        0   1   0   0
        0   0   0   1

       eye (4) (:,[1,3,2,4])
     ⇒
     Permutation Matrix

        1   0   0   0
        0   0   1   0
        0   1   0   0
        0   0   0   1

   Mathematically, an identity matrix is both diagonal and permutation
matrix.  In Octave, ‘eye (n)’ returns a diagonal matrix, because a
matrix can only have one class.  You can convert this diagonal matrix to
a permutation matrix by indexing it by an identity permutation, as shown
below.  This is a special property of the identity matrix; indexing
other diagonal matrices generally produces a full matrix.

       eye (3)
     ⇒
     Diagonal Matrix

        1   0   0
        0   1   0
        0   0   1

       eye(3)(1:3,:)
     ⇒
     Permutation Matrix

        1   0   0
        0   1   0
        0   0   1

   Some other built-in functions can also return permutation matrices.
Examples include “inv” or “lu”.


File: octave.info,  Node: Explicit and Implicit Conversions,  Prev: Creating Permutation Matrices,  Up: Basic Usage

21.1.3 Explicit and Implicit Conversions
----------------------------------------

The diagonal and permutation matrices are special objects in their own
right.  A number of operations and built-in functions are defined for
these matrices to use special, more efficient code than would be used
for a full matrix in the same place.  Examples are given in further
sections.

   To facilitate smooth mixing with full matrices, backward
compatibility, and compatibility with MATLAB, the diagonal and
permutation matrices should allow any operation that works on full
matrices, and will either treat it specially, or implicitly convert
themselves to full matrices.

   Instances include matrix indexing, except for extracting a single
element or a leading submatrix, indexed assignment, or applying most
mapper functions, such as “exp”.

   An explicit conversion to a full matrix can be requested using the
built-in function “full”.  It should also be noted that the diagonal and
permutation matrix objects will cache the result of the conversion after
it is first requested (explicitly or implicitly), so that subsequent
conversions will be very cheap.


File: octave.info,  Node: Matrix Algebra,  Next: Function Support,  Prev: Basic Usage,  Up: Diagonal and Permutation Matrices

21.2 Linear Algebra with Diagonal/Permutation Matrices
======================================================

As has been already said, diagonal and permutation matrices make it
possible to use efficient algorithms while preserving natural linear
algebra syntax.  This section describes in detail the operations that
are treated specially when performed on these special matrix objects.

* Menu:

* Expressions Involving Diagonal Matrices::
* Expressions Involving Permutation Matrices::


File: octave.info,  Node: Expressions Involving Diagonal Matrices,  Next: Expressions Involving Permutation Matrices,  Up: Matrix Algebra

21.2.1 Expressions Involving Diagonal Matrices
----------------------------------------------

Assume D is a diagonal matrix.  If M is a full matrix, then ‘D*M’ will
scale the rows of M.  That means, if ‘S = D*M’, then for each pair of
indices i,j it holds

     S(i,j) = D(i,i) * M(i,j).

   Similarly, ‘M*D’ will do a column scaling.

   The matrix D may also be rectangular, m-by-n where ‘m != n’.  If ‘m <
n’, then the expression ‘D*M’ is equivalent to

     D(:,1:m) * M(1:m,:),

i.e., trailing ‘n-m’ rows of M are ignored.  If ‘m > n’, then ‘D*M’ is
equivalent to

     [D(1:n,:) * M; zeros(m-n, columns (M))],

i.e., null rows are appended to the result.  The situation for
right-multiplication ‘M*D’ is analogous.

   The expressions ‘D \ M’ and ‘M / D’ perform inverse scaling.  They
are equivalent to solving a diagonal (or rectangular diagonal) in a
least-squares minimum-norm sense.  In exact arithmetic, this is
equivalent to multiplying by a pseudoinverse.  The pseudoinverse of a
rectangular diagonal matrix is again a rectangular diagonal matrix with
swapped dimensions, where each nonzero diagonal element is replaced by
its reciprocal.  The matrix division algorithms do, in fact, use
division rather than multiplication by reciprocals for better numerical
accuracy; otherwise, they honor the above definition.  Note that a
diagonal matrix is never truncated due to ill-conditioning; otherwise,
it would not be of much use for scaling.  This is typically consistent
with linear algebra needs.  A full matrix that only happens to be
diagonal (and is thus not a special object) is of course treated
normally.

   Multiplication and division by diagonal matrices work efficiently
also when combined with sparse matrices, i.e., ‘D*S’, where D is a
diagonal matrix and S is a sparse matrix scales the rows of the sparse
matrix and returns a sparse matrix.  The expressions ‘S*D’, ‘D\S’, ‘S/D’
work analogically.

   If D1 and D2 are both diagonal matrices, then the expressions

     D1 + D2
     D1 - D2
     D1 * D2
     D1 / D2
     D1 \ D2

again produce diagonal matrices, provided that normal dimension matching
rules are obeyed.  The relations used are same as described above.

   Also, a diagonal matrix D can be multiplied or divided by a scalar,
or raised to a scalar power if it is square, producing diagonal matrix
result in all cases.

   A diagonal matrix can also be transposed or conjugate-transposed,
giving the expected result.  Extracting a leading submatrix of a
diagonal matrix, i.e., ‘D(1:m,1:n)’, will produce a diagonal matrix,
other indexing expressions will implicitly convert to full matrix.

   Adding a diagonal matrix to a full matrix only operates on the
diagonal elements.  Thus,

     A = A + eps * eye (n)

is an efficient method of augmenting the diagonal of a matrix.
Subtraction works analogically.

   When involved in expressions with other element-by-element operators,
‘.*’, ‘./’, ‘.\’ or ‘.^’, an implicit conversion to full matrix will
take place.  This is not always strictly necessary but chosen to
facilitate better consistency with MATLAB.


File: octave.info,  Node: Expressions Involving Permutation Matrices,  Prev: Expressions Involving Diagonal Matrices,  Up: Matrix Algebra

21.2.2 Expressions Involving Permutation Matrices
-------------------------------------------------

If P is a permutation matrix and M a matrix, the expression ‘P*M’ will
permute the rows of M.  Similarly, ‘M*P’ will yield a column
permutation.  Matrix division ‘P\M’ and ‘M/P’ can be used to do inverse
permutation.

   The previously described syntax for creating permutation matrices can
actually help an user to understand the connection between a permutation
matrix and a permuting vector.  Namely, the following holds, where ‘I =
eye (n)’ is an identity matrix:

       I(p,:) * M = (I*M) (p,:) = M(p,:)

   Similarly,

       M * I(:,p) = (M*I) (:,p) = M(:,p)

   The expressions ‘I(p,:)’ and ‘I(:,p)’ are permutation matrices.

   A permutation matrix can be transposed (or conjugate-transposed,
which is the same, because a permutation matrix is never complex),
inverting the permutation, or equivalently, turning a row-permutation
matrix into a column-permutation one.  For permutation matrices,
transpose is equivalent to inversion, thus ‘P\M’ is equivalent to
‘P'*M’.  Transpose of a permutation matrix (or inverse) is a
constant-time operation, flipping only a flag internally, and thus the
choice between the two above equivalent expressions for inverse
permuting is completely up to the user’s taste.

   Multiplication and division by permutation matrices works efficiently
also when combined with sparse matrices, i.e., ‘P*S’, where P is a
permutation matrix and S is a sparse matrix permutes the rows of the
sparse matrix and returns a sparse matrix.  The expressions ‘S*P’,
‘P\S’, ‘S/P’ work analogically.

   Two permutation matrices can be multiplied or divided (if their sizes
match), performing a composition of permutations.  Also a permutation
matrix can be indexed by a permutation vector (or two vectors), giving
again a permutation matrix.  Any other operations do not generally yield
a permutation matrix and will thus trigger the implicit conversion.


File: octave.info,  Node: Function Support,  Next: Example Code,  Prev: Matrix Algebra,  Up: Diagonal and Permutation Matrices

21.3 Functions That Are Aware of These Matrices
===============================================

This section lists the built-in functions that are aware of diagonal and
permutation matrices on input, or can return them as output.  Passed to
other functions, these matrices will in general trigger an implicit
conversion.  (Of course, user-defined dynamically linked functions may
also work with diagonal or permutation matrices).

* Menu:

* Diagonal Matrix Functions::
* Permutation Matrix Functions::


File: octave.info,  Node: Diagonal Matrix Functions,  Next: Permutation Matrix Functions,  Up: Function Support

21.3.1 Diagonal Matrix Functions
--------------------------------

“inv” and “pinv” can be applied to a diagonal matrix, yielding again a
diagonal matrix.  “det” will use an efficient straightforward
calculation when given a diagonal matrix, as well as “cond”.  The
following mapper functions can be applied to a diagonal matrix without
converting it to a full one: “abs”, “real”, “imag”, “conj”, “sqrt”.  A
diagonal matrix can also be returned from the “balance” and “svd”
functions.  The “sparse” function will convert a diagonal matrix
efficiently to a sparse matrix.


File: octave.info,  Node: Permutation Matrix Functions,  Prev: Diagonal Matrix Functions,  Up: Function Support

21.3.2 Permutation Matrix Functions
-----------------------------------

“inv” and “pinv” will invert a permutation matrix, preserving its
specialness.  “det” can be applied to a permutation matrix, efficiently
calculating the sign of the permutation (which is equal to the
determinant).

   A permutation matrix can also be returned from the built-in functions
“lu” and “qr”, if a pivoted factorization is requested.

   The “sparse” function will convert a permutation matrix efficiently
to a sparse matrix.  The “find” function will also work efficiently with
a permutation matrix, making it possible to conveniently obtain the
permutation indices.


File: octave.info,  Node: Example Code,  Next: Zeros Treatment,  Prev: Function Support,  Up: Diagonal and Permutation Matrices

21.4 Examples of Usage
======================

The following can be used to solve a linear system ‘A*x = b’ using the
pivoted LU factorization:

       [L, U, P] = lu (A); ## now L*U = P*A
       x = U \ (L \ P) * b;

This is one way to normalize columns of a matrix X to unit norm:

       s = norm (X, "columns");
       X /= diag (s);

The same can also be accomplished with broadcasting (*note
Broadcasting::):

       s = norm (X, "columns");
       X ./= s;

The following expression is a way to efficiently calculate the sign of a
permutation, given by a permutation vector P.  It will also work in
earlier versions of Octave, but slowly.

       det (eye (length (p))(p, :))

Finally, here’s how to solve a linear system ‘A*x = b’ with Tikhonov
regularization (ridge regression) using SVD (a skeleton only):

       m = rows (A); n = columns (A);
       [U, S, V] = svd (A);
       ## determine the regularization factor alpha
       ## alpha = ...
       ## transform to orthogonal basis
       b = U'*b;
       ## Use the standard formula, replacing A with S.
       ## S is diagonal, so the following will be very fast and accurate.
       x = (S'*S + alpha^2 * eye (n)) \ (S' * b);
       ## transform to solution basis
       x = V*x;


File: octave.info,  Node: Zeros Treatment,  Prev: Example Code,  Up: Diagonal and Permutation Matrices

21.5 Differences in Treatment of Zero Elements
==============================================

Making diagonal and permutation matrices special matrix objects in their
own right and the consequent usage of smarter algorithms for certain
operations implies, as a side effect, small differences in treating
zeros.  The contents of this section apply also to sparse matrices,
discussed in the following chapter.  (*note Sparse Matrices::)

   The IEEE floating point standard defines the result of the
expressions ‘0*Inf’ and ‘0*NaN’ as ‘NaN’.  This is widely agreed to be a
good compromise.  Numerical software dealing with structured and sparse
matrices (including Octave) however, almost always makes a distinction
between a "numerical zero" and an "assumed zero".  A "numerical zero" is
a zero value occurring in a place where any floating-point value could
occur.  It is normally stored somewhere in memory as an explicit value.
An "assumed zero", on the contrary, is a zero matrix element implied by
the matrix structure (diagonal, triangular) or a sparsity pattern; its
value is usually not stored explicitly anywhere, but is implied by the
underlying data structure.

   The primary distinction is that an assumed zero, when multiplied by
any number, or divided by any nonzero number, yields *always* a zero,
even when, e.g., multiplied by ‘Inf’ or divided by ‘NaN’.  The reason
for this behavior is that the numerical multiplication is not actually
performed anywhere by the underlying algorithm; the result is just
assumed to be zero.  Equivalently, one can say that the part of the
computation involving assumed zeros is performed symbolically, not
numerically.

   This behavior not only facilitates the most straightforward and
efficient implementation of algorithms, but also preserves certain
useful invariants, like:

   • scalar * diagonal matrix is a diagonal matrix

   • sparse matrix / scalar preserves the sparsity pattern

   • permutation matrix * matrix is equivalent to permuting rows

   all of these natural mathematical truths would be invalidated by
treating assumed zeros as numerical ones.

   Note that MATLAB does not strictly follow this principle and converts
assumed zeros to numerical zeros in certain cases, while not doing so in
other cases.  As of today, there are no intentions to mimic such
behavior in Octave.

   Examples of effects of assumed zeros vs.  numerical zeros:

     Inf * eye (3)
     ⇒
        Inf     0     0
          0   Inf     0
          0     0   Inf

     Inf * speye (3)
     ⇒
     Compressed Column Sparse (rows = 3, cols = 3, nnz = 3 [33%])

       (1, 1) -> Inf
       (2, 2) -> Inf
       (3, 3) -> Inf

     Inf * full (eye (3))
     ⇒
        Inf   NaN   NaN
        NaN   Inf   NaN
        NaN   NaN   Inf


     diag (1:3) * [NaN; 1; 1]
     ⇒
        NaN
          2
          3

     sparse (1:3,1:3,1:3) * [NaN; 1; 1]
     ⇒
        NaN
          2
          3
     [1,0,0;0,2,0;0,0,3] * [NaN; 1; 1]
     ⇒
        NaN
        NaN
        NaN


File: octave.info,  Node: Sparse Matrices,  Next: Numerical Integration,  Prev: Diagonal and Permutation Matrices,  Up: Top

22 Sparse Matrices
******************

* Menu:

* Basics::                      Creation and Manipulation of Sparse Matrices
* Sparse Linear Algebra::       Linear Algebra on Sparse Matrices
* Iterative Techniques::        Iterative Techniques
* Real Life Example::           Using Sparse Matrices


File: octave.info,  Node: Basics,  Next: Sparse Linear Algebra,  Up: Sparse Matrices

22.1 Creation and Manipulation of Sparse Matrices
=================================================

The size of mathematical problems that can be treated at any particular
time is generally limited by the available computing resources.  Both,
the speed of the computer and its available memory place limitation on
the problem size.

   There are many classes of mathematical problems which give rise to
matrices, where a large number of the elements are zero.  In this case
it makes sense to have a special matrix type to handle this class of
problems where only the nonzero elements of the matrix are stored.  Not
only does this reduce the amount of memory to store the matrix, but it
also means that operations on this type of matrix can take advantage of
the a priori knowledge of the positions of the nonzero elements to
accelerate their calculations.

   A matrix type that stores only the nonzero elements is generally
called sparse.  It is the purpose of this document to discuss the basics
of the storage and creation of sparse matrices and the fundamental
operations on them.

* Menu:

* Storage of Sparse Matrices::
* Creating Sparse Matrices::
* Information::
* Operators and Functions::


File: octave.info,  Node: Storage of Sparse Matrices,  Next: Creating Sparse Matrices,  Up: Basics

22.1.1 Storage of Sparse Matrices
---------------------------------

It is not strictly speaking necessary for the user to understand how
sparse matrices are stored.  However, such an understanding will help to
get an understanding of the size of sparse matrices.  Understanding the
storage technique is also necessary for those users wishing to create
their own oct-files.

   There are many different means of storing sparse matrix data.  What
all of the methods have in common is that they attempt to reduce the
complexity and storage given a priori knowledge of the particular class
of problems that will be solved.  A good summary of the available
techniques for storing sparse matrix is given by Saad (1).  With full
matrices, knowledge of the point of an element of the matrix within the
matrix is implied by its position in the computers memory.  However,
this is not the case for sparse matrices, and so the positions of the
nonzero elements of the matrix must equally be stored.

   An obvious way to do this is by storing the elements of the matrix as
triplets, with two elements being their position in the array (rows and
column) and the third being the data itself.  This is conceptually easy
to grasp, but requires more storage than is strictly needed.

   The storage technique used within Octave is the compressed column
format.  It is similar to the Yale format.  (2) In this format the
position of each element in a row and the data are stored as previously.
However, if we assume that all elements in the same column are stored
adjacent in the computers memory, then we only need to store information
on the number of nonzero elements in each column, rather than their
positions.  Thus assuming that the matrix has more nonzero elements than
there are columns in the matrix, we win in terms of the amount of memory
used.

   In fact, the column index contains one more element than the number
of columns, with the first element always being zero.  The advantage of
this is a simplification in the code, in that there is no special case
for the first or last columns.  A short example, demonstrating this in C
is.

       for (j = 0; j < nc; j++)
         for (i = cidx(j); i < cidx(j+1); i++)
            printf ("nonzero element (%i,%i) is %d\n",
                ridx(i), j, data(i));

   A clear understanding might be had by considering an example of how
the above applies to an example matrix.  Consider the matrix

         1   2   0  0
         0   0   0  3
         0   0   0  4

   The nonzero elements of this matrix are

        (1, 1)  ⇒ 1
        (1, 2)  ⇒ 2
        (2, 4)  ⇒ 3
        (3, 4)  ⇒ 4

   This will be stored as three vectors CIDX, RIDX and DATA,
representing the column indexing, row indexing and data respectively.
The contents of these three vectors for the above matrix will be

       CIDX = [0, 1, 2, 2, 4]
       RIDX = [0, 0, 1, 2]
       DATA = [1, 2, 3, 4]

   Note that this is the representation of these elements with the first
row and column assumed to start at zero, while in Octave itself the row
and column indexing starts at one.  Thus the number of elements in the
I-th column is given by ‘CIDX (I + 1) - CIDX (I)’.

   Although Octave uses a compressed column format, it should be noted
that compressed row formats are equally possible.  However, in the
context of mixed operations between mixed sparse and dense matrices, it
makes sense that the elements of the sparse matrices are in the same
order as the dense matrices.  Octave stores dense matrices in column
major ordering, and so sparse matrices are equally stored in this
manner.

   A further constraint on the sparse matrix storage used by Octave is
that all elements in the rows are stored in increasing order of their
row index, which makes certain operations faster.  However, it imposes
the need to sort the elements on the creation of sparse matrices.
Having disordered elements is potentially an advantage in that it makes
operations such as concatenating two sparse matrices together easier and
faster, however it adds complexity and speed problems elsewhere.

   ---------- Footnotes ----------

   (1) Y. Saad "SPARSKIT: A basic toolkit for sparse matrix
computation", 1994,
<http://www-users.cs.umn.edu/~saad/software/SPARSKIT/paper.ps>

   (2) <http://en.wikipedia.org/wiki/Sparse_matrix#Yale_format>


File: octave.info,  Node: Creating Sparse Matrices,  Next: Information,  Prev: Storage of Sparse Matrices,  Up: Basics

22.1.2 Creating Sparse Matrices
-------------------------------

There are several means to create sparse matrix.

Returned from a function
     There are many functions that directly return sparse matrices.
     These include “speye”, “sprand”, “diag”, etc.

Constructed from matrices or vectors
     The function “sparse” allows a sparse matrix to be constructed from
     three vectors representing the row, column and data.
     Alternatively, the function “spconvert” uses a three column matrix
     format to allow easy importation of data from elsewhere.

Created and then filled
     The function “sparse” or “spalloc” can be used to create an empty
     matrix that is then filled by the user

From a user binary program
     The user can directly create the sparse matrix within an oct-file.

   There are several basic functions to return specific sparse matrices.
For example the sparse identity matrix, is a matrix that is often
needed.  It therefore has its own function to create it as ‘speye (N)’
or ‘speye (R, C)’, which creates an N-by-N or R-by-C sparse identity
matrix.

   Another typical sparse matrix that is often needed is a random
distribution of random elements.  The functions “sprand” and “sprandn”
perform this for uniform and normal random distributions of elements.
They have exactly the same calling convention, where ‘sprand (R, C, D)’,
creates an R-by-C sparse matrix with a density of filled elements of D.

   Other functions of interest that directly create sparse matrices, are
“diag” or its generalization “spdiags”, that can take the definition of
the diagonals of the matrix and create the sparse matrix that
corresponds to this.  For example,

     s = diag (sparse (randn (1,n)), -1);

creates a sparse (N+1)-by-(N+1) sparse matrix with a single diagonal
defined.

 -- : B = spdiags (A)
 -- : [B, D] = spdiags (A)
 -- : B = spdiags (A, D)
 -- : A = spdiags (V, D, A)
 -- : A = spdiags (V, D, M, N)
     A generalization of the function ‘diag’.

     Called with a single input argument, the nonzero diagonals D of A
     are extracted.

     With two arguments the diagonals to extract are given by the vector
     D.

     The other two forms of ‘spdiags’ modify the input matrix by
     replacing the diagonals.  They use the columns of V to replace the
     diagonals represented by the vector D.  If the sparse matrix A is
     defined then the diagonals of this matrix are replaced.  Otherwise
     a matrix of M by N is created with the diagonals given by the
     columns of V.

     Negative values of D represent diagonals below the main diagonal,
     and positive values of D diagonals above the main diagonal.

     For example:

          spdiags (reshape (1:12, 4, 3), [-1 0 1], 5, 4)
             ⇒ 5 10  0  0
                1  6 11  0
                0  2  7 12
                0  0  3  8
                0  0  0  4

     See also: *note diag: XREFdiag.

 -- : S = speye (M, N)
 -- : S = speye (M)
 -- : S = speye (SZ)
     Return a sparse identity matrix of size MxN.

     The implementation is significantly more efficient than ‘sparse
     (eye (M))’ as the full matrix is not constructed.

     Called with a single argument a square matrix of size M-by-M is
     created.  If called with a single vector argument SZ, this argument
     is taken to be the size of the matrix to create.

     See also: *note sparse: XREFsparse, *note spdiags: XREFspdiags,
     *note eye: XREFeye.

 -- : R = spones (S)
     Replace the nonzero entries of S with ones.

     This creates a sparse matrix with the same structure as S.

     See also: *note sparse: XREFsparse, *note sprand: XREFsprand, *note
     sprandn: XREFsprandn, *note sprandsym: XREFsprandsym, *note spfun:
     XREFspfun, *note spy: XREFspy.

 -- : sprand (M, N, D)
 -- : sprand (M, N, D, RC)
 -- : sprand (S)
     Generate a sparse matrix with uniformly distributed random values.

     The size of the matrix is MxN with a density of values D.  D must
     be between 0 and 1.  Values will be uniformly distributed on the
     interval (0, 1).

     If called with a single matrix argument, a sparse matrix is
     generated with random values wherever the matrix S is nonzero.

     If called with a scalar fourth argument RC, a random sparse matrix
     with reciprocal condition number RC is generated.  If RC is a
     vector, then it specifies the first singular values of the
     generated matrix (‘length (RC) <= min (M, N)’).

     See also: *note sprandn: XREFsprandn, *note sprandsym:
     XREFsprandsym, *note rand: XREFrand.

 -- : sprandn (M, N, D)
 -- : sprandn (M, N, D, RC)
 -- : sprandn (S)
     Generate a sparse matrix with normally distributed random values.

     The size of the matrix is MxN with a density of values D.  D must
     be between 0 and 1.  Values will be normally distributed with a
     mean of 0 and a variance of 1.

     If called with a single matrix argument, a sparse matrix is
     generated with random values wherever the matrix S is nonzero.

     If called with a scalar fourth argument RC, a random sparse matrix
     with reciprocal condition number RC is generated.  If RC is a
     vector, then it specifies the first singular values of the
     generated matrix (‘length (RC) <= min (M, N)’).

     See also: *note sprand: XREFsprand, *note sprandsym: XREFsprandsym,
     *note randn: XREFrandn.

 -- : sprandsym (N, D)
 -- : sprandsym (S)
     Generate a symmetric random sparse matrix.

     The size of the matrix will be NxN, with a density of values given
     by D.  D must be between 0 and 1 inclusive.  Values will be
     normally distributed with a mean of zero and a variance of 1.

     If called with a single matrix argument, a random sparse matrix is
     generated wherever the matrix S is nonzero in its lower triangular
     part.

     See also: *note sprand: XREFsprand, *note sprandn: XREFsprandn,
     *note spones: XREFspones, *note sparse: XREFsparse.

   The recommended way for the user to create a sparse matrix, is to
create two vectors containing the row and column index of the data and a
third vector of the same size containing the data to be stored.  For
example,

       ri = ci = d = [];
       for j = 1:c
         ri = [ri; randperm(r,n)'];
         ci = [ci; j*ones(n,1)];
         d = [d; rand(n,1)];
       endfor
       s = sparse (ri, ci, d, r, c);

creates an R-by-C sparse matrix with a random distribution of N (<R)
elements per column.  The elements of the vectors do not need to be
sorted in any particular order as Octave will sort them prior to storing
the data.  However, pre-sorting the data will make the creation of the
sparse matrix faster.

   The function “spconvert” takes a three or four column real matrix.
The first two columns represent the row and column index respectively
and the third and four columns, the real and imaginary parts of the
sparse matrix.  The matrix can contain zero elements and the elements
can be sorted in any order.  Adding zero elements is a convenient way to
define the size of the sparse matrix.  For example:

     s = spconvert ([1 2 3 4; 1 3 4 4; 1 2 3 0]')
     ⇒ Compressed Column Sparse (rows=4, cols=4, nnz=3)
           (1 , 1) -> 1
           (2 , 3) -> 2
           (3 , 4) -> 3

   An example of creating and filling a matrix might be

     k = 5;
     nz = r * k;
     s = spalloc (r, c, nz)
     for j = 1:c
       idx = randperm (r);
       s (:, j) = [zeros(r - k, 1); ...
             rand(k, 1)] (idx);
     endfor

   It should be noted, that due to the way that the Octave assignment
functions are written that the assignment will reallocate the memory
used by the sparse matrix at each iteration of the above loop.
Therefore the “spalloc” function ignores the NZ argument and does not
pre-assign the memory for the matrix.  Therefore, it is vitally
important that code using to above structure should be vectorized as
much as possible to minimize the number of assignments and reduce the
number of memory allocations.

 -- : FM = full (SM)
     Return a full storage matrix from a sparse, diagonal, or
     permutation matrix, or a range.

     See also: *note sparse: XREFsparse, *note issparse: XREFissparse.

 -- : S = spalloc (M, N, NZ)
     Create an M-by-N sparse matrix with pre-allocated space for at most
     NZ nonzero elements.

     This is useful for building a matrix incrementally by a sequence of
     indexed assignments.  Subsequent indexed assignments after
     ‘spalloc’ will reuse the pre-allocated memory, provided they are of
     one of the simple forms

        • ‘S(I:J) = X’

        • ‘S(:,I:J) = X’

        • ‘S(K:L,I:J) = X’

     and that the following conditions are met:

        • the assignment does not decrease nnz (S).

        • after the assignment, nnz (S) does not exceed NZ.

        • no index is out of bounds.

     Partial movement of data may still occur, but in general the
     assignment will be more memory and time efficient under these
     circumstances.  In particular, it is possible to efficiently build
     a pre-allocated sparse matrix from a contiguous block of columns.

     The amount of pre-allocated memory for a given matrix may be
     queried using the function ‘nzmax’.

     See also: *note nzmax: XREFnzmax, *note sparse: XREFsparse.

 -- : S = sparse (A)
 -- : S = sparse (I, J, SV, M, N)
 -- : S = sparse (I, J, SV)
 -- : S = sparse (M, N)
 -- : S = sparse (I, J, S, M, N, "unique")
 -- : S = sparse (I, J, SV, M, N, NZMAX)
     Create a sparse matrix from a full matrix, or row, column, value
     triplets.

     If A is a full matrix, convert it to a sparse matrix
     representation, removing all zero values in the process.

     Given the integer index vectors I and J, and a 1-by-‘nnz’ vector of
     real or complex values SV, construct the sparse matrix
     ‘S(I(K),J(K)) = SV(K)’ with overall dimensions M and N.  If any of
     SV, I or J are scalars, they are expanded to have a common size.

     If M or N are not specified their values are derived from the
     maximum index in the vectors I and J as given by ‘M = max (I)’, ‘N
     = max (J)’.

     *Note*: if multiple values are specified with the same I, J
     indices, the corresponding value in S will be the sum of the values
     at the repeated location.  See ‘accumarray’ for an example of how
     to produce different behavior, such as taking the minimum instead.

     If the option "unique" is given, and more than one value is
     specified at the same I, J indices, then the last specified value
     will be used.

     ‘sparse (M, N)’ will create an empty MxN sparse matrix and is
     equivalent to ‘sparse ([], [], [], M, N)’

     The argument ‘nzmax’ is ignored but accepted for compatibility with
     MATLAB.

     Example 1 (sum at repeated indices):

          I = [1 1 2]; J = [1 1 2]; SV = [3 4 5];
          sparse (I, J, SV, 3, 4)
          ⇒
          Compressed Column Sparse (rows = 3, cols = 4, nnz = 2 [17%])

            (1, 1) ->  7
            (2, 2) ->  5

     Example 2 ("unique" option):

          I = [1 1 2]; J = [1 1 2]; SV = [3 4 5];
          sparse (I, J, SV, 3, 4, "unique")
          ⇒
          Compressed Column Sparse (rows = 3, cols = 4, nnz = 2 [17%])

            (1, 1) ->  4
            (2, 2) ->  5

     See also: *note full: XREFfull, *note accumarray: XREFaccumarray,
     *note spalloc: XREFspalloc, *note spdiags: XREFspdiags, *note
     speye: XREFspeye, *note spones: XREFspones, *note sprand:
     XREFsprand, *note sprandn: XREFsprandn, *note sprandsym:
     XREFsprandsym, *note spconvert: XREFspconvert, *note spfun:
     XREFspfun.

 -- : X = spconvert (M)
     Convert a simple sparse matrix format easily generated by other
     programs into Octave’s internal sparse format.

     The input M is either a 3 or 4 column real matrix, containing the
     row, column, real, and imaginary parts of the elements of the
     sparse matrix.  An element with a zero real and imaginary part can
     be used to force a particular matrix size.

     See also: *note sparse: XREFsparse.

   The above problem of memory reallocation can be avoided in oct-files.
However, the construction of a sparse matrix from an oct-file is more
complex than can be discussed here.  *Note External Code Interface::,
for a full description of the techniques involved.


File: octave.info,  Node: Information,  Next: Operators and Functions,  Prev: Creating Sparse Matrices,  Up: Basics

22.1.3 Finding Information about Sparse Matrices
------------------------------------------------

There are a number of functions that allow information concerning sparse
matrices to be obtained.  The most basic of these is “issparse” that
identifies whether a particular Octave object is in fact a sparse
matrix.

   Another very basic function is “nnz” that returns the number of
nonzero entries there are in a sparse matrix, while the function “nzmax”
returns the amount of storage allocated to the sparse matrix.  Note that
Octave tends to crop unused memory at the first opportunity for sparse
objects.  There are some cases of user created sparse objects where the
value returned by “nzmax” will not be the same as “nnz”, but in general
they will give the same result.  The function “spstats” returns some
basic statistics on the columns of a sparse matrix including the number
of elements, the mean and the variance of each column.

 -- : issparse (X)
     Return true if X is a sparse matrix.

     See also: *note ismatrix: XREFismatrix.

 -- : N = nnz (A)
     Return the number of nonzero elements in A.

     See also: *note nzmax: XREFnzmax, *note nonzeros: XREFnonzeros,
     *note find: XREFfind.

 -- : nonzeros (S)
     Return a vector of the nonzero values of the sparse matrix S.

     See also: *note find: XREFfind, *note nnz: XREFnnz.

 -- : N = nzmax (SM)
     Return the amount of storage allocated to the sparse matrix SM.

     Note that Octave tends to crop unused memory at the first
     opportunity for sparse objects.  Thus, in general the value of
     ‘nzmax’ will be the same as ‘nnz’ except for some cases of
     user-created sparse objects.

     See also: *note nnz: XREFnnz, *note spalloc: XREFspalloc, *note
     sparse: XREFsparse.

 -- : [COUNT, MEAN, VAR] = spstats (S)
 -- : [COUNT, MEAN, VAR] = spstats (S, J)
     Return the stats for the nonzero elements of the sparse matrix S.

     COUNT is the number of nonzeros in each column, MEAN is the mean of
     the nonzeros in each column, and VAR is the variance of the
     nonzeros in each column.

     Called with two input arguments, if S is the data and J is the bin
     number for the data, compute the stats for each bin.  In this case,
     bins can contain data values of zero, whereas with ‘spstats (S)’
     the zeros may disappear.

   When solving linear equations involving sparse matrices Octave
determines the means to solve the equation based on the type of the
matrix (*note Sparse Linear Algebra::).  Octave probes the matrix type
when the div (/) or ldiv (\) operator is first used with the matrix and
then caches the type.  However the “matrix_type” function can be used to
determine the type of the sparse matrix prior to use of the div or ldiv
operators.  For example,

     a = tril (sprandn (1024, 1024, 0.02), -1) ...
         + speye (1024);
     matrix_type (a);
     ans = Lower

shows that Octave correctly determines the matrix type for lower
triangular matrices.  “matrix_type” can also be used to force the type
of a matrix to be a particular type.  For example:

     a = matrix_type (tril (sprandn (1024, ...
        1024, 0.02), -1) + speye (1024), "Lower");

   This allows the cost of determining the matrix type to be avoided.
However, incorrectly defining the matrix type will result in incorrect
results from solutions of linear equations, and so it is entirely the
responsibility of the user to correctly identify the matrix type

   There are several graphical means of finding out information about
sparse matrices.  The first is the “spy” command, which displays the
structure of the nonzero elements of the matrix.  *Note Figure 22.1:
fig:spmatrix, for an example of the use of “spy”.  More advanced
graphical information can be obtained with the “treeplot”, “etreeplot”
and “gplot” commands.

             
                         |  * *                          
                         |  * * * *                      
                         |    * *   * *                  
                         |    *   *     * *              
                       5 -      *   *       * *          
                         |      *     *         * *      
                         |        *     *           * *  
                         |        *       *             *
                         |          *       *            
                      10 -          *         *          
                         |            *         *        
                         |            *           *      
                         |              *           *    
                         |              *             *  
                      15 -                *             *
                         |----------|---------|---------|
                                    5        10        15

Figure 22.1: Structure of simple sparse matrix.

   One use of sparse matrices is in graph theory, where the
interconnections between nodes are represented as an adjacency matrix.
That is, if the i-th node in a graph is connected to the j-th node.
Then the ij-th node (and in the case of undirected graphs the ji-th
node) of the sparse adjacency matrix is nonzero.  If each node is then
associated with a set of coordinates, then the “gplot” command can be
used to graphically display the interconnections between nodes.

   As a trivial example of the use of “gplot” consider the example,

     A = sparse ([2,6,1,3,2,4,3,5,4,6,1,5],
         [1,1,2,2,3,3,4,4,5,5,6,6],1,6,6);
     xy = [0,4,8,6,4,2;5,0,5,7,5,7]';
     gplot (A,xy)

which creates an adjacency matrix ‘A’ where node 1 is connected to nodes
2 and 6, node 2 with nodes 1 and 3, etc.  The coordinates of the nodes
are given in the n-by-2 matrix ‘xy’.

   The dependencies between the nodes of a Cholesky factorization can be
calculated in linear time without explicitly needing to calculate the
Cholesky factorization by the ‘etree’ command.  This command returns the
elimination tree of the matrix and can be displayed graphically by the
command ‘treeplot (etree (A))’ if ‘A’ is symmetric or ‘treeplot (etree
(A+A'))’ otherwise.

 -- : spy (X)
 -- : spy (..., MARKERSIZE)
 -- : spy (..., LINE_SPEC)
     Plot the sparsity pattern of the sparse matrix X.

     If the argument MARKERSIZE is given as a scalar value, it is used
     to determine the point size in the plot.

     If the string LINE_SPEC is given it is passed to ‘plot’ and
     determines the appearance of the plot.

     See also: *note plot: XREFplot, *note gplot: XREFgplot.

 -- : P = etree (S)
 -- : P = etree (S, TYP)
 -- : [P, Q] = etree (S, TYP)

     Return the elimination tree for the matrix S.

     By default S is assumed to be symmetric and the symmetric
     elimination tree is returned.  The argument TYP controls whether a
     symmetric or column elimination tree is returned.  Valid values of
     TYP are "sym" or "col", for symmetric or column elimination tree
     respectively.

     Called with a second argument, ‘etree’ also returns the postorder
     permutations on the tree.

 -- : etreeplot (A)
 -- : etreeplot (A, NODE_STYLE, EDGE_STYLE)
     Plot the elimination tree of the matrix A or A+A’ if A in not
     symmetric.

     The optional parameters NODE_STYLE and EDGE_STYLE define the output
     style.

     See also: *note treeplot: XREFtreeplot, *note gplot: XREFgplot.

 -- : gplot (A, XY)
 -- : gplot (A, XY, LINE_STYLE)
 -- : [X, Y] = gplot (A, XY)
     Plot a graph defined by A and XY in the graph theory sense.

     A is the adjacency matrix of the array to be plotted and XY is an
     N-by-2 matrix containing the coordinates of the nodes of the graph.

     The optional parameter LINE_STYLE defines the output style for the
     plot.  Called with no output arguments the graph is plotted
     directly.  Otherwise, return the coordinates of the plot in X and
     Y.

     See also: *note treeplot: XREFtreeplot, *note etreeplot:
     XREFetreeplot, *note spy: XREFspy.

 -- : treeplot (TREE)
 -- : treeplot (TREE, NODE_STYLE, EDGE_STYLE)
     Produce a graph of tree or forest.

     The first argument is vector of predecessors.

     The optional parameters NODE_STYLE and EDGE_STYLE define the output
     plot style.

     The complexity of the algorithm is O(n) in terms of is time and
     memory requirements.

     See also: *note etreeplot: XREFetreeplot, *note gplot: XREFgplot.

 -- : treelayout (TREE)
 -- : treelayout (TREE, PERMUTATION)
     treelayout lays out a tree or a forest.

     The first argument TREE is a vector of predecessors.

     The parameter PERMUTATION is an optional postorder permutation.

     The complexity of the algorithm is O(n) in terms of time and memory
     requirements.

     See also: *note etreeplot: XREFetreeplot, *note gplot: XREFgplot,
     *note treeplot: XREFtreeplot.


File: octave.info,  Node: Operators and Functions,  Prev: Information,  Up: Basics

22.1.4 Basic Operators and Functions on Sparse Matrices
-------------------------------------------------------

* Menu:

* Sparse Functions::
* Return Types of Operators and Functions::
* Mathematical Considerations::


File: octave.info,  Node: Sparse Functions,  Next: Return Types of Operators and Functions,  Up: Operators and Functions

22.1.4.1 Sparse Functions
.........................

Many Octave functions have been overloaded to work with either sparse or
full matrices.  There is no difference in calling convention when using
an overloaded function with a sparse matrix, however, there is also no
access to potentially sparse-specific features.  At any time the sparse
matrix specific version of a function can be used by explicitly calling
its function name.

   The table below lists all of the sparse functions of Octave.  Note
that the names of the specific sparse forms of the functions are
typically the same as the general versions with a “sp” prefix.  In the
table below, and in the rest of this article, the specific sparse
versions of functions are used.

Generate sparse matrices:
     “spalloc”, “spdiags”, “speye”, “sprand”, “sprandn”, “sprandsym”

Sparse matrix conversion:
     “full”, “sparse”, “spconvert”

Manipulate sparse matrices
     “issparse”, “nnz”, “nonzeros”, “nzmax”, “spfun”, “spones”, “spy”

Graph Theory:
     “etree”, “etreeplot”, “gplot”, “treeplot”

Sparse matrix reordering:
     “amd”, “ccolamd”, “colamd”, “colperm”, “csymamd”, “dmperm”,
     “symamd”, “randperm”, “symrcm”

Linear algebra:
     “condest”, “eigs”, “matrix_type”, “normest”, “normest1”, “sprank”,
     “spaugment”, “svds”

Iterative techniques:
     “ichol”, “ilu”, “pcg”, “pcr”

Miscellaneous:
     “spparms”, “symbfact”, “spstats”

   In addition all of the standard Octave mapper functions (i.e., basic
math functions that take a single argument) such as “abs”, etc.  can
accept sparse matrices.  The reader is referred to the documentation
supplied with these functions within Octave itself for further details.


File: octave.info,  Node: Return Types of Operators and Functions,  Next: Mathematical Considerations,  Prev: Sparse Functions,  Up: Operators and Functions

22.1.4.2 Return Types of Operators and Functions
................................................

The two basic reasons to use sparse matrices are to reduce the memory
usage and to not have to do calculations on zero elements.  The two are
closely related in that the computation time on a sparse matrix operator
or function is roughly linear with the number of nonzero elements.

   Therefore, there is a certain density of nonzero elements of a matrix
where it no longer makes sense to store it as a sparse matrix, but
rather as a full matrix.  For this reason operators and functions that
have a high probability of returning a full matrix will always return
one.  For example adding a scalar constant to a sparse matrix will
almost always make it a full matrix, and so the example,

     speye (3) + 0
     ⇒   1  0  0
       0  1  0
       0  0  1

returns a full matrix as can be seen.

   Additionally, if ‘sparse_auto_mutate’ is true, all sparse functions
test the amount of memory occupied by the sparse matrix to see if the
amount of storage used is larger than the amount used by the full
equivalent.  Therefore ‘speye (2) * 1’ will return a full matrix as the
memory used is smaller for the full version than the sparse version.

   As all of the mixed operators and functions between full and sparse
matrices exist, in general this does not cause any problems.  However,
one area where it does cause a problem is where a sparse matrix is
promoted to a full matrix, where subsequent operations would resparsify
the matrix.  Such cases are rare, but can be artificially created, for
example ‘(fliplr (speye (3)) + speye (3)) - speye (3)’ gives a full
matrix when it should give a sparse one.  In general, where such cases
occur, they impose only a small memory penalty.

   There is however one known case where this behavior of Octave’s
sparse matrices will cause a problem.  That is in the handling of the
“diag” function.  Whether “diag” returns a sparse or full matrix
depending on the type of its input arguments.  So

      a = diag (sparse ([1,2,3]), -1);

should return a sparse matrix.  To ensure this actually happens, the
“sparse” function, and other functions based on it like “speye”, always
returns a sparse matrix, even if the memory used will be larger than its
full representation.

 -- : VAL = sparse_auto_mutate ()
 -- : OLD_VAL = sparse_auto_mutate (NEW_VAL)
 -- : sparse_auto_mutate (NEW_VAL, "local")
     Query or set the internal variable that controls whether Octave
     will automatically mutate sparse matrices to full matrices to save
     memory.

     For example:

          s = speye (3);
          sparse_auto_mutate (false);
          s(:, 1) = 1;
          typeinfo (s)
          ⇒ sparse matrix
          sparse_auto_mutate (true);
          s(1, :) = 1;
          typeinfo (s)
          ⇒ matrix

     When called from inside a function with the "local" option, the
     variable is changed locally for the function and any subroutines it
     calls.  The original variable value is restored when exiting the
     function.

   Note that the ‘sparse_auto_mutate’ option is incompatible with
MATLAB, and so it is off by default.


File: octave.info,  Node: Mathematical Considerations,  Prev: Return Types of Operators and Functions,  Up: Operators and Functions

22.1.4.3 Mathematical Considerations
....................................

The attempt has been made to make sparse matrices behave in exactly the
same manner as there full counterparts.  However, there are certain
differences and especially differences with other products sparse
implementations.

   First, the "./" and ".^" operators must be used with care.  Consider
what the examples

       s = speye (4);
       a1 = s .^ 2;
       a2 = s .^ s;
       a3 = s .^ -2;
       a4 = s ./ 2;
       a5 = 2 ./ s;
       a6 = s ./ s;

will give.  The first example of S raised to the power of 2 causes no
problems.  However S raised element-wise to itself involves a large
number of terms ‘0 .^ 0’ which is 1.  There ‘S .^ S’ is a full matrix.

   Likewise ‘S .^ -2’ involves terms like ‘0 .^ -2’ which is infinity,
and so ‘S .^ -2’ is equally a full matrix.

   For the "./" operator ‘S ./ 2’ has no problems, but ‘2 ./ S’ involves
a large number of infinity terms as well and is equally a full matrix.
The case of ‘S ./ S’ involves terms like ‘0 ./ 0’ which is a ‘NaN’ and
so this is equally a full matrix with the zero elements of S filled with
‘NaN’ values.

   The above behavior is consistent with full matrices, but is not
consistent with sparse implementations in other products.

   A particular problem of sparse matrices comes about due to the fact
that as the zeros are not stored, the sign-bit of these zeros is equally
not stored.  In certain cases the sign-bit of zero is important.  For
example:

      a = 0 ./ [-1, 1; 1, -1];
      b = 1 ./ a
      ⇒ -Inf            Inf
          Inf           -Inf
      c = 1 ./ sparse (a)
      ⇒  Inf            Inf
          Inf            Inf

   To correct this behavior would mean that zero elements with a
negative sign-bit would need to be stored in the matrix to ensure that
their sign-bit was respected.  This is not done at this time, for
reasons of efficiency, and so the user is warned that calculations where
the sign-bit of zero is important must not be done using sparse
matrices.

   In general any function or operator used on a sparse matrix will
result in a sparse matrix with the same or a larger number of nonzero
elements than the original matrix.  This is particularly true for the
important case of sparse matrix factorizations.  The usual way to
address this is to reorder the matrix, such that its factorization is
sparser than the factorization of the original matrix.  That is the
factorization of ‘L * U = P * S * Q’ has sparser terms ‘L’ and ‘U’ than
the equivalent factorization ‘L * U = S’.

   Several functions are available to reorder depending on the type of
the matrix to be factorized.  If the matrix is symmetric
positive-definite, then “symamd” or “csymamd” should be used.  Otherwise
“amd”, “colamd” or “ccolamd” should be used.  For completeness the
reordering functions “colperm” and “randperm” are also available.

   *Note Figure 22.2: fig:simplematrix, for an example of the structure
of a simple positive definite matrix.

             
                         |  * *                          
                         |  * * * *                      
                         |    * *   * *                  
                         |    *   *     * *              
                       5 -      *   *       * *          
                         |      *     *         * *      
                         |        *     *           * *  
                         |        *       *             *
                         |          *       *            
                      10 -          *         *          
                         |            *         *        
                         |            *           *      
                         |              *           *    
                         |              *             *  
                      15 -                *             *
                         |----------|---------|---------|
                                    5        10        15

Figure 22.2: Structure of simple sparse matrix.

   The standard Cholesky factorization of this matrix can be obtained by
the same command that would be used for a full matrix.  This can be
visualized with the command ‘r = chol (A); spy (r);’.  *Note Figure
22.3: fig:simplechol.  The original matrix had 43 nonzero terms, while
this Cholesky factorization has 71, with only half of the symmetric
matrix being stored.  This is a significant level of fill in, and
although not an issue for such a small test case, can represents a large
overhead in working with other sparse matrices.

   The appropriate sparsity preserving permutation of the original
matrix is given by “symamd” and the factorization using this reordering
can be visualized using the command ‘q = symamd (A); r = chol (A(q,q));
spy (r)’.  This gives 29 nonzero terms which is a significant
improvement.

   The Cholesky factorization itself can be used to determine the
appropriate sparsity preserving reordering of the matrix during the
factorization, In that case this might be obtained with three return
arguments as ‘[r, p, q] = chol (A); spy (r)’.

             
                         |  * *                          
                         |    * * *                      
                         |      * * * *                  
                         |        * * * * *              
                       5 -          * * * * * *          
                         |            * * * * * * *      
                         |              * * * * * * * *  
                         |                * * * * * * * *
                         |                  * * * * * * *
                      10 -                    * * * * * *
                         |                      * * * * *
                         |                        * * * *
                         |                          * * *
                         |                            * *
                      15 -                              *
                         |----------|---------|---------|
                                    5        10        15

Figure 22.3: Structure of the unpermuted Cholesky factorization of the
above matrix.

             
                         |  * *                          
                         |    *       *                  
                         |      *   *                    
                         |        * *                    
                       5 -          * *                  
                         |            *                 *
                         |              *   *            
                         |                * *            
                         |                  *       *    
                      10 -                    *   *      
                         |                      * *      
                         |                        * *    
                         |                          *   *
                         |                            * *
                      15 -                              *
                         |----------|---------|---------|
                                    5        10        15

Figure 22.4: Structure of the permuted Cholesky factorization of the
above matrix.

   In the case of an asymmetric matrix, the appropriate sparsity
preserving permutation is “colamd” and the factorization using this
reordering can be visualized using the command ‘q = colamd (A); [l, u,
p] = lu (A(:,q)); spy (l+u)’.

   Finally, Octave implicitly reorders the matrix when using the div (/)
and ldiv (\) operators, and so no the user does not need to explicitly
reorder the matrix to maximize performance.

 -- : P = amd (S)
 -- : P = amd (S, OPTS)

     Return the approximate minimum degree permutation of a matrix.

     This is a permutation such that the Cholesky factorization of ‘S
     (P, P)’ tends to be sparser than the Cholesky factorization of S
     itself.  ‘amd’ is typically faster than ‘symamd’ but serves a
     similar purpose.

     The optional parameter OPTS is a structure that controls the
     behavior of ‘amd’.  The fields of the structure are

     OPTS.dense
          Determines what ‘amd’ considers to be a dense row or column of
          the input matrix.  Rows or columns with more than ‘max (16,
          (dense * sqrt (N)))’ entries, where N is the order of the
          matrix S, are ignored by ‘amd’ during the calculation of the
          permutation.  The value of dense must be a positive scalar and
          the default value is 10.0

     OPTS.aggressive
          If this value is a nonzero scalar, then ‘amd’ performs
          aggressive absorption.  The default is not to perform
          aggressive absorption.

     The author of the code itself is Timothy A. Davis
     <davis@cise.ufl.edu>, University of Florida (see
     <http://www.cise.ufl.edu/research/sparse/amd>).

     See also: *note symamd: XREFsymamd, *note colamd: XREFcolamd.

 -- : P = ccolamd (S)
 -- : P = ccolamd (S, KNOBS)
 -- : P = ccolamd (S, KNOBS, CMEMBER)
 -- : [P, STATS] = ccolamd (...)

     Constrained column approximate minimum degree permutation.

     ‘P = ccolamd (S)’ returns the column approximate minimum degree
     permutation vector for the sparse matrix S.  For a non-symmetric
     matrix S, ‘S(:, P)’ tends to have sparser LU factors than S.  ‘chol
     (S(:, P)' * S(:, P))’ also tends to be sparser than ‘chol (S' *
     S)’.  ‘P = ccolamd (S, 1)’ optimizes the ordering for ‘lu (S(:,
     P))’.  The ordering is followed by a column elimination tree
     post-ordering.

     KNOBS is an optional 1-element to 5-element input vector, with a
     default value of ‘[0 10 10 1 0]’ if not present or empty.  Entries
     not present are set to their defaults.

     ‘KNOBS(1)’
          if nonzero, the ordering is optimized for ‘lu (S(:, p))’.  It
          will be a poor ordering for ‘chol (S(:, P)' * S(:, P))’.  This
          is the most important knob for ccolamd.

     ‘KNOBS(2)’
          if S is m-by-n, rows with more than ‘max (16, KNOBS(2) * sqrt
          (n))’ entries are ignored.

     ‘KNOBS(3)’
          columns with more than ‘max (16, KNOBS(3) * sqrt (min (M,
          N)))’ entries are ignored and ordered last in the output
          permutation (subject to the cmember constraints).

     ‘KNOBS(4)’
          if nonzero, aggressive absorption is performed.

     ‘KNOBS(5)’
          if nonzero, statistics and knobs are printed.

     CMEMBER is an optional vector of length n.  It defines the
     constraints on the column ordering.  If ‘CMEMBER(j) = C’, then
     column J is in constraint set C (C must be in the range 1 to n).
     In the output permutation P, all columns in set 1 appear first,
     followed by all columns in set 2, and so on.  ‘CMEMBER = ones
     (1,n)’ if not present or empty.  ‘ccolamd (S, [], 1 : n)’ returns
     ‘1 : n’

     ‘P = ccolamd (S)’ is about the same as ‘P = colamd (S)’.  KNOBS and
     its default values differ.  ‘colamd’ always does aggressive
     absorption, and it finds an ordering suitable for both ‘lu (S(:,
     P))’ and ‘chol (S(:, P)' * S(:, P))’; it cannot optimize its
     ordering for ‘lu (S(:, P))’ to the extent that ‘ccolamd (S, 1)’
     can.

     STATS is an optional 20-element output vector that provides data
     about the ordering and the validity of the input matrix S.
     Ordering statistics are in ‘STATS(1 : 3)’.  ‘STATS(1)’ and
     ‘STATS(2)’ are the number of dense or empty rows and columns
     ignored by CCOLAMD and ‘STATS(3)’ is the number of garbage
     collections performed on the internal data structure used by
     CCOLAMD (roughly of size ‘2.2 * nnz (S) + 4 * M + 7 * N’ integers).

     ‘STATS(4 : 7)’ provide information if CCOLAMD was able to continue.
     The matrix is OK if ‘STATS(4)’ is zero, or 1 if invalid.
     ‘STATS(5)’ is the rightmost column index that is unsorted or
     contains duplicate entries, or zero if no such column exists.
     ‘STATS(6)’ is the last seen duplicate or out-of-order row index in
     the column index given by ‘STATS(5)’, or zero if no such row index
     exists.  ‘STATS(7)’ is the number of duplicate or out-of-order row
     indices.  ‘STATS(8 : 20)’ is always zero in the current version of
     CCOLAMD (reserved for future use).

     The authors of the code itself are S. Larimore, T. Davis (Univ.  of
     Florida) and S. Rajamanickam in collaboration with J. Bilbert and
     E. Ng.  Supported by the National Science Foundation (DMS-9504974,
     DMS-9803599, CCR-0203270), and a grant from Sandia National Lab.
     See <http://www.cise.ufl.edu/research/sparse> for ccolamd, csymamd,
     amd, colamd, symamd, and other related orderings.

     See also: *note colamd: XREFcolamd, *note csymamd: XREFcsymamd.

 -- : P = colamd (S)
 -- : P = colamd (S, KNOBS)
 -- : [P, STATS] = colamd (S)
 -- : [P, STATS] = colamd (S, KNOBS)

     Compute the column approximate minimum degree permutation.

     ‘P = colamd (S)’ returns the column approximate minimum degree
     permutation vector for the sparse matrix S.  For a non-symmetric
     matrix S, ‘S(:,P)’ tends to have sparser LU factors than S.  The
     Cholesky factorization of ‘S(:,P)' * S(:,P)’ also tends to be
     sparser than that of ‘S' * S’.

     KNOBS is an optional one- to three-element input vector.  If S is
     m-by-n, then rows with more than ‘max(16,KNOBS(1)*sqrt(n))’ entries
     are ignored.  Columns with more than ‘max
     (16,KNOBS(2)*sqrt(min(m,n)))’ entries are removed prior to
     ordering, and ordered last in the output permutation P.  Only
     completely dense rows or columns are removed if ‘KNOBS(1)’ and
     ‘KNOBS(2)’ are < 0, respectively.  If ‘KNOBS(3)’ is nonzero, STATS
     and KNOBS are printed.  The default is ‘KNOBS = [10 10 0]’.  Note
     that KNOBS differs from earlier versions of colamd.

     STATS is an optional 20-element output vector that provides data
     about the ordering and the validity of the input matrix S.
     Ordering statistics are in ‘STATS(1:3)’.  ‘STATS(1)’ and ‘STATS(2)’
     are the number of dense or empty rows and columns ignored by COLAMD
     and ‘STATS(3)’ is the number of garbage collections performed on
     the internal data structure used by COLAMD (roughly of size ‘2.2 *
     nnz(S) + 4 * M + 7 * N’ integers).

     Octave built-in functions are intended to generate valid sparse
     matrices, with no duplicate entries, with ascending row indices of
     the nonzeros in each column, with a non-negative number of entries
     in each column (!)  and so on.  If a matrix is invalid, then COLAMD
     may or may not be able to continue.  If there are duplicate entries
     (a row index appears two or more times in the same column) or if
     the row indices in a column are out of order, then COLAMD can
     correct these errors by ignoring the duplicate entries and sorting
     each column of its internal copy of the matrix S (the input matrix
     S is not repaired, however).  If a matrix is invalid in other ways
     then COLAMD cannot continue, an error message is printed, and no
     output arguments (P or STATS) are returned.  COLAMD is thus a
     simple way to check a sparse matrix to see if it’s valid.

     ‘STATS(4:7)’ provide information if COLAMD was able to continue.
     The matrix is OK if ‘STATS(4)’ is zero, or 1 if invalid.
     ‘STATS(5)’ is the rightmost column index that is unsorted or
     contains duplicate entries, or zero if no such column exists.
     ‘STATS(6)’ is the last seen duplicate or out-of-order row index in
     the column index given by ‘STATS(5)’, or zero if no such row index
     exists.  ‘STATS(7)’ is the number of duplicate or out-of-order row
     indices.  ‘STATS(8:20)’ is always zero in the current version of
     COLAMD (reserved for future use).

     The ordering is followed by a column elimination tree
     post-ordering.

     The authors of the code itself are Stefan I. Larimore and Timothy
     A. Davis <davis@cise.ufl.edu>, University of Florida.  The
     algorithm was developed in collaboration with John Gilbert, Xerox
     PARC, and Esmond Ng, Oak Ridge National Laboratory.  (see
     <http://www.cise.ufl.edu/research/sparse/colamd>)

     See also: *note colperm: XREFcolperm, *note symamd: XREFsymamd,
     *note ccolamd: XREFccolamd.

 -- : P = colperm (S)
     Return the column permutations such that the columns of ‘S (:, P)’
     are ordered in terms of increasing number of nonzero elements.

     If S is symmetric, then P is chosen such that ‘S (P, P)’ orders the
     rows and columns with increasing number of nonzeros elements.

 -- : P = csymamd (S)
 -- : P = csymamd (S, KNOBS)
 -- : P = csymamd (S, KNOBS, CMEMBER)
 -- : [P, STATS] = csymamd (...)

     For a symmetric positive definite matrix S, return the permutation
     vector P such that ‘S(P,P)’ tends to have a sparser Cholesky factor
     than S.

     Sometimes ‘csymamd’ works well for symmetric indefinite matrices
     too.  The matrix S is assumed to be symmetric; only the strictly
     lower triangular part is referenced.  S must be square.  The
     ordering is followed by an elimination tree post-ordering.

     KNOBS is an optional 1-element to 3-element input vector, with a
     default value of ‘[10 1 0]’.  Entries not present are set to their
     defaults.

     ‘KNOBS(1)’
          If S is n-by-n, then rows and columns with more than
          ‘max(16,KNOBS(1)*sqrt(n))’ entries are ignored, and ordered
          last in the output permutation (subject to the cmember
          constraints).

     ‘KNOBS(2)’
          If nonzero, aggressive absorption is performed.

     ‘KNOBS(3)’
          If nonzero, statistics and knobs are printed.

     CMEMBER is an optional vector of length n.  It defines the
     constraints on the ordering.  If ‘CMEMBER(j) = S’, then row/column
     j is in constraint set C (C must be in the range 1 to n).  In the
     output permutation P, rows/columns in set 1 appear first, followed
     by all rows/columns in set 2, and so on.  ‘CMEMBER = ones (1,n)’ if
     not present or empty.  ‘csymamd (S,[],1:n)’ returns ‘1:n’.

     ‘P = csymamd (S)’ is about the same as ‘P = symamd (S)’.  KNOBS and
     its default values differ.

     ‘STATS(4:7)’ provide information if CCOLAMD was able to continue.
     The matrix is OK if ‘STATS(4)’ is zero, or 1 if invalid.
     ‘STATS(5)’ is the rightmost column index that is unsorted or
     contains duplicate entries, or zero if no such column exists.
     ‘STATS(6)’ is the last seen duplicate or out-of-order row index in
     the column index given by ‘STATS(5)’, or zero if no such row index
     exists.  ‘STATS(7)’ is the number of duplicate or out-of-order row
     indices.  ‘STATS(8:20)’ is always zero in the current version of
     CCOLAMD (reserved for future use).

     The authors of the code itself are S. Larimore, T. Davis (Univ.  of
     Florida) and S. Rajamanickam in collaboration with J. Bilbert and
     E. Ng.  Supported by the National Science Foundation (DMS-9504974,
     DMS-9803599, CCR-0203270), and a grant from Sandia National Lab.
     See <http://www.cise.ufl.edu/research/sparse> for ccolamd, csymamd,
     amd, colamd, symamd, and other related orderings.

     See also: *note symamd: XREFsymamd, *note ccolamd: XREFccolamd.

 -- : P = dmperm (S)
 -- : [P, Q, R, S] = dmperm (S)

     Perform a Dulmage-Mendelsohn permutation of the sparse matrix S.

     With a single output argument ‘dmperm’ performs the row
     permutations P such that ‘S(P,:)’ has no zero elements on the
     diagonal.

     Called with two or more output arguments, returns the row and
     column permutations, such that ‘S(P, Q)’ is in block triangular
     form.  The values of R and S define the boundaries of the blocks.
     If S is square then ‘R == S’.

     The method used is described in: A. Pothen & C.-J. Fan.  ‘Computing
     the Block Triangular Form of a Sparse Matrix’.  ACM Trans.  Math.
     Software, 16(4):303-324, 1990.

     See also: *note colamd: XREFcolamd, *note ccolamd: XREFccolamd.

 -- : P = symamd (S)
 -- : P = symamd (S, KNOBS)
 -- : [P, STATS] = symamd (S)
 -- : [P, STATS] = symamd (S, KNOBS)

     For a symmetric positive definite matrix S, returns the permutation
     vector p such that ‘S(P, P)’ tends to have a sparser
     Cholesky factor than S.

     Sometimes ‘symamd’ works well for symmetric indefinite matrices
     too.  The matrix S is assumed to be symmetric; only the strictly
     lower triangular part is referenced.  S must be square.

     KNOBS is an optional one- to two-element input vector.  If S is
     n-by-n, then rows and columns with more than ‘max
     (16,KNOBS(1)*sqrt(n))’ entries are removed prior to ordering, and
     ordered last in the output permutation P.  No rows/columns are
     removed if ‘KNOBS(1) < 0’.  If ‘KNOBS (2)’ is nonzero, ‘stats’ and
     KNOBS are printed.  The default is ‘KNOBS = [10 0]’.  Note that
     KNOBS differs from earlier versions of ‘symamd’.

     STATS is an optional 20-element output vector that provides data
     about the ordering and the validity of the input matrix S.
     Ordering statistics are in ‘STATS(1:3)’.  ‘STATS(1) = STATS(2)’ is
     the number of dense or empty rows and columns ignored by SYMAMD and
     ‘STATS(3)’ is the number of garbage collections performed on the
     internal data structure used by SYMAMD (roughly of size ‘8.4 * nnz
     (tril (S, -1)) + 9 * N’ integers).

     Octave built-in functions are intended to generate valid sparse
     matrices, with no duplicate entries, with ascending row indices of
     the nonzeros in each column, with a non-negative number of entries
     in each column (!)  and so on.  If a matrix is invalid, then SYMAMD
     may or may not be able to continue.  If there are duplicate entries
     (a row index appears two or more times in the same column) or if
     the row indices in a column are out of order, then SYMAMD can
     correct these errors by ignoring the duplicate entries and sorting
     each column of its internal copy of the matrix S (the input matrix
     S is not repaired, however).  If a matrix is invalid in other ways
     then SYMAMD cannot continue, an error message is printed, and no
     output arguments (P or STATS) are returned.  SYMAMD is thus a
     simple way to check a sparse matrix to see if it’s valid.

     ‘STATS(4:7)’ provide information if SYMAMD was able to continue.
     The matrix is OK if ‘STATS (4)’ is zero, or 1 if invalid.
     ‘STATS(5)’ is the rightmost column index that is unsorted or
     contains duplicate entries, or zero if no such column exists.
     ‘STATS(6)’ is the last seen duplicate or out-of-order row index in
     the column index given by ‘STATS(5)’, or zero if no such row index
     exists.  ‘STATS(7)’ is the number of duplicate or out-of-order row
     indices.  ‘STATS(8:20)’ is always zero in the current version of
     SYMAMD (reserved for future use).

     The ordering is followed by a column elimination tree
     post-ordering.

     The authors of the code itself are Stefan I. Larimore and Timothy
     A. Davis <davis@cise.ufl.edu>, University of Florida.  The
     algorithm was developed in collaboration with John Gilbert, Xerox
     PARC, and Esmond Ng, Oak Ridge National Laboratory.  (see
     <http://www.cise.ufl.edu/research/sparse/colamd>)

     See also: *note colperm: XREFcolperm, *note colamd: XREFcolamd.

 -- : P = symrcm (S)
     Return the symmetric reverse Cuthill-McKee permutation of S.

     P is a permutation vector such that ‘S(P, P)’ tends to have its
     diagonal elements closer to the diagonal than S.  This is a good
     preordering for LU or Cholesky factorization of matrices that come
     from “long, skinny” problems.  It works for both symmetric and
     asymmetric S.

     The algorithm represents a heuristic approach to the NP-complete
     bandwidth minimization problem.  The implementation is based in the
     descriptions found in

     E. Cuthill, J. McKee.  ‘Reducing the Bandwidth of Sparse Symmetric
     Matrices’.  Proceedings of the 24th ACM National Conference,
     157–172 1969, Brandon Press, New Jersey.

     A. George, J.W.H. Liu.  ‘Computer Solution of Large Sparse Positive
     Definite Systems’, Prentice Hall Series in Computational
     Mathematics, ISBN 0-13-165274-5, 1981.

     See also: *note colperm: XREFcolperm, *note colamd: XREFcolamd,
     *note symamd: XREFsymamd.


File: octave.info,  Node: Sparse Linear Algebra,  Next: Iterative Techniques,  Prev: Basics,  Up: Sparse Matrices

22.2 Linear Algebra on Sparse Matrices
======================================

Octave includes a polymorphic solver for sparse matrices, where the
exact solver used to factorize the matrix, depends on the properties of
the sparse matrix itself.  Generally, the cost of determining the matrix
type is small relative to the cost of factorizing the matrix itself, but
in any case the matrix type is cached once it is calculated, so that it
is not re-determined each time it is used in a linear equation.

   The selection tree for how the linear equation is solve is

  1. If the matrix is diagonal, solve directly and goto 8

  2. If the matrix is a permuted diagonal, solve directly taking into
     account the permutations.  Goto 8

  3. If the matrix is square, banded and if the band density is less
     than that given by ‘spparms ("bandden")’ continue, else goto 4.

       a. If the matrix is tridiagonal and the right-hand side is not
          sparse continue, else goto 3b.

            1. If the matrix is Hermitian, with a positive real
               diagonal, attempt Cholesky factorization using LAPACK
               xPTSV.

            2. If the above failed or the matrix is not Hermitian with a
               positive real diagonal use Gaussian elimination with
               pivoting using LAPACK xGTSV, and goto 8.

       b. If the matrix is Hermitian with a positive real diagonal,
          attempt Cholesky factorization using LAPACK xPBTRF.

       c. if the above failed or the matrix is not Hermitian with a
          positive real diagonal use Gaussian elimination with pivoting
          using LAPACK xGBTRF, and goto 8.

  4. If the matrix is upper or lower triangular perform a sparse forward
     or backward substitution, and goto 8

  5. If the matrix is an upper triangular matrix with column
     permutations or lower triangular matrix with row permutations,
     perform a sparse forward or backward substitution, and goto 8

  6. If the matrix is square, Hermitian with a real positive diagonal,
     attempt sparse Cholesky factorization using CHOLMOD.

  7. If the sparse Cholesky factorization failed or the matrix is not
     Hermitian with a real positive diagonal, and the matrix is square,
     factorize using UMFPACK.

  8. If the matrix is not square, or any of the previous solvers flags a
     singular or near singular matrix, find a minimum norm solution
     using CXSPARSE(1).

   The band density is defined as the number of nonzero values in the
band divided by the total number of values in the full band.  The banded
matrix solvers can be entirely disabled by using “spparms” to set
‘bandden’ to 1 (i.e., ‘spparms ("bandden", 1)’).

   The QR solver factorizes the problem with a Dulmage-Mendelsohn
decomposition, to separate the problem into blocks that can be treated
as over-determined, multiple well determined blocks, and a final
over-determined block.  For matrices with blocks of strongly connected
nodes this is a big win as LU decomposition can be used for many blocks.
It also significantly improves the chance of finding a solution to
over-determined problems rather than just returning a vector of “NaN”’s.

   All of the solvers above, can calculate an estimate of the condition
number.  This can be used to detect numerical stability problems in the
solution and force a minimum norm solution to be used.  However, for
narrow banded, triangular or diagonal matrices, the cost of calculating
the condition number is significant, and can in fact exceed the cost of
factoring the matrix.  Therefore the condition number is not calculated
in these cases, and Octave relies on simpler techniques to detect
singular matrices or the underlying LAPACK code in the case of banded
matrices.

   The user can force the type of the matrix with the ‘matrix_type’
function.  This overcomes the cost of discovering the type of the
matrix.  However, it should be noted that identifying the type of the
matrix incorrectly will lead to unpredictable results, and so
‘matrix_type’ should be used with care.

 -- : NEST = normest (A)
 -- : NEST = normest (A, TOL)
 -- : [NEST, ITER] = normest (...)
     Estimate the 2-norm of the matrix A using a power series analysis.

     This is typically used for large matrices, where the cost of
     calculating ‘norm (A)’ is prohibitive and an approximation to the
     2-norm is acceptable.

     TOL is the tolerance to which the 2-norm is calculated.  By default
     TOL is 1e-6.

     The optional output ITER returns the number of iterations that were
     required for ‘normest’ to converge.

     See also: *note normest1: XREFnormest1, *note norm: XREFnorm, *note
     cond: XREFcond, *note condest: XREFcondest.

 -- : NEST = normest1 (A)
 -- : NEST = normest1 (A, T)
 -- : NEST = normest1 (A, T, X0)
 -- : NEST = normest1 (AFUN, T, X0, P1, P2, ...)
 -- : [NEST, V] = normest1 (A, ...)
 -- : [NEST, V, W] = normest1 (A, ...)
 -- : [NEST, V, W, ITER] = normest1 (A, ...)
     Estimate the 1-norm of the matrix A using a block algorithm.

     ‘normest1’ is best for large sparse matrices where only an estimate
     of the norm is required.  For small to medium sized matrices,
     consider using ‘norm (A, 1)’.  In addition, ‘normest1’ can be used
     for the estimate of the 1-norm of a linear operator A when
     matrix-vector products ‘A * X’ and ‘A' * X’ can be cheaply
     computed.  In this case, instead of the matrix A, a function ‘AFUN
     (FLAG, X)’ is used; it must return:

        • the dimension N of A, if FLAG is "dim"

        • true if A is a real operator, if FLAG is "real"

        • the result ‘A * X’, if FLAG is "notransp"

        • the result ‘A' * X’, if FLAG is "transp"

     A typical case is A defined by ‘B ^ M’, in which the result ‘A * X’
     can be computed without even forming explicitly ‘B ^ M’ by:

          Y = X;
          for I = 1:M
            Y = B * Y;
          endfor

     The parameters P1, P2, ... are arguments of ‘AFUN (FLAG, X, P1, P2,
     ...)’.

     The default value for T is 2.  The algorithm requires matrix-matrix
     products with sizes N x N and N x T.

     The initial matrix X0 should have columns of unit 1-norm.  The
     default initial matrix X0 has the first column ‘ones (N, 1) / N’
     and, if T > 1, the remaining columns with random elements ‘-1 / N’,
     ‘1 / N’, divided by N.

     On output, NEST is the desired estimate, V and W are vectors such
     that ‘W = A * V’, with ‘norm (W, 1)’ = ‘C * norm (V, 1)’.  ITER
     contains in ‘ITER(1)’ the number of iterations (the maximum is
     hardcoded to 5) and in ‘ITER(2)’ the total number of products ‘A *
     X’ or ‘A' * X’ performed by the algorithm.

     Algorithm Note: ‘normest1’ uses random numbers during evaluation.
     Therefore, if consistent results are required, the "state" of the
     random generator should be fixed before invoking ‘normest1’.

     Reference: N. J. Higham and F. Tisseur, ‘A block algorithm for
     matrix 1-norm estimation, with and application to 1-norm
     pseudospectra’, SIAM J. Matrix Anal.  Appl., pp.  1185–1201, Vol
     21, No.  4, 2000.

     See also: *note normest: XREFnormest, *note norm: XREFnorm, *note
     cond: XREFcond, *note condest: XREFcondest.

 -- : CEST = condest (A)
 -- : CEST = condest (A, T)
 -- : CEST = condest (A, SOLVEFUN, T, P1, P2, ...)
 -- : CEST = condest (AFCN, SOLVEFUN, T, P1, P2, ...)
 -- : [CEST, V] = condest (...)

     Estimate the 1-norm condition number of a square matrix A using T
     test vectors and a randomized 1-norm estimator.

     The optional input T specifies the number of test vectors (default
     5).

     If the matrix is not explicit, e.g., when estimating the condition
     number of A given an LU factorization, ‘condest’ uses the following
     functions:

        − AFCN which must return

             • the dimension N of A, if FLAG is "dim"

             • true if A is a real operator, if FLAG is "real"

             • the result ‘A * X’, if FLAG is "notransp"

             • the result ‘A' * X’, if FLAG is "transp"

        − SOLVEFUN which must return

             • the dimension N of A, if FLAG is "dim"

             • true if A is a real operator, if FLAG is "real"

             • the result ‘A \ X’, if FLAG is "notransp"

             • the result ‘A' \ X’, if FLAG is "transp"

     The parameters P1, P2, ... are arguments of ‘AFCN (FLAG, X, P1, P2,
     ...)’ and ‘SOLVEFCN (FLAG, X, P1, P2, ...)’.

     The principal output is the 1-norm condition number estimate CEST.

     The optional second output is an approximate null vector when CEST
     is large; it satisfies the equation ‘norm (A*v, 1) == norm (A, 1) *
     norm (V, 1) / EST’.

     Algorithm Note: ‘condest’ uses a randomized algorithm to
     approximate the 1-norms.  Therefore, if consistent results are
     required, the "state" of the random generator should be fixed
     before invoking ‘condest’.

     References:

        • N.J. Higham and F. Tisseur, ‘A Block Algorithm for Matrix
          1-Norm Estimation, with an Application to 1-Norm
          Pseudospectra’.  SIMAX vol 21, no 4, pp 1185-1201.
          <http://dx.doi.org/10.1137/S0895479899356080>

        • N.J. Higham and F. Tisseur, ‘A Block Algorithm for Matrix
          1-Norm Estimation, with an Application to 1-Norm
          Pseudospectra’.  <http://citeseer.ist.psu.edu/223007.html>

     See also: *note cond: XREFcond, *note norm: XREFnorm, *note
     normest1: XREFnormest1, *note normest: XREFnormest.

 -- : spparms ()
 -- : VALS = spparms ()
 -- : [KEYS, VALS] = spparms ()
 -- : VAL = spparms (KEY)
 -- : spparms (VALS)
 -- : spparms ("default")
 -- : spparms ("tight")
 -- : spparms (KEY, VAL)
     Query or set the parameters used by the sparse solvers and
     factorization functions.

     The first four calls above get information about the current
     settings, while the others change the current settings.  The
     parameters are stored as pairs of keys and values, where the values
     are all floats and the keys are one of the following strings:

     ‘spumoni’
          Printing level of debugging information of the solvers
          (default 0)

     ‘ths_rel’
          Included for compatibility.  Not used.  (default 1)

     ‘ths_abs’
          Included for compatibility.  Not used.  (default 1)

     ‘exact_d’
          Included for compatibility.  Not used.  (default 0)

     ‘supernd’
          Included for compatibility.  Not used.  (default 3)

     ‘rreduce’
          Included for compatibility.  Not used.  (default 3)

     ‘wh_frac’
          Included for compatibility.  Not used.  (default 0.5)

     ‘autommd’
          Flag whether the LU/QR and the ’\’ and ’/’ operators will
          automatically use the sparsity preserving mmd functions
          (default 1)

     ‘autoamd’
          Flag whether the LU and the ’\’ and ’/’ operators will
          automatically use the sparsity preserving amd functions
          (default 1)

     ‘piv_tol’
          The pivot tolerance of the UMFPACK solvers (default 0.1)

     ‘sym_tol’
          The pivot tolerance of the UMFPACK symmetric solvers (default
          0.001)

     ‘bandden’
          The density of nonzero elements in a banded matrix before it
          is treated by the LAPACK banded solvers (default 0.5)

     ‘umfpack’
          Flag whether the UMFPACK or mmd solvers are used for the LU,
          ’\’ and ’/’ operations (default 1)

     The value of individual keys can be set with ‘spparms (KEY, VAL)’.
     The default values can be restored with the special keyword
     "default".  The special keyword "tight" can be used to set the mmd
     solvers to attempt a sparser solution at the potential cost of
     longer running time.

     See also: *note chol: XREFchol, *note colamd: XREFcolamd, *note lu:
     XREFlu, *note qr: XREFqr, *note symamd: XREFsymamd.

 -- : P = sprank (S)

     Calculate the structural rank of the sparse matrix S.

     Note that only the structure of the matrix is used in this
     calculation based on a Dulmage-Mendelsohn permutation to block
     triangular form.  As such the numerical rank of the matrix S is
     bounded by ‘sprank (S) >= rank (S)’.  Ignoring floating point
     errors ‘sprank (S) == rank (S)’.

     See also: *note dmperm: XREFdmperm.

 -- : [COUNT, H, PARENT, POST, R] = symbfact (S)
 -- : [...] = symbfact (S, TYP)
 -- : [...] = symbfact (S, TYP, MODE)

     Perform a symbolic factorization analysis of the sparse matrix S.

     The input variables are

     S
          S is a real or complex sparse matrix.

     TYP
          Is the type of the factorization and can be one of

          "sym" (default)
               Factorize S.  Assumes S is symmetric and uses the upper
               triangular portion of the matrix.

          "col"
               Factorize S’ * S.

          "row"
               Factorize S * S’.

          "lo"
               Factorize S’.  Assumes S is symmetric and uses the lower
               triangular portion of the matrix.

     MODE
          When MODE is unspecified return the Cholesky factorization for
          R.  If MODE is "lower" or "L" then return the conjugate
          transpose R’ which is a lower triangular factor.  The
          conjugate transpose version is faster and uses less memory,
          but still returns the same values for all other outputs:
          COUNT, H, PARENT, and POST.

     The output variables are:

     COUNT
          The row counts of the Cholesky factorization as determined by
          TYP.  The computational difficulty of performing the true
          factorization using ‘chol’ is ‘sum (COUNT .^ 2)’.

     H
          The height of the elimination tree.

     PARENT
          The elimination tree itself.

     POST
          A sparse boolean matrix whose structure is that of the
          Cholesky factorization as determined by TYP.

     See also: *note chol: XREFchol, *note etree: XREFetree, *note
     treelayout: XREFtreelayout.

   For non square matrices, the user can also utilize the ‘spaugment’
function to find a least squares solution to a linear equation.

 -- : S = spaugment (A, C)
     Create the augmented matrix of A.

     This is given by

          [C * eye(M, M), A;
                      A', zeros(N, N)]

     This is related to the least squares solution of ‘A \ B’, by

          S * [ R / C; x] = [ B, zeros(N, columns(B)) ]

     where R is the residual error

          R = B - A * X

     As the matrix S is symmetric indefinite it can be factorized with
     ‘lu’, and the minimum norm solution can therefore be found without
     the need for a ‘qr’ factorization.  As the residual error will be
     ‘zeros (M, M)’ for underdetermined problems, and example can be

          m = 11; n = 10; mn = max (m, n);
          A = spdiags ([ones(mn,1), 10*ones(mn,1), -ones(mn,1)],
                       [-1, 0, 1], m, n);
          x0 = A \ ones (m,1);
          s = spaugment (A);
          [L, U, P, Q] = lu (s);
          x1 = Q * (U \ (L \ (P  * [ones(m,1); zeros(n,1)])));
          x1 = x1(end - n + 1 : end);

     To find the solution of an overdetermined problem needs an estimate
     of the residual error R and so it is more complex to formulate a
     minimum norm solution using the ‘spaugment’ function.

     In general the left division operator is more stable and faster
     than using the ‘spaugment’ function.

     See also: *note mldivide: XREFmldivide.

   Finally, the function ‘eigs’ can be used to calculate a limited
number of eigenvalues and eigenvectors based on a selection criteria and
likewise for ‘svds’ which calculates a limited number of singular values
and vectors.

 -- : D = eigs (A)
 -- : D = eigs (A, K)
 -- : D = eigs (A, K, SIGMA)
 -- : D = eigs (A, K, SIGMA, OPTS)
 -- : D = eigs (A, B)
 -- : D = eigs (A, B, K)
 -- : D = eigs (A, B, K, SIGMA)
 -- : D = eigs (A, B, K, SIGMA, OPTS)
 -- : D = eigs (AF, N)
 -- : D = eigs (AF, N, B)
 -- : D = eigs (AF, N, K)
 -- : D = eigs (AF, N, B, K)
 -- : D = eigs (AF, N, K, SIGMA)
 -- : D = eigs (AF, N, B, K, SIGMA)
 -- : D = eigs (AF, N, K, SIGMA, OPTS)
 -- : D = eigs (AF, N, B, K, SIGMA, OPTS)
 -- : [V, D] = eigs (A, ...)
 -- : [V, D] = eigs (AF, N, ...)
 -- : [V, D, FLAG] = eigs (A, ...)
 -- : [V, D, FLAG] = eigs (AF, N, ...)
     Calculate a limited number of eigenvalues and eigenvectors of A,
     based on a selection criteria.

     The number of eigenvalues and eigenvectors to calculate is given by
     K and defaults to 6.

     By default, ‘eigs’ solve the equation ‘A * v = lambda * v’, where
     ‘lambda’ is a scalar representing one of the eigenvalues, and ‘v’
     is the corresponding eigenvector.  If given the positive definite
     matrix B then ‘eigs’ solves the general eigenvalue equation ‘A * v
     = lambda * B * v’.

     The argument SIGMA determines which eigenvalues are returned.
     SIGMA can be either a scalar or a string.  When SIGMA is a scalar,
     the K eigenvalues closest to SIGMA are returned.  If SIGMA is a
     string, it must have one of the following values.

     "lm"
          Largest Magnitude (default).

     "sm"
          Smallest Magnitude.

     "la"
          Largest Algebraic (valid only for real symmetric problems).

     "sa"
          Smallest Algebraic (valid only for real symmetric problems).

     "be"
          Both Ends, with one more from the high-end if K is odd (valid
          only for real symmetric problems).

     "lr"
          Largest Real part (valid only for complex or unsymmetric
          problems).

     "sr"
          Smallest Real part (valid only for complex or unsymmetric
          problems).

     "li"
          Largest Imaginary part (valid only for complex or unsymmetric
          problems).

     "si"
          Smallest Imaginary part (valid only for complex or unsymmetric
          problems).

     If OPTS is given, it is a structure defining possible options that
     ‘eigs’ should use.  The fields of the OPTS structure are:

     ‘issym’
          If AF is given, then flags whether the function AF defines a
          symmetric problem.  It is ignored if A is given.  The default
          is false.

     ‘isreal’
          If AF is given, then flags whether the function AF defines a
          real problem.  It is ignored if A is given.  The default is
          true.

     ‘tol’
          Defines the required convergence tolerance, calculated as ‘tol
          * norm (A)’.  The default is ‘eps’.

     ‘maxit’
          The maximum number of iterations.  The default is 300.

     ‘p’
          The number of Lanzcos basis vectors to use.  More vectors will
          result in faster convergence, but a greater use of memory.
          The optimal value of ‘p’ is problem dependent and should be in
          the range K to N.  The default value is ‘2 * K’.

     ‘v0’
          The starting vector for the algorithm.  An initial vector
          close to the final vector will speed up convergence.  The
          default is for ARPACK to randomly generate a starting vector.
          If specified, ‘v0’ must be an N-by-1 vector where ‘N = rows
          (A)’

     ‘disp’
          The level of diagnostic printout (0|1|2).  If ‘disp’ is 0 then
          diagnostics are disabled.  The default value is 0.

     ‘cholB’
          Flag if ‘chol (B)’ is passed rather than B.  The default is
          false.

     ‘permB’
          The permutation vector of the Cholesky factorization of B if
          ‘cholB’ is true.  It is obtained by ‘[R, ~, permB] = chol (B,
          "vector")’.  The default is ‘1:N’.

     It is also possible to represent A by a function denoted AF.  AF
     must be followed by a scalar argument N defining the length of the
     vector argument accepted by AF.  AF can be a function handle, an
     inline function, or a string.  When AF is a string it holds the
     name of the function to use.

     AF is a function of the form ‘y = af (x)’ where the required return
     value of AF is determined by the value of SIGMA.  The four possible
     forms are

     ‘A * x’
          if SIGMA is not given or is a string other than "sm".

     ‘A \ x’
          if SIGMA is 0 or "sm".

     ‘(A - sigma * I) \ x’
          for the standard eigenvalue problem, where ‘I’ is the identity
          matrix of the same size as A.

     ‘(A - sigma * B) \ x’
          for the general eigenvalue problem.

     The return arguments of ‘eigs’ depend on the number of return
     arguments requested.  With a single return argument, a vector D of
     length K is returned containing the K eigenvalues that have been
     found.  With two return arguments, V is a N-by-K matrix whose
     columns are the K eigenvectors corresponding to the returned
     eigenvalues.  The eigenvalues themselves are returned in D in the
     form of a N-by-K matrix, where the elements on the diagonal are the
     eigenvalues.

     Given a third return argument FLAG, ‘eigs’ returns the status of
     the convergence.  If FLAG is 0 then all eigenvalues have converged.
     Any other value indicates a failure to converge.

     This function is based on the ARPACK package, written by R.
     Lehoucq, K. Maschhoff, D. Sorensen, and C. Yang.  For more
     information see <http://www.caam.rice.edu/software/ARPACK/>.

     See also: *note eig: XREFeig, *note svds: XREFsvds.

 -- : S = svds (A)
 -- : S = svds (A, K)
 -- : S = svds (A, K, SIGMA)
 -- : S = svds (A, K, SIGMA, OPTS)
 -- : [U, S, V] = svds (...)
 -- : [U, S, V, FLAG] = svds (...)

     Find a few singular values of the matrix A.

     The singular values are calculated using

          [M, N] = size (A);
          S = eigs ([sparse(M, M), A;
                               A', sparse(N, N)])

     The eigenvalues returned by ‘eigs’ correspond to the singular
     values of A.  The number of singular values to calculate is given
     by K and defaults to 6.

     The argument SIGMA specifies which singular values to find.  When
     SIGMA is the string ’L’, the default, the largest singular values
     of A are found.  Otherwise, SIGMA must be a real scalar and the
     singular values closest to SIGMA are found.  As a corollary, ‘SIGMA
     = 0’ finds the smallest singular values.  Note that for relatively
     small values of SIGMA, there is a chance that the requested number
     of singular values will not be found.  In that case SIGMA should be
     increased.

     OPTS is a structure defining options that ‘svds’ will pass to
     ‘eigs’.  The possible fields of this structure are documented in
     ‘eigs’.  By default, ‘svds’ sets the following three fields:

     ‘tol’
          The required convergence tolerance for the singular values.
          The default value is 1e-10.  ‘eigs’ is passed ‘TOL / sqrt(2)’.

     ‘maxit’
          The maximum number of iterations.  The default is 300.

     ‘disp’
          The level of diagnostic printout (0|1|2).  If ‘disp’ is 0 then
          diagnostics are disabled.  The default value is 0.

     If more than one output is requested then ‘svds’ will return an
     approximation of the singular value decomposition of A

          A_approx = U*S*V'

     where A_approx is a matrix of size A but only rank K.

     FLAG returns 0 if the algorithm has succesfully converged, and 1
     otherwise.  The test for convergence is

          norm (A*V - U*S, 1) <= TOL * norm (A, 1)

     ‘svds’ is best for finding only a few singular values from a large
     sparse matrix.  Otherwise, ‘svd (full (A))’ will likely be more
     efficient.

See also: *note svd: XREFsvd, *note eigs: XREFeigs.

   ---------- Footnotes ----------

   (1) The CHOLMOD, UMFPACK and CXSPARSE packages were written by Tim
Davis and are available at <http://www.cise.ufl.edu/research/sparse/>


File: octave.info,  Node: Iterative Techniques,  Next: Real Life Example,  Prev: Sparse Linear Algebra,  Up: Sparse Matrices

22.3 Iterative Techniques Applied to Sparse Matrices
====================================================

The left division ‘\’ and right division ‘/’ operators, discussed in the
previous section, use direct solvers to resolve a linear equation of the
form ‘X = A \ B’ or ‘X = B / A’.  Octave also includes a number of
functions to solve sparse linear equations using iterative techniques.

 -- : X = pcg (A, B, TOL, MAXIT, M1, M2, X0, ...)
 -- : [X, FLAG, RELRES, ITER, RESVEC, EIGEST] = pcg (...)

     Solve the linear system of equations ‘A * X = B’ by means of the
     Preconditioned Conjugate Gradient iterative method.

     The input arguments are

        • A can be either a square (preferably sparse) matrix or a
          function handle, inline function or string containing the name
          of a function which computes ‘A * X’.  In principle, A should
          be symmetric and positive definite; if ‘pcg’ finds A not to be
          positive definite, a warning is printed and the FLAG output
          will be set.

        • B is the right-hand side vector.

        • TOL is the required relative tolerance for the residual error,
          ‘B - A * X’.  The iteration stops if
          ‘norm (B - A * X)’ ≤ TOL * norm (B).  If TOL is omitted or
          empty then a tolerance of 1e-6 is used.

        • MAXIT is the maximum allowable number of iterations; if MAXIT
          is omitted or empty then a value of 20 is used.

        • M = M1 * M2 is the (left) preconditioning matrix, so that the
          iteration is (theoretically) equivalent to solving by ‘pcg’
          ‘P * X = M \ B’, with ‘P = M \ A’.  Note that a proper choice
          of the preconditioner may dramatically improve the overall
          performance of the method.  Instead of matrices M1 and M2, the
          user may pass two functions which return the results of
          applying the inverse of M1 and M2 to a vector (usually this is
          the preferred way of using the preconditioner).  If M1 is
          omitted or empty ‘[]’ then no preconditioning is applied.  If
          M2 is omitted, M = M1 will be used as a preconditioner.

        • X0 is the initial guess.  If X0 is omitted or empty then the
          function sets X0 to a zero vector by default.

     The arguments which follow X0 are treated as parameters, and passed
     in a proper way to any of the functions (A or M) which are passed
     to ‘pcg’.  See the examples below for further details.  The output
     arguments are

        • X is the computed approximation to the solution of
          ‘A * X = B’.

        • FLAG reports on the convergence.  A value of 0 means the
          solution converged and the tolerance criterion given by TOL is
          satisfied.  A value of 1 means that the MAXIT limit for the
          iteration count was reached.  A value of 3 indicates that the
          (preconditioned) matrix was found not to be positive definite.

        • RELRES is the ratio of the final residual to its initial
          value, measured in the Euclidean norm.

        • ITER is the actual number of iterations performed.

        • RESVEC describes the convergence history of the method.
          ‘RESVEC(i,1)’ is the Euclidean norm of the residual, and
          ‘RESVEC(i,2)’ is the preconditioned residual norm, after the
          (I-1)-th iteration, ‘I = 1, 2, ..., ITER+1’.  The
          preconditioned residual norm is defined as ‘norm (R) ^ 2 = R'
          * (M \ R)’ where ‘R = B - A * X’, see also the description of
          M.  If EIGEST is not required, only ‘RESVEC(:,1)’ is returned.

        • EIGEST returns the estimate for the smallest ‘EIGEST(1)’ and
          largest ‘EIGEST(2)’ eigenvalues of the preconditioned matrix
          ‘P = M \ A’.  In particular, if no preconditioning is used,
          the estimates for the extreme eigenvalues of A are returned.
          ‘EIGEST(1)’ is an overestimate and ‘EIGEST(2)’ is an
          underestimate, so that ‘EIGEST(2) / EIGEST(1)’ is a lower
          bound for ‘cond (P, 2)’, which nevertheless in the limit
          should theoretically be equal to the actual value of the
          condition number.  The method which computes EIGEST works only
          for symmetric positive definite A and M, and the user is
          responsible for verifying this assumption.

     Let us consider a trivial problem with a diagonal matrix (we
     exploit the sparsity of A)

          n = 10;
          A = diag (sparse (1:n));
          b = rand (n, 1);
          [l, u, p] = ilu (A, struct ("droptol", 1.e-3));

     EXAMPLE 1: Simplest use of ‘pcg’

          x = pcg (A, b)

     EXAMPLE 2: ‘pcg’ with a function which computes ‘A * X’

          function y = apply_a (x)
            y = [1:N]' .* x;
          endfunction

          x = pcg ("apply_a", b)

     EXAMPLE 3: ‘pcg’ with a preconditioner: L * U

          x = pcg (A, b, 1.e-6, 500, l*u)

     EXAMPLE 4: ‘pcg’ with a preconditioner: L * U.  Faster than EXAMPLE
     3 since lower and upper triangular matrices are easier to invert

          x = pcg (A, b, 1.e-6, 500, l, u)

     EXAMPLE 5: Preconditioned iteration, with full diagnostics.  The
     preconditioner (quite strange, because even the original matrix A
     is trivial) is defined as a function

          function y = apply_m (x)
            k = floor (length (x) - 2);
            y = x;
            y(1:k) = x(1:k) ./ [1:k]';
          endfunction

          [x, flag, relres, iter, resvec, eigest] = ...
                             pcg (A, b, [], [], "apply_m");
          semilogy (1:iter+1, resvec);

     EXAMPLE 6: Finally, a preconditioner which depends on a parameter
     K.

          function y = apply_M (x, varargin)
            K = varargin{1};
            y = x;
            y(1:K) = x(1:K) ./ [1:K]';
          endfunction

          [x, flag, relres, iter, resvec, eigest] = ...
               pcg (A, b, [], [], "apply_m", [], [], 3)

     References:

       1. C.T. Kelley, ‘Iterative Methods for Linear and Nonlinear
          Equations’, SIAM, 1995.  (the base PCG algorithm)

       2. Y. Saad, ‘Iterative Methods for Sparse Linear Systems’, PWS
          1996.  (condition number estimate from PCG) Revised version of
          this book is available online at
          <http://www-users.cs.umn.edu/~saad/books.html>

     See also: *note sparse: XREFsparse, *note pcr: XREFpcr.

 -- : X = pcr (A, B, TOL, MAXIT, M, X0, ...)
 -- : [X, FLAG, RELRES, ITER, RESVEC] = pcr (...)

     Solve the linear system of equations ‘A * X = B’ by means of the
     Preconditioned Conjugate Residuals iterative method.

     The input arguments are

        • A can be either a square (preferably sparse) matrix or a
          function handle, inline function or string containing the name
          of a function which computes ‘A * X’.  In principle A should
          be symmetric and non-singular; if ‘pcr’ finds A to be
          numerically singular, you will get a warning message and the
          FLAG output parameter will be set.

        • B is the right hand side vector.

        • TOL is the required relative tolerance for the residual error,
          ‘B - A * X’.  The iteration stops if ‘norm (B - A * X) <= TOL
          * norm (B - A * X0)’.  If TOL is empty or is omitted, the
          function sets ‘TOL = 1e-6’ by default.

        • MAXIT is the maximum allowable number of iterations; if ‘[]’
          is supplied for ‘maxit’, or ‘pcr’ has less arguments, a
          default value equal to 20 is used.

        • M is the (left) preconditioning matrix, so that the iteration
          is (theoretically) equivalent to solving by ‘pcr’ ‘P * X = M \
          B’, with ‘P = M \ A’.  Note that a proper choice of the
          preconditioner may dramatically improve the overall
          performance of the method.  Instead of matrix M, the user may
          pass a function which returns the results of applying the
          inverse of M to a vector (usually this is the preferred way of
          using the preconditioner).  If ‘[]’ is supplied for M, or M is
          omitted, no preconditioning is applied.

        • X0 is the initial guess.  If X0 is empty or omitted, the
          function sets X0 to a zero vector by default.

     The arguments which follow X0 are treated as parameters, and passed
     in a proper way to any of the functions (A or M) which are passed
     to ‘pcr’.  See the examples below for further details.

     The output arguments are

        • X is the computed approximation to the solution of ‘A * X =
          B’.

        • FLAG reports on the convergence.  ‘FLAG = 0’ means the
          solution converged and the tolerance criterion given by TOL is
          satisfied.  ‘FLAG = 1’ means that the MAXIT limit for the
          iteration count was reached.  ‘FLAG = 3’ reports a ‘pcr’
          breakdown, see [1] for details.

        • RELRES is the ratio of the final residual to its initial
          value, measured in the Euclidean norm.

        • ITER is the actual number of iterations performed.

        • RESVEC describes the convergence history of the method, so
          that ‘RESVEC (i)’ contains the Euclidean norms of the residual
          after the (I-1)-th iteration, ‘I = 1,2, ..., ITER+1’.

     Let us consider a trivial problem with a diagonal matrix (we
     exploit the sparsity of A)

          n = 10;
          A = sparse (diag (1:n));
          b = rand (N, 1);

     EXAMPLE 1: Simplest use of ‘pcr’

          x = pcr (A, b)

     EXAMPLE 2: ‘pcr’ with a function which computes ‘A * X’.

          function y = apply_a (x)
            y = [1:10]' .* x;
          endfunction

          x = pcr ("apply_a", b)

     EXAMPLE 3: Preconditioned iteration, with full diagnostics.  The
     preconditioner (quite strange, because even the original matrix A
     is trivial) is defined as a function

          function y = apply_m (x)
            k = floor (length (x) - 2);
            y = x;
            y(1:k) = x(1:k) ./ [1:k]';
          endfunction

          [x, flag, relres, iter, resvec] = ...
                             pcr (A, b, [], [], "apply_m")
          semilogy ([1:iter+1], resvec);

     EXAMPLE 4: Finally, a preconditioner which depends on a parameter
     K.

          function y = apply_m (x, varargin)
            k = varargin{1};
            y = x;
            y(1:k) = x(1:k) ./ [1:k]';
          endfunction

          [x, flag, relres, iter, resvec] = ...
                             pcr (A, b, [], [], "apply_m"', [], 3)

     References:

     [1] W. Hackbusch, ‘Iterative Solution of Large Sparse Systems of
     Equations’, section 9.5.4; Springer, 1994

     See also: *note sparse: XREFsparse, *note pcg: XREFpcg.

   The speed with which an iterative solver converges to a solution can
be accelerated with the use of a pre-conditioning matrix M.  In this
case the linear equation ‘M^-1 * X = M^-1 * A \ B’ is solved instead.
Typical pre-conditioning matrices are partial factorizations of the
original matrix.

 -- : L = ichol (A)
 -- : L = ichol (A, OPTS)

     Compute the incomplete Cholesky factorization of the sparse square
     matrix A.

     By default, ‘ichol’ uses only the lower triangle of A and produces
     a lower triangular factor L such that L*L’ approximates A.

     The factor given by this routine may be useful as a preconditioner
     for a system of linear equations being solved by iterative methods
     such as PCG (Preconditioned Conjugate Gradient).

     The factorization may be modified by passing options in a structure
     OPTS.  The option name is a field of the structure and the setting
     is the value of field.  Names and specifiers are case sensitive.

     type
          Type of factorization.

          "nofill" (default)
               Incomplete Cholesky factorization with no fill-in
               (IC(0)).

          "ict"
               Incomplete Cholesky factorization with threshold dropping
               (ICT).

     diagcomp
          A non-negative scalar ALPHA for incomplete Cholesky
          factorization of ‘A + ALPHA * diag (diag (A))’ instead of A.
          This can be useful when A is not positive definite.  The
          default value is 0.

     droptol
          A non-negative scalar specifying the drop tolerance for
          factorization if performing ICT.  The default value is 0 which
          produces the complete Cholesky factorization.

          Non-diagonal entries of L are set to 0 unless

          ‘abs (L(i,j)) >= droptol * norm (A(j:end, j), 1)’.

     michol
          Modified incomplete Cholesky factorization:

          "off" (default)
               Row and column sums are not necessarily preserved.

          "on"
               The diagonal of L is modified so that row (and column)
               sums are preserved even when elements have been dropped
               during the factorization.  The relationship preserved is:
               ‘A * e = L * L' * e’, where e is a vector of ones.

     shape

          "lower" (default)
               Use only the lower triangle of A and return a lower
               triangular factor L such that L*L’ approximates A.

          "upper"
               Use only the upper triangle of A and return an upper
               triangular factor U such that ‘U'*U’ approximates A.

     EXAMPLES

     The following problem demonstrates how to factorize a sample
     symmetric positive definite matrix with the full Cholesky
     decomposition and with the incomplete one.

          A = [ 0.37, -0.05,  -0.05,  -0.07;
               -0.05,  0.116,  0.0,   -0.05;
               -0.05,  0.0,    0.116, -0.05;
               -0.07, -0.05,  -0.05,   0.202];
          A = sparse (A);
          nnz (tril (A))
          ans =  9
          L = chol (A, "lower");
          nnz (L)
          ans =  10
          norm (A - L * L', "fro") / norm (A, "fro")
          ans =  1.1993e-16
          opts.type = "nofill";
          L = ichol (A, opts);
          nnz (L)
          ans =  9
          norm (A - L * L', "fro") / norm (A, "fro")
          ans =  0.019736

     Another example for decomposition is a finite difference matrix
     used to solve a boundary value problem on the unit square.

          nx = 400; ny = 200;
          hx = 1 / (nx + 1); hy = 1 / (ny + 1);
          Dxx = spdiags ([ones(nx, 1), -2*ones(nx, 1), ones(nx, 1)],
                         [-1 0 1 ], nx, nx) / (hx ^ 2);
          Dyy = spdiags ([ones(ny, 1), -2*ones(ny, 1), ones(ny, 1)],
                         [-1 0 1 ], ny, ny) / (hy ^ 2);
          A = -kron (Dxx, speye (ny)) - kron (speye (nx), Dyy);
          nnz (tril (A))
          ans =  239400
          opts.type = "nofill";
          L = ichol (A, opts);
          nnz (tril (A))
          ans =  239400
          norm (A - L * L', "fro") / norm (A, "fro")
          ans =  0.062327

     References for implemented algorithms:

     [1] Y. Saad.  "Preconditioning Techniques."  ‘Iterative Methods for
     Sparse Linear Systems’, PWS Publishing Company, 1996.

     [2] M. Jones, P. Plassmann: ‘An Improved Incomplete Cholesky
     Factorization’, 1992.

     See also: *note chol: XREFchol, *note ilu: XREFilu, *note pcg:
     XREFpcg.

 -- : ilu (A)
 -- : ilu (A, OPTS)
 -- : [L, U] = ilu (...)
 -- : [L, U, P] = ilu (...)

     Compute the incomplete LU factorization of the sparse square matrix
     A.

     ‘ilu’ returns a unit lower triangular matrix L, an upper triangular
     matrix U, and optionally a permutation matrix P, such that ‘L*U’
     approximates ‘P*A’.

     The factors given by this routine may be useful as preconditioners
     for a system of linear equations being solved by iterative methods
     such as BICG (BiConjugate Gradients) or GMRES (Generalized Minimum
     Residual Method).

     The factorization may be modified by passing options in a structure
     OPTS.  The option name is a field of the structure and the setting
     is the value of field.  Names and specifiers are case sensitive.

     ‘type’
          Type of factorization.

          "nofill"
               ILU factorization with no fill-in (ILU(0)).

               Additional supported options: ‘milu’.

          "crout"
               Crout version of ILU factorization (ILUC).

               Additional supported options: ‘milu’, ‘droptol’.

          "ilutp" (default)
               ILU factorization with threshold and pivoting.

               Additional supported options: ‘milu’, ‘droptol’, ‘udiag’,
               ‘thresh’.

     ‘droptol’
          A non-negative scalar specifying the drop tolerance for
          factorization.  The default value is 0 which produces the
          complete LU factorization.

          Non-diagonal entries of U are set to 0 unless

          ‘abs (U(i,j)) >= droptol * norm (A(:,j))’.

          Non-diagonal entries of L are set to 0 unless

          ‘abs (L(i,j)) >= droptol * norm (A(:,j))/U(j,j)’.

     ‘milu’
          Modified incomplete LU factorization:

          "row"
               Row-sum modified incomplete LU factorization.  The
               factorization preserves row sums: ‘A * e = L * U * e’,
               where e is a vector of ones.

          "col"
               Column-sum modified incomplete LU factorization.  The
               factorization preserves column sums: ‘e' * A = e' * L *
               U’.

          "off" (default)
               Row and column sums are not necessarily preserved.

     ‘udiag’
          If true, any zeros on the diagonal of the upper triangular
          factor are replaced by the local drop tolerance ‘droptol *
          norm (A(:,j))/U(j,j)’.  The default is false.

     ‘thresh’
          Pivot threshold for factorization.  It can range between 0
          (diagonal pivoting) and 1 (default), where the maximum
          magnitude entry in the column is chosen to be the pivot.

     If ‘ilu’ is called with just one output, the returned matrix is ‘L
     + U - speye (size (A))’, where L is unit lower triangular and U is
     upper triangular.

     With two outputs, ‘ilu’ returns a unit lower triangular matrix L
     and an upper triangular matrix U.  For OPTS.type == "ilutp", one of
     the factors is permuted based on the value of OPTS.milu.  When
     OPTS.milu == "row", U is a column permuted upper triangular factor.
     Otherwise, L is a row-permuted unit lower triangular factor.

     If there are three named outputs and OPTS.milu != "row", P is
     returned such that L and U are incomplete factors of ‘P*A’.  When
     OPTS.milu == "row", P is returned such that L and U are incomplete
     factors of ‘A*P’.

     EXAMPLES

          A = gallery ("neumann", 1600) + speye (1600);
          opts.type = "nofill";
          nnz (A)
          ans = 7840

          nnz (lu (A))
          ans = 126478

          nnz (ilu (A, opts))
          ans = 7840

     This shows that A has 7,840 nonzeros, the complete LU factorization
     has 126,478 nonzeros, and the incomplete LU factorization, with 0
     level of fill-in, has 7,840 nonzeros, the same amount as A.  Taken
     from: http://www.mathworks.com/help/matlab/ref/ilu.html

          A = gallery ("wathen", 10, 10);
          b = sum (A, 2);
          tol = 1e-8;
          maxit = 50;
          opts.type = "crout";
          opts.droptol = 1e-4;
          [L, U] = ilu (A, opts);
          x = bicg (A, b, tol, maxit, L, U);
          norm (A * x - b, inf)

     This example uses ILU as preconditioner for a random FEM-Matrix,
     which has a large condition number.  Without L and U BICG would not
     converge.

     See also: *note lu: XREFlu, *note ichol: XREFichol, *note bicg:
     XREFbicg, *note gmres: XREFgmres.


File: octave.info,  Node: Real Life Example,  Prev: Iterative Techniques,  Up: Sparse Matrices

22.4 Real Life Example using Sparse Matrices
============================================

A common application for sparse matrices is in the solution of Finite
Element Models.  Finite element models allow numerical solution of
partial differential equations that do not have closed form solutions,
typically because of the complex shape of the domain.

   In order to motivate this application, we consider the boundary value
Laplace equation.  This system can model scalar potential fields, such
as heat or electrical potential.  Given a medium Omega with boundary
dOmega.  At all points on the dOmega the boundary conditions are known,
and we wish to calculate the potential in Omega.  Boundary conditions
may specify the potential (Dirichlet boundary condition), its normal
derivative across the boundary (Neumann boundary condition), or a
weighted sum of the potential and its derivative (Cauchy boundary
condition).

   In a thermal model, we want to calculate the temperature in Omega and
know the boundary temperature (Dirichlet condition) or heat flux (from
which we can calculate the Neumann condition by dividing by the thermal
conductivity at the boundary).  Similarly, in an electrical model, we
want to calculate the voltage in Omega and know the boundary voltage
(Dirichlet) or current (Neumann condition after diving by the electrical
conductivity).  In an electrical model, it is common for much of the
boundary to be electrically isolated; this is a Neumann boundary
condition with the current equal to zero.

   The simplest finite element models will divide Omega into simplexes
(triangles in 2D, pyramids in 3D).

   The following example creates a simple rectangular 2-D electrically
conductive medium with 10 V and 20 V imposed on opposite sides
(Dirichlet boundary conditions).  All other edges are electrically
isolated.

        node_y = [1;1.2;1.5;1.8;2]*ones(1,11);
        node_x = ones(5,1)*[1,1.05,1.1,1.2, ...
                  1.3,1.5,1.7,1.8,1.9,1.95,2];
        nodes = [node_x(:), node_y(:)];

        [h,w] = size (node_x);
        elems = [];
        for idx = 1:w-1
          widx = (idx-1)*h;
          elems = [elems; ...
            widx+[(1:h-1);(2:h);h+(1:h-1)]'; ...
            widx+[(2:h);h+(2:h);h+(1:h-1)]' ];
        endfor

        E = size (elems,1); # No. of simplices
        N = size (nodes,1); # No. of vertices
        D = size (elems,2); # dimensions+1

   This creates a N-by-2 matrix ‘nodes’ and a E-by-3 matrix ‘elems’ with
values, which define finite element triangles:

       nodes(1:7,:)'
         1.00 1.00 1.00 1.00 1.00 1.05 1.05 ...
         1.00 1.20 1.50 1.80 2.00 1.00 1.20 ...

       elems(1:7,:)'
         1    2    3    4    2    3    4 ...
         2    3    4    5    7    8    9 ...
         6    7    8    9    6    7    8 ...

   Using a first order FEM, we approximate the electrical conductivity
distribution in Omega as constant on each simplex (represented by the
vector ‘conductivity’).  Based on the finite element geometry, we first
calculate a system (or stiffness) matrix for each simplex (represented
as 3-by-3 elements on the diagonal of the element-wise system matrix
‘SE’).  Based on ‘SE’ and a N-by-DE connectivity matrix ‘C’,
representing the connections between simplices and vertices, the global
connectivity matrix ‘S’ is calculated.

       ## Element conductivity
       conductivity = [1*ones(1,16), ...
              2*ones(1,48), 1*ones(1,16)];

       ## Connectivity matrix
       C = sparse ((1:D*E), reshape (elems', ...
              D*E, 1), 1, D*E, N);

       ## Calculate system matrix
       Siidx = floor ([0:D*E-1]'/D) * D * ...
              ones(1,D) + ones(D*E,1)*(1:D) ;
       Sjidx = [1:D*E]'*ones (1,D);
       Sdata = zeros (D*E,D);
       dfact = factorial (D-1);
       for j = 1:E
          a = inv ([ones(D,1), ...
              nodes(elems(j,:), :)]);
          const = conductivity(j) * 2 / ...
              dfact / abs (det (a));
          Sdata(D*(j-1)+(1:D),:) = const * ...
              a(2:D,:)' * a(2:D,:);
       endfor
       ## Element-wise system matrix
       SE = sparse(Siidx,Sjidx,Sdata);
       ## Global system matrix
       S = C'* SE *C;

   The system matrix acts like the conductivity ‘S’ in Ohm’s law ‘S * V
= I’.  Based on the Dirichlet and Neumann boundary conditions, we are
able to solve for the voltages at each vertex ‘V’.

       ## Dirichlet boundary conditions
       D_nodes = [1:5, 51:55];
       D_value = [10*ones(1,5), 20*ones(1,5)];

       V = zeros (N,1);
       V(D_nodes) = D_value;
       idx = 1:N; # vertices without Dirichlet
                  # boundary condns
       idx(D_nodes) = [];

       ## Neumann boundary conditions.  Note that
       ## N_value must be normalized by the
       ## boundary length and element conductivity
       N_nodes = [];
       N_value = [];

       Q = zeros (N,1);
       Q(N_nodes) = N_value;

       V(idx) = S(idx,idx) \ ( Q(idx) - ...
                 S(idx,D_nodes) * V(D_nodes));

   Finally, in order to display the solution, we show each solved
voltage value in the z-axis for each simplex vertex.

       elemx = elems(:,[1,2,3,1])';
       xelems = reshape (nodes(elemx, 1), 4, E);
       yelems = reshape (nodes(elemx, 2), 4, E);
       velems = reshape (V(elemx), 4, E);
       plot3 (xelems,yelems,velems,"k");
       print "grid.eps";


File: octave.info,  Node: Numerical Integration,  Next: Differential Equations,  Prev: Sparse Matrices,  Up: Top

23 Numerical Integration
************************

Octave comes with several built-in functions for computing the integral
of a function numerically (termed quadrature).  These functions all
solve 1-dimensional integration problems.

* Menu:

* Functions of One Variable::
* Orthogonal Collocation::
* Functions of Multiple Variables::


File: octave.info,  Node: Functions of One Variable,  Next: Orthogonal Collocation,  Up: Numerical Integration

23.1 Functions of One Variable
==============================

Octave supports five different algorithms for computing the integral of
a function f over the interval from a to b.  These are

‘quad’
     Numerical integration based on Gaussian quadrature.

‘quadv’
     Numerical integration using an adaptive vectorized Simpson’s rule.

‘quadl’
     Numerical integration using an adaptive Lobatto rule.

‘quadgk’
     Numerical integration using an adaptive Gauss-Konrod rule.

‘quadcc’
     Numerical integration using adaptive Clenshaw-Curtis rules.

‘trapz, cumtrapz’
     Numerical integration of data using the trapezoidal method.

The best quadrature algorithm to use depends on the integrand.  If you
have empirical data, rather than a function, the choice is ‘trapz’ or
‘cumtrapz’.  If you are uncertain about the characteristics of the
integrand, ‘quadcc’ will be the most robust as it can handle
discontinuities, singularities, oscillatory functions, and infinite
intervals.  When the integrand is smooth ‘quadgk’ may be the fastest of
the algorithms.

     Function    Characteristics
----------------------------------------------------------------------------
     quad        Low accuracy with nonsmooth integrands
     quadv       Medium accuracy with smooth integrands
     quadl       Medium accuracy with smooth integrands.  Slower than
                 quadgk.
     quadgk      Medium accuracy (1e-6 – 1e-9) with smooth integrands.
                 Handles oscillatory functions and infinite bounds
     quadcc      Low to High accuracy with nonsmooth/smooth integrands
                 Handles oscillatory functions, singularities, and
                 infinite bounds

   Here is an example of using ‘quad’ to integrate the function

       F(X) = X * sin (1/X) * sqrt (abs (1 - X))

from X = 0 to X = 3.

   This is a fairly difficult integration (plot the function over the
range of integration to see why).

   The first step is to define the function:

     function y = f (x)
       y = x .* sin (1./x) .* sqrt (abs (1 - x));
     endfunction

   Note the use of the ‘dot’ forms of the operators.  This is not
necessary for the ‘quad’ integrator, but is required by the other
integrators.  In any case, it makes it much easier to generate a set of
points for plotting because it is possible to call the function with a
vector argument to produce a vector result.

   The second step is to call quad with the limits of integration:

     [q, ier, nfun, err] = quad ("f", 0, 3)
          ⇒ 1.9819
          ⇒ 1
          ⇒ 5061
          ⇒ 1.1522e-07

   Although ‘quad’ returns a nonzero value for IER, the result is
reasonably accurate (to see why, examine what happens to the result if
you move the lower bound to 0.1, then 0.01, then 0.001, etc.).

   The function "f" can be the string name of a function, a function
handle, or an inline function.  These options make it quite easy to do
integration without having to fully define a function in an m-file.  For
example:

     # Verify integral (x^3) = x^4/4
     f = inline ("x.^3");
     quadgk (f, 0, 1)
          ⇒ 0.25000

     # Verify gamma function = (n-1)! for n = 4
     f = @(x) x.^3 .* exp (-x);
     quadcc (f, 0, Inf)
          ⇒ 6.0000

 -- : Q = quad (F, A, B)
 -- : Q = quad (F, A, B, TOL)
 -- : Q = quad (F, A, B, TOL, SING)
 -- : [Q, IER, NFUN, ERR] = quad (...)
     Numerically evaluate the integral of F from A to B using Fortran
     routines from QUADPACK.

     F is a function handle, inline function, or a string containing the
     name of the function to evaluate.  The function must have the form
     ‘y = f (x)’ where Y and X are scalars.

     A and B are the lower and upper limits of integration.  Either or
     both may be infinite.

     The optional argument TOL is a vector that specifies the desired
     accuracy of the result.  The first element of the vector is the
     desired absolute tolerance, and the second element is the desired
     relative tolerance.  To choose a relative test only, set the
     absolute tolerance to zero.  To choose an absolute test only, set
     the relative tolerance to zero.  Both tolerances default to ‘sqrt
     (eps)’ or approximately 1.5e-8.

     The optional argument SING is a vector of values at which the
     integrand is known to be singular.

     The result of the integration is returned in Q.

     IER contains an integer error code (0 indicates a successful
     integration).

     NFUN indicates the number of function evaluations that were made.

     ERR contains an estimate of the error in the solution.

     The function ‘quad_options’ can set other optional parameters for
     ‘quad’.

     Note: because ‘quad’ is written in Fortran it cannot be called
     recursively.  This prevents its use in integrating over more than
     one variable by routines ‘dblquad’ and ‘triplequad’.

     See also: *note quad_options: XREFquad_options, *note quadv:
     XREFquadv, *note quadl: XREFquadl, *note quadgk: XREFquadgk, *note
     quadcc: XREFquadcc, *note trapz: XREFtrapz, *note dblquad:
     XREFdblquad, *note triplequad: XREFtriplequad.

 -- : quad_options ()
 -- : val = quad_options (OPT)
 -- : quad_options (OPT, VAL)
     Query or set options for the function ‘quad’.

     When called with no arguments, the names of all available options
     and their current values are displayed.

     Given one argument, return the value of the option OPT.

     When called with two arguments, ‘quad_options’ sets the option OPT
     to value VAL.

     Options include

     "absolute tolerance"
          Absolute tolerance; may be zero for pure relative error test.

     "relative tolerance"
          Non-negative relative tolerance.  If the absolute tolerance is
          zero, the relative tolerance must be greater than or equal to
          ‘max (50*eps, 0.5e-28)’.

     "single precision absolute tolerance"
          Absolute tolerance for single precision; may be zero for pure
          relative error test.

     "single precision relative tolerance"
          Non-negative relative tolerance for single precision.  If the
          absolute tolerance is zero, the relative tolerance must be
          greater than or equal to ‘max (50*eps, 0.5e-28)’.

 -- : Q = quadv (F, A, B)
 -- : Q = quadv (F, A, B, TOL)
 -- : Q = quadv (F, A, B, TOL, TRACE)
 -- : Q = quadv (F, A, B, TOL, TRACE, P1, P2, ...)
 -- : [Q, NFUN] = quadv (...)

     Numerically evaluate the integral of F from A to B using an
     adaptive Simpson’s rule.

     F is a function handle, inline function, or string containing the
     name of the function to evaluate.  ‘quadv’ is a vectorized version
     of ‘quad’ and the function defined by F must accept a scalar or
     vector as input and return a scalar, vector, or array as output.

     A and B are the lower and upper limits of integration.  Both limits
     must be finite.

     The optional argument TOL defines the absolute tolerance used to
     stop the adaptation procedure.  The default value is 1e-6.

     The algorithm used by ‘quadv’ involves recursively subdividing the
     integration interval and applying Simpson’s rule on each
     subinterval.  If TRACE is true then after computing each of these
     partial integrals display: (1) the total number of function
     evaluations, (2) the left end of the subinterval, (3) the length of
     the subinterval, (4) the approximation of the integral over the
     subinterval.

     Additional arguments P1, etc., are passed directly to the function
     F.  To use default values for TOL and TRACE, one may pass empty
     matrices ([]).

     The result of the integration is returned in Q.

     The optional output NFUN indicates the total number of function
     evaluations performed.

     Note: ‘quadv’ is written in Octave’s scripting language and can be
     used recursively in ‘dblquad’ and ‘triplequad’, unlike the ‘quad’
     function.

     See also: *note quad: XREFquad, *note quadl: XREFquadl, *note
     quadgk: XREFquadgk, *note quadcc: XREFquadcc, *note trapz:
     XREFtrapz, *note dblquad: XREFdblquad, *note triplequad:
     XREFtriplequad.

 -- : Q = quadl (F, A, B)
 -- : Q = quadl (F, A, B, TOL)
 -- : Q = quadl (F, A, B, TOL, TRACE)
 -- : Q = quadl (F, A, B, TOL, TRACE, P1, P2, ...)
 -- : [Q, NFUN] = quadl (...)

     Numerically evaluate the integral of F from A to B using an
     adaptive Lobatto rule.

     F is a function handle, inline function, or string containing the
     name of the function to evaluate.  The function F must be
     vectorized and return a vector of output values when given a vector
     of input values.

     A and B are the lower and upper limits of integration.  Both limits
     must be finite.

     The optional argument TOL defines the absolute tolerance with which
     to perform the integration.  The default value is 1e-6.

     The algorithm used by ‘quadl’ involves recursively subdividing the
     integration interval.  If TRACE is defined then for each
     subinterval display: (1) the total number of function evaluations,
     (2) the left end of the subinterval, (3) the length of the
     subinterval, (4) the approximation of the integral over the
     subinterval.

     Additional arguments P1, etc., are passed directly to the function
     F.  To use default values for TOL and TRACE, one may pass empty
     matrices ([]).

     The result of the integration is returned in Q.

     The optional output NFUN indicates the total number of function
     evaluations performed.

     Reference: W. Gander and W. Gautschi, ‘Adaptive Quadrature -
     Revisited’, BIT Vol.  40, No.  1, March 2000, pp.  84–101.
     <http://www.inf.ethz.ch/personal/gander/>

     See also: *note quad: XREFquad, *note quadv: XREFquadv, *note
     quadgk: XREFquadgk, *note quadcc: XREFquadcc, *note trapz:
     XREFtrapz, *note dblquad: XREFdblquad, *note triplequad:
     XREFtriplequad.

 -- : Q = quadgk (F, A, B)
 -- : Q = quadgk (F, A, B, ABSTOL)
 -- : Q = quadgk (F, A, B, ABSTOL, TRACE)
 -- : Q = quadgk (F, A, B, PROP, VAL, ...)
 -- : [Q, ERR] = quadgk (...)

     Numerically evaluate the integral of F from A to B using adaptive
     Gauss-Konrod quadrature.

     F is a function handle, inline function, or string containing the
     name of the function to evaluate.  The function F must be
     vectorized and return a vector of output values when given a vector
     of input values.

     A and B are the lower and upper limits of integration.  Either or
     both limits may be infinite or contain weak end singularities.
     Variable transformation will be used to treat any infinite
     intervals and weaken the singularities.  For example:

          quadgk (@(x) 1 ./ (sqrt (x) .* (x + 1)), 0, Inf)

     Note that the formulation of the integrand uses the
     element-by-element operator ‘./’ and all user functions to ‘quadgk’
     should do the same.

     The optional argument TOL defines the absolute tolerance used to
     stop the integration procedure.  The default value is 1e-10.

     The algorithm used by ‘quadgk’ involves subdividing the integration
     interval and evaluating each subinterval.  If TRACE is true then
     after computing each of these partial integrals display: (1) the
     number of subintervals at this step, (2) the current estimate of
     the error ERR, (3) the current estimate for the integral Q.

     Alternatively, properties of ‘quadgk’ can be passed to the function
     as pairs "PROP", VAL.  Valid properties are

     ‘AbsTol’
          Define the absolute error tolerance for the quadrature.  The
          default absolute tolerance is 1e-10 (1e-5 for single).

     ‘RelTol’
          Define the relative error tolerance for the quadrature.  The
          default relative tolerance is 1e-6 (1e-4 for single).

     ‘MaxIntervalCount’
          ‘quadgk’ initially subdivides the interval on which to perform
          the quadrature into 10 intervals.  Subintervals that have an
          unacceptable error are subdivided and re-evaluated.  If the
          number of subintervals exceeds 650 subintervals at any point
          then a poor convergence is signaled and the current estimate
          of the integral is returned.  The property "MaxIntervalCount"
          can be used to alter the number of subintervals that can exist
          before exiting.

     ‘WayPoints’
          Discontinuities in the first derivative of the function to
          integrate can be flagged with the "WayPoints" property.  This
          forces the ends of a subinterval to fall on the breakpoints of
          the function and can result in significantly improved
          estimation of the error in the integral, faster computation,
          or both.  For example,

               quadgk (@(x) abs (1 - x.^2), 0, 2, "Waypoints", 1)

          signals the breakpoint in the integrand at ‘X = 1’.

     ‘Trace’
          If logically true ‘quadgk’ prints information on the
          convergence of the quadrature at each iteration.

     If any of A, B, or WAYPOINTS is complex then the quadrature is
     treated as a contour integral along a piecewise continuous path
     defined by the above.  In this case the integral is assumed to have
     no edge singularities.  For example,

          quadgk (@(z) log (z), 1+1i, 1+1i, "WayPoints",
                  [1-1i, -1,-1i, -1+1i])

     integrates ‘log (z)’ along the square defined by ‘[1+1i, 1-1i,
     -1-1i, -1+1i]’.

     The result of the integration is returned in Q.

     ERR is an approximate bound on the error in the integral ‘abs (Q -
     I)’, where I is the exact value of the integral.

     Reference: L.F. Shampine, ‘"Vectorized adaptive quadrature in
     MATLAB"’, Journal of Computational and Applied Mathematics, pp.
     131–140, Vol 211, Issue 2, Feb 2008.

     See also: *note quad: XREFquad, *note quadv: XREFquadv, *note
     quadl: XREFquadl, *note quadcc: XREFquadcc, *note trapz: XREFtrapz,
     *note dblquad: XREFdblquad, *note triplequad: XREFtriplequad.

 -- : Q = quadcc (F, A, B)
 -- : Q = quadcc (F, A, B, TOL)
 -- : Q = quadcc (F, A, B, TOL, SING)
 -- : [Q, ERR, NR_POINTS] = quadcc (...)
     Numerically evaluate the integral of F from A to B using
     doubly-adaptive Clenshaw-Curtis quadrature.

     F is a function handle, inline function, or string containing the
     name of the function to evaluate.  The function F must be
     vectorized and must return a vector of output values if given a
     vector of input values.  For example,

          f = @(x) x .* sin (1./x) .* sqrt (abs (1 - x));

     which uses the element-by-element “dot” form for all operators.

     A and B are the lower and upper limits of integration.  Either or
     both limits may be infinite.  ‘quadcc’ handles an inifinite limit
     by substituting the variable of integration with ‘x = tan
     (pi/2*u)’.

     The optional argument TOL defines the relative tolerance used to
     stop the integration procedure.  The default value is 1e-6.

     The optional argument SING contains a list of points where the
     integrand has known singularities, or discontinuities in any of its
     derivatives, inside the integration interval.  For the example
     above, which has a discontinuity at x=1, the call to ‘quadcc’ would
     be as follows

          int = quadcc (f, a, b, 1.0e-6, [ 1 ]);

     The result of the integration is returned in Q.

     ERR is an estimate of the absolute integration error.

     NR_POINTS is the number of points at which the integrand was
     evaluated.

     If the adaptive integration did not converge, the value of ERR will
     be larger than the requested tolerance.  Therefore, it is
     recommended to verify this value for difficult integrands.

     ‘quadcc’ is capable of dealing with non-numeric values of the
     integrand such as ‘NaN’ or ‘Inf’.  If the integral diverges, and
     ‘quadcc’ detects this, then a warning is issued and ‘Inf’ or ‘-Inf’
     is returned.

     Note: ‘quadcc’ is a general purpose quadrature algorithm and, as
     such, may be less efficient for a smooth or otherwise well-behaved
     integrand than other methods such as ‘quadgk’.

     The algorithm uses Clenshaw-Curtis quadrature rules of increasing
     degree in each interval and bisects the interval if either the
     function does not appear to be smooth or a rule of maximum degree
     has been reached.  The error estimate is computed from the L2-norm
     of the difference between two successive interpolations of the
     integrand over the nodes of the respective quadrature rules.

     Reference: P. Gonnet, ‘Increasing the Reliability of Adaptive
     Quadrature Using Explicit Interpolants’, ACM Transactions on
     Mathematical Software, Vol.  37, Issue 3, Article No.  3, 2010.

     See also: *note quad: XREFquad, *note quadv: XREFquadv, *note
     quadl: XREFquadl, *note quadgk: XREFquadgk, *note trapz: XREFtrapz,
     *note dblquad: XREFdblquad, *note triplequad: XREFtriplequad.

   Sometimes one does not have the function, but only the raw (x, y)
points from which to perform an integration.  This can occur when
collecting data in an experiment.  The ‘trapz’ function can integrate
these values as shown in the following example where "data" has been
collected on the cosine function over the range [0, pi/2).

     x = 0:0.1:pi/2;  # Uniformly spaced points
     y = cos (x);
     trapz (x, y)
          ⇒ 0.99666

   The answer is reasonably close to the exact value of 1.  Ordinary
quadrature is sensitive to the characteristics of the integrand.
Empirical integration depends not just on the integrand, but also on the
particular points chosen to represent the function.  Repeating the
example above with the sine function over the range [0, pi/2) produces
far inferior results.

     x = 0:0.1:pi/2;  # Uniformly spaced points
     y = sin (x);
     trapz (x, y)
          ⇒ 0.92849

   However, a slightly different choice of data points can change the
result significantly.  The same integration, with the same number of
points, but spaced differently produces a more accurate answer.

     x = linspace (0, pi/2, 16);  # Uniformly spaced, but including endpoint
     y = sin (x);
     trapz (x, y)
          ⇒ 0.99909

   In general there may be no way of knowing the best distribution of
points ahead of time.  Or the points may come from an experiment where
there is no freedom to select the best distribution.  In any case, one
must remain aware of this issue when using ‘trapz’.

 -- : Q = trapz (Y)
 -- : Q = trapz (X, Y)
 -- : Q = trapz (..., DIM)

     Numerically evaluate the integral of points Y using the trapezoidal
     method.

     ‘trapz (Y)’ computes the integral of Y along the first
     non-singleton dimension.  When the argument X is omitted an equally
     spaced X vector with unit spacing (1) is assumed.  ‘trapz (X, Y)’
     evaluates the integral with respect to the spacing in X and the
     values in Y.  This is useful if the points in Y have been sampled
     unevenly.

     If the optional DIM argument is given, operate along this
     dimension.

     Application Note: If X is not specified then unit spacing will be
     used.  To scale the integral to the correct value you must multiply
     by the actual spacing value (deltaX). As an example, the integral
     of x^3 over the range [0, 1] is x^4/4 or 0.25.  The following code
     uses ‘trapz’ to calculate the integral in three different ways.

          x = 0:0.1:1;
          y = x.^3;
          q = trapz (y)
            ⇒ q = 2.525   # No scaling
          q * 0.1
            ⇒ q = 0.2525  # Approximation to integral by scaling
          trapz (x, y)
            ⇒ q = 0.2525  # Same result by specifying X

     See also: *note cumtrapz: XREFcumtrapz.

 -- : Q = cumtrapz (Y)
 -- : Q = cumtrapz (X, Y)
 -- : Q = cumtrapz (..., DIM)
     Cumulative numerical integration of points Y using the trapezoidal
     method.

     ‘cumtrapz (Y)’ computes the cumulative integral of Y along the
     first non-singleton dimension.  Where ‘trapz’ reports only the
     overall integral sum, ‘cumtrapz’ reports the current partial sum
     value at each point of Y.

     When the argument X is omitted an equally spaced X vector with unit
     spacing (1) is assumed.  ‘cumtrapz (X, Y)’ evaluates the integral
     with respect to the spacing in X and the values in Y.  This is
     useful if the points in Y have been sampled unevenly.

     If the optional DIM argument is given, operate along this
     dimension.

     Application Note: If X is not specified then unit spacing will be
     used.  To scale the integral to the correct value you must multiply
     by the actual spacing value (deltaX).

     See also: *note trapz: XREFtrapz, *note cumsum: XREFcumsum.


File: octave.info,  Node: Orthogonal Collocation,  Next: Functions of Multiple Variables,  Prev: Functions of One Variable,  Up: Numerical Integration

23.2 Orthogonal Collocation
===========================

 -- : [R, AMAT, BMAT, Q] = colloc (N, "left", "right")
     Compute derivative and integral weight matrices for orthogonal
     collocation.

     Reference: J. Villadsen, M. L. Michelsen, ‘Solution of Differential
     Equation Models by Polynomial Approximation’.

   Here is an example of using ‘colloc’ to generate weight matrices for
solving the second order differential equation U’ - ALPHA * U” = 0 with
the boundary conditions U(0) = 0 and U(1) = 1.

   First, we can generate the weight matrices for N points (including
the endpoints of the interval), and incorporate the boundary conditions
in the right hand side (for a specific value of ALPHA).

     n = 7;
     alpha = 0.1;
     [r, a, b] = colloc (n-2, "left", "right");
     at = a(2:n-1,2:n-1);
     bt = b(2:n-1,2:n-1);
     rhs = alpha * b(2:n-1,n) - a(2:n-1,n);

   Then the solution at the roots R is

     u = [ 0; (at - alpha * bt) \ rhs; 1]
          ⇒ [ 0.00; 0.004; 0.01 0.00; 0.12; 0.62; 1.00 ]


File: octave.info,  Node: Functions of Multiple Variables,  Prev: Orthogonal Collocation,  Up: Numerical Integration

23.3 Functions of Multiple Variables
====================================

Octave does not have built-in functions for computing the integral of
functions of multiple variables directly.  It is possible, however, to
compute the integral of a function of multiple variables using the
existing functions for one-dimensional integrals.

   To illustrate how the integration can be performed, we will integrate
the function

     f(x, y) = sin(pi*x*y)*sqrt(x*y)

   for x and y between 0 and 1.

   The first approach creates a function that integrates f with respect
to x, and then integrates that function with respect to y.  Because
‘quad’ is written in Fortran it cannot be called recursively.  This
means that ‘quad’ cannot integrate a function that calls ‘quad’, and
hence cannot be used to perform the double integration.  Any of the
other integrators, however, can be used which is what the following code
demonstrates.

     function q = g(y)
       q = ones (size (y));
       for i = 1:length (y)
         f = @(x) sin (pi*x.*y(i)) .* sqrt (x.*y(i));
         q(i) = quadgk (f, 0, 1);
       endfor
     endfunction

     I = quadgk ("g", 0, 1)
           ⇒ 0.30022

   The above process can be simplified with the ‘dblquad’ and
‘triplequad’ functions for integrals over two and three variables.  For
example:

     I = dblquad (@(x, y) sin (pi*x.*y) .* sqrt (x.*y), 0, 1, 0, 1)
           ⇒ 0.30022

 -- : dblquad (F, XA, XB, YA, YB)
 -- : dblquad (F, XA, XB, YA, YB, TOL)
 -- : dblquad (F, XA, XB, YA, YB, TOL, QUADF)
 -- : dblquad (F, XA, XB, YA, YB, TOL, QUADF, ...)
     Numerically evaluate the double integral of F.

     F is a function handle, inline function, or string containing the
     name of the function to evaluate.  The function F must have the
     form z = f(x,y) where X is a vector and Y is a scalar.  It should
     return a vector of the same length and orientation as X.

     XA, YA and XB, YB are the lower and upper limits of integration for
     x and y respectively.  The underlying integrator determines whether
     infinite bounds are accepted.

     The optional argument TOL defines the absolute tolerance used to
     integrate each sub-integral.  The default value is 1e-6.

     The optional argument QUADF specifies which underlying integrator
     function to use.  Any choice but ‘quad’ is available and the
     default is ‘quadcc’.

     Additional arguments, are passed directly to F.  To use the default
     value for TOL or QUADF one may pass ’:’ or an empty matrix ([]).

     See also: *note triplequad: XREFtriplequad, *note quad: XREFquad,
     *note quadv: XREFquadv, *note quadl: XREFquadl, *note quadgk:
     XREFquadgk, *note quadcc: XREFquadcc, *note trapz: XREFtrapz.

 -- : triplequad (F, XA, XB, YA, YB, ZA, ZB)
 -- : triplequad (F, XA, XB, YA, YB, ZA, ZB, TOL)
 -- : triplequad (F, XA, XB, YA, YB, ZA, ZB, TOL, QUADF)
 -- : triplequad (F, XA, XB, YA, YB, ZA, ZB, TOL, QUADF, ...)
     Numerically evaluate the triple integral of F.

     F is a function handle, inline function, or string containing the
     name of the function to evaluate.  The function F must have the
     form w = f(x,y,z) where either X or Y is a vector and the remaining
     inputs are scalars.  It should return a vector of the same length
     and orientation as X or Y.

     XA, YA, ZA and XB, YB, ZB are the lower and upper limits of
     integration for x, y, and z respectively.  The underlying
     integrator determines whether infinite bounds are accepted.

     The optional argument TOL defines the absolute tolerance used to
     integrate each sub-integral.  The default value is 1e-6.

     The optional argument QUADF specifies which underlying integrator
     function to use.  Any choice but ‘quad’ is available and the
     default is ‘quadcc’.

     Additional arguments, are passed directly to F.  To use the default
     value for TOL or QUADF one may pass ’:’ or an empty matrix ([]).

     See also: *note dblquad: XREFdblquad, *note quad: XREFquad, *note
     quadv: XREFquadv, *note quadl: XREFquadl, *note quadgk: XREFquadgk,
     *note quadcc: XREFquadcc, *note trapz: XREFtrapz.

   The above mentioned approach works, but is fairly slow, and that
problem increases exponentially with the dimensionality of the integral.
Another possible solution is to use Orthogonal Collocation as described
in the previous section (*note Orthogonal Collocation::).  The integral
of a function f(x,y) for x and y between 0 and 1 can be approximated
using n points by the sum over ‘i=1:n’ and ‘j=1:n’ of
‘q(i)*q(j)*f(r(i),r(j))’, where q and r is as returned by ‘colloc (n)’.
The generalization to more than two variables is straight forward.  The
following code computes the studied integral using n=8 points.

     f = @(x,y) sin (pi*x*y') .* sqrt (x*y');
     n = 8;
     [t, ~, ~, q] = colloc (n);
     I = q'*f(t,t)*q;
           ⇒ 0.30022

It should be noted that the number of points determines the quality of
the approximation.  If the integration needs to be performed between a
and b, instead of 0 and 1, then a change of variables is needed.


File: octave.info,  Node: Differential Equations,  Next: Optimization,  Prev: Numerical Integration,  Up: Top

24 Differential Equations
*************************

Octave has built-in functions for solving ordinary differential
equations, and differential-algebraic equations.  All solvers are based
on reliable ODE routines written in Fortran.

* Menu:

* Ordinary Differential Equations::
* Differential-Algebraic Equations::


File: octave.info,  Node: Ordinary Differential Equations,  Next: Differential-Algebraic Equations,  Up: Differential Equations

24.1 Ordinary Differential Equations
====================================

The function ‘lsode’ can be used to solve ODEs of the form

     dx
     -- = f (x, t)
     dt

using Hindmarsh’s ODE solver LSODE.

 -- : [X, ISTATE, MSG] = lsode (FCN, X_0, T)
 -- : [X, ISTATE, MSG] = lsode (FCN, X_0, T, T_CRIT)
     Ordinary Differential Equation (ODE) solver.

     The set of differential equations to solve is

          dx
          -- = f (x, t)
          dt

     with

          x(t_0) = x_0

     The solution is returned in the matrix X, with each row
     corresponding to an element of the vector T.  The first element of
     T should be t_0 and should correspond to the initial state of the
     system X_0, so that the first row of the output is X_0.

     The first argument, FCN, is a string, inline, or function handle
     that names the function f to call to compute the vector of right
     hand sides for the set of equations.  The function must have the
     form

          XDOT = f (X, T)

     in which XDOT and X are vectors and T is a scalar.

     If FCN is a two-element string array or a two-element cell array of
     strings, inline functions, or function handles, the first element
     names the function f described above, and the second element names
     a function to compute the Jacobian of f.  The Jacobian function
     must have the form

          JAC = j (X, T)

     in which JAC is the matrix of partial derivatives

                       | df_1  df_1       df_1 |
                       | ----  ----  ...  ---- |
                       | dx_1  dx_2       dx_N |
                       |                       |
                       | df_2  df_2       df_2 |
                       | ----  ----  ...  ---- |
                df_i   | dx_1  dx_2       dx_N |
          jac = ---- = |                       |
                dx_j   |  .    .     .    .    |
                       |  .    .      .   .    |
                       |  .    .       .  .    |
                       |                       |
                       | df_N  df_N       df_N |
                       | ----  ----  ...  ---- |
                       | dx_1  dx_2       dx_N |

     The second argument specifies the initial state of the system x_0.
     The third argument is a vector, T, specifying the time values for
     which a solution is sought.

     The fourth argument is optional, and may be used to specify a set
     of times that the ODE solver should not integrate past.  It is
     useful for avoiding difficulties with singularities and points
     where there is a discontinuity in the derivative.

     After a successful computation, the value of ISTATE will be 2
     (consistent with the Fortran version of LSODE).

     If the computation is not successful, ISTATE will be something
     other than 2 and MSG will contain additional information.

     You can use the function ‘lsode_options’ to set optional parameters
     for ‘lsode’.

     See also: *note daspk: XREFdaspk, *note dassl: XREFdassl, *note
     dasrt: XREFdasrt.

 -- : lsode_options ()
 -- : val = lsode_options (OPT)
 -- : lsode_options (OPT, VAL)
     Query or set options for the function ‘lsode’.

     When called with no arguments, the names of all available options
     and their current values are displayed.

     Given one argument, return the value of the option OPT.

     When called with two arguments, ‘lsode_options’ sets the option OPT
     to value VAL.

     Options include

     "absolute tolerance"
          Absolute tolerance.  May be either vector or scalar.  If a
          vector, it must match the dimension of the state vector.

     "relative tolerance"
          Relative tolerance parameter.  Unlike the absolute tolerance,
          this parameter may only be a scalar.

          The local error test applied at each integration step is

                 abs (local error in x(i)) <= ...
                     rtol * abs (y(i)) + atol(i)

     "integration method"
          A string specifying the method of integration to use to solve
          the ODE system.  Valid values are

          "adams"
          "non-stiff"
               No Jacobian used (even if it is available).

          "bdf"
          "stiff"
               Use stiff backward differentiation formula (BDF) method.
               If a function to compute the Jacobian is not supplied,
               ‘lsode’ will compute a finite difference approximation of
               the Jacobian matrix.

     "initial step size"
          The step size to be attempted on the first step (default is
          determined automatically).

     "maximum order"
          Restrict the maximum order of the solution method.  If using
          the Adams method, this option must be between 1 and 12.
          Otherwise, it must be between 1 and 5, inclusive.

     "maximum step size"
          Setting the maximum stepsize will avoid passing over very
          large regions (default is not specified).

     "minimum step size"
          The minimum absolute step size allowed (default is 0).

     "step limit"
          Maximum number of steps allowed (default is 100000).

   Here is an example of solving a set of three differential equations
using ‘lsode’.  Given the function

     ## oregonator differential equation
     function xdot = f (x, t)

       xdot = zeros (3,1);

       xdot(1) = 77.27 * (x(2) - x(1)*x(2) + x(1) ...
                 - 8.375e-06*x(1)^2);
       xdot(2) = (x(3) - x(1)*x(2) - x(2)) / 77.27;
       xdot(3) = 0.161*(x(1) - x(3));

     endfunction

and the initial condition ‘x0 = [ 4; 1.1; 4 ]’, the set of equations can
be integrated using the command

     t = linspace (0, 500, 1000);

     y = lsode ("f", x0, t);

   If you try this, you will see that the value of the result changes
dramatically between T = 0 and 5, and again around T = 305.  A more
efficient set of output points might be

     t = [0, logspace(-1, log10(303), 150), ...
             logspace(log10(304), log10(500), 150)];

   See Alan C. Hindmarsh, ‘ODEPACK, A Systematized Collection of ODE
Solvers’, in Scientific Computing, R. S. Stepleman, editor, (1983) for
more information about the inner workings of ‘lsode’.

   An m-file for the differential equation used above is included with
the Octave distribution in the examples directory under the name
‘oregonator.m’.

* Menu:

* Matlab-compatible solvers::


File: octave.info,  Node: Matlab-compatible solvers,  Up: Ordinary Differential Equations

24.1.1 Matlab-compatible solvers
--------------------------------

Octave also provides a set of solvers for initial value problems for
Ordinary Differential Equations that have a MATLAB-compatible interface.
The options for this class of methods are set using the functions.

   • ‘odeset’

   • ‘odeget’

   Currently implemented solvers are:

   • Runge-Kutta methods

        • ‘ode45’ Integrates a system of non–stiff ordinary differential
          equations (non–stiff ODEs and DAEs) using second order
          Dormand-Prince method.  This is a fourth–order accurate
          integrator therefore the local error normally expected is
          O(h^5).  This solver requires six function evaluations per
          integration step.

        • ‘ode23’ Integrates a system of non–stiff ordinary differential
          equations (non-stiff ODEs and DAEs) using second order
          Bogacki-Shampine method.  This is a second-order accurate
          integrator therefore the local error normally expected is
          O(h^3).  This solver requires three function evaluations per
          integration step.

 -- : [T, Y] = ode45 (FUN, TRANGE, INIT)
 -- : [T, Y] = ode45 (FUN, TRANGE, INIT, ODE_OPT)
 -- : [T, Y, TE, YE, IE] = ode45 (...)
 -- : SOLUTION = ode45 (...)
 -- : ode45 (...)

     Solve a set of non-stiff Ordinary Differential Equations (non-stiff
     ODEs) with the well known explicit Dormand-Prince method of order
     4.

     FUN is a function handle, inline function, or string containing the
     name of the function that defines the ODE: ‘y' = f(t,y)’.  The
     function must accept two inputs where the first is time T and the
     second is a column vector of unknowns Y.

     TRANGE specifies the time interval over which the ODE will be
     evaluated.  Typically, it is a two-element vector specifying the
     initial and final times (‘[tinit, tfinal]’).  If there are more
     than two elements then the solution will also be evaluated at these
     intermediate time instances.

     By default, ‘ode45’ uses an adaptive timestep with the
     ‘integrate_adaptive’ algorithm.  The tolerance for the timestep
     computation may be changed by using the options "RelTol" and
     "AbsTol".

     INIT contains the initial value for the unknowns.  If it is a row
     vector then the solution Y will be a matrix in which each column is
     the solution for the corresponding initial value in INIT.

     The optional fourth argument ODE_OPT specifies non-default options
     to the ODE solver.  It is a structure generated by ‘odeset’.

     The function typically returns two outputs.  Variable T is a column
     vector and contains the times where the solution was found.  The
     output Y is a matrix in which each column refers to a different
     unknown of the problem and each row corresponds to a time in T.

     The output can also be returned as a structure SOLUTION which has a
     field X containing a row vector of times where the solution was
     evaluated and a field Y containing the solution matrix such that
     each column corresponds to a time in X.  Use ‘fieldnames
     (SOLUTION)’ to see the other fields and additional information
     returned.

     If no output arguments are requested, and no ‘OutputFcn’ is
     specified in ODE_OPT, then the ‘OutputFcn’ is set to ‘odeplot’ and
     the results of the solver are plotted immediately.

     If using the "Events" option then three additional outputs may be
     returned.  TE holds the time when an Event function returned a
     zero.  YE holds the value of the solution at time TE.  IE contains
     an index indicating which Event function was triggered in the case
     of multiple Event functions.

     Example: Solve the Van der Pol equation

          fvdp = @(T,Y) [Y(2); (1 - Y(1)^2) * Y(2) - Y(1)];
          [T,Y] = ode45 (fvdp, [0, 20], [2, 0]);

     See also: *note odeset: XREFodeset, *note odeget: XREFodeget, *note
     ode23: XREFode23.

 -- : [T, Y] = ode23 (FUN, TRANGE, INIT)
 -- : [T, Y] = ode23 (FUN, TRANGE, INIT, ODE_OPT)
 -- : [T, Y, TE, YE, IE] = ode23 (...)
 -- : SOLUTION = ode23 (...)
 -- : ode23 (...)

     Solve a set of non-stiff Ordinary Differential Equations (non-stiff
     ODEs) with the well known explicit Bogacki-Shampine method of order
     3.  For the definition of this method see
     <http://en.wikipedia.org/wiki/List_of_Runge%E2%80%93Kutta_methods>.

     FUN is a function handle, inline function, or string containing the
     name of the function that defines the ODE: ‘y' = f(t,y)’.  The
     function must accept two inputs where the first is time T and the
     second is a column vector of unknowns Y.

     TRANGE specifies the time interval over which the ODE will be
     evaluated.  Typically, it is a two-element vector specifying the
     initial and final times (‘[tinit, tfinal]’).  If there are more
     than two elements then the solution will also be evaluated at these
     intermediate time instances.

     By default, ‘ode23’ uses an adaptive timestep with the
     ‘integrate_adaptive’ algorithm.  The tolerance for the timestep
     computation may be changed by using the options "RelTol" and
     "AbsTol".

     INIT contains the initial value for the unknowns.  If it is a row
     vector then the solution Y will be a matrix in which each column is
     the solution for the corresponding initial value in INIT.

     The optional fourth argument ODE_OPT specifies non-default options
     to the ODE solver.  It is a structure generated by ‘odeset’.

     The function typically returns two outputs.  Variable T is a column
     vector and contains the times where the solution was found.  The
     output Y is a matrix in which each column refers to a different
     unknown of the problem and each row corresponds to a time in T.

     The output can also be returned as a structure SOLUTION which has a
     field X containing a row vector of times where the solution was
     evaluated and a field Y containing the solution matrix such that
     each column corresponds to a time in X.  Use ‘fieldnames
     (SOLUTION)’ to see the other fields and additional information
     returned.

     If no output arguments are requested, and no ‘OutputFcn’ is
     specified in ODE_OPT, then the ‘OutputFcn’ is set to ‘odeplot’ and
     the results of the solver are plotted immediately.

     If using the "Events" option then three additional outputs may be
     returned.  TE holds the time when an Event function returned a
     zero.  YE holds the value of the solution at time TE.  IE contains
     an index indicating which Event function was triggered in the case
     of multiple Event functions.

     Example: Solve the Van der Pol equation

          fvdp = @(T,Y) [Y(2); (1 - Y(1)^2) * Y(2) - Y(1)];
          [T,Y] = ode23 (fvdp, [0, 20], [2, 0]);

     See also: *note odeset: XREFodeset, *note odeget: XREFodeget, *note
     ode45: XREFode45.

 -- : ODESTRUCT = odeset ()
 -- : ODESTRUCT = odeset ("FIELD1", VALUE1, "FIELD2", VALUE2, ...)
 -- : ODESTRUCT = odeset (OLDSTRUCT, "FIELD1", VALUE1, "FIELD2", VALUE2,
          ...)
 -- : ODESTRUCT = odeset (OLDSTRUCT, NEWSTRUCT)
 -- : odeset ()

     Create or modify an ODE options structure.

     When called with no input argument and one output argument, return
     a new ODE options structure that contains all possible fields
     initialized to their default values.  If no output argument is
     requested, display a list of the common ODE solver options along
     with their default value.

     If called with name-value input argument pairs "FIELD1", "VALUE1",
     "FIELD2", "VALUE2", ... return a new ODE options structure with all
     the most common option fields initialized, *and* set the values of
     the fields "FIELD1", "FIELD2", ... to the values VALUE1, VALUE2,
     ....

     If called with an input structure OLDSTRUCT then overwrite the
     values of the options "FIELD1", "FIELD2", ... with new values
     VALUE1, VALUE2, ... and return the modified structure.

     When called with two input ODE options structures OLDSTRUCT and
     NEWSTRUCT overwrite all values from the structure OLDSTRUCT with
     new values from the structure NEWSTRUCT.  Empty values in NEWSTRUCT
     will not overwrite values in OLDSTRUCT.

     The most commonly used ODE options, which are always assigned a
     value by odeset, are the following:

     AbsTol
          Absolute error tolerance.

     BDF
          Use BDF formulas in implicit multistep methods.  *Note:* This
          option is not yet implemented.

     Events
          Event function.  An event function must have the form ‘[value,
          isterminal, direction] = my_events_f (t, y)’

     InitialSlope
          Consistent initial slope vector for DAE solvers.

     InitialStep
          Initial time step size.

     Jacobian
          Jacobian matrix, specified as a constant matrix or a function
          of time and state.

     JConstant
          Specify whether the Jacobian is a constant matrix or depends
          on the state.

     JPattern
          If the Jacobian matrix is sparse and non-constant but
          maintains a constant sparsity pattern, specify the sparsity
          pattern.

     Mass
          Mass matrix, specified as a constant matrix or a function of
          time and state.

     MassSingular
          Specify whether the mass matrix is singular.  Accepted values
          include "yes", "no", "maybe".

     MaxOrder
          Maximum order of formula.

     MaxStep
          Maximum time step value.

     MStateDependence
          Specify whether the mass matrix depends on the state or only
          on time.

     MvPattern
          If the mass matrix is sparse and non-constant but maintains a
          constant sparsity pattern, specify the sparsity pattern.
          *Note:* This option is not yet implemented.

     NonNegative
          Specify elements of the state vector that are expected to
          remain nonnegative during the simulation.

     NormControl
          Control error relative to the 2-norm of the solution, rather
          than its absolute value.

     OutputFcn
          Function to monitor the state during the simulation.  For the
          form of the function to use see odeplot.

     OutputSel
          Indices of elements of the state vector to be passed to the
          output monitoring function.

     Refine
          Specify whether output should be returned only at the end of
          each time step or also at intermediate time instances.  The
          value should be a scalar indicating the number of equally
          spaced time points to use within each timestep at which to
          return output.  *Note:* This option is not yet implemented.

     RelTol
          Relative error tolerance.

     Stats
          Print solver statistics after simulation.

     Vectorized
          Specify whether odefun can be passed multiple values of the
          state at once.

     Field names that are not in the above list are also accepted and
     added to the result structure.

     See also: *note odeget: XREFodeget.

 -- : VAL = odeget (ODE_OPT, FIELD)
 -- : VAL = odeget (ODE_OPT, FIELD, DEFAULT)

     Query the value of the property FIELD in the ODE options structure
     ODE_OPT.

     If called with two input arguments and the first input argument
     ODE_OPT is an ODE option structure and the second input argument
     FIELD is a string specifying an option name, then return the option
     value VAL corresponding to FIELD from ODE_OPT.

     If called with an optional third input argument, and FIELD is not
     set in the structure ODE_OPT, then return the default value DEFAULT
     instead.

     See also: *note odeset: XREFodeset.

 -- : STOP_SOLVE = odeplot (T, Y, FLAG)

     Open a new figure window and plot the solution of an ode problem at
     each time step during the integration.

     The types and values of the input parameters T and Y depend on the
     input FLAG that is of type string.  Valid values of FLAG are:

     ‘"init"’
          The input T must be a column vector of length 2 with the first
          and last time step (‘[TFIRST TLAST]’.  The input Y contains
          the initial conditions for the ode problem (Y0).

     ‘""’
          The input T must be a scalar double specifying the time for
          which the solution in input Y was calculated.

     ‘"done"’
          The inputs should be empty, but are ignored if they are
          present.

     ‘odeplot’ always returns false, i.e., don’t stop the ode solver.

     Example: solve an anonymous implementation of the "Van der Pol"
     equation and display the results while solving.

          fvdp = @(t,y) [y(2); (1 - y(1)^2) * y(2) - y(1)];

          opt = odeset ("OutputFcn", @odeplot, "RelTol", 1e-6);
          sol = ode45 (fvdp, [0 20], [2 0], opt);

     Background Information: This function is called by an ode solver
     function if it was specified in the "OutputFcn" property of an
     options structure created with ‘odeset’.  The ode solver will
     initially call the function with the syntax ‘odeplot ([TFIRST,
     TLAST], Y0, "init")’.  The function initializes internal variables,
     creates a new figure window, and sets the x limits of the plot.
     Subsequently, at each time step during the integration the ode
     solver calls ‘odeplot (T, Y, [])’.  At the end of the solution the
     ode solver calls ‘odeplot ([], [], "done")’ so that odeplot can
     perform any clean-up actions required.

     See also: *note odeset: XREFodeset, *note odeget: XREFodeget, *note
     ode23: XREFode23, *note ode45: XREFode45.


File: octave.info,  Node: Differential-Algebraic Equations,  Prev: Ordinary Differential Equations,  Up: Differential Equations

24.2 Differential-Algebraic Equations
=====================================

The function ‘daspk’ can be used to solve DAEs of the form

     0 = f (x-dot, x, t),    x(t=0) = x_0, x-dot(t=0) = x-dot_0

where x-dot is the derivative of x.  The equation is solved using
Petzold’s DAE solver DASPK.

 -- : [X, XDOT, ISTATE, MSG] = daspk (FCN, X_0, XDOT_0, T, T_CRIT)
     Solve the set of differential-algebraic equations

          0 = f (x, xdot, t)

     with

          x(t_0) = x_0, xdot(t_0) = xdot_0

     The solution is returned in the matrices X and XDOT, with each row
     in the result matrices corresponding to one of the elements in the
     vector T.  The first element of T should be t_0 and correspond to
     the initial state of the system X_0 and its derivative XDOT_0, so
     that the first row of the output X is X_0 and the first row of the
     output XDOT is XDOT_0.

     The first argument, FCN, is a string, inline, or function handle
     that names the function f to call to compute the vector of
     residuals for the set of equations.  It must have the form

          RES = f (X, XDOT, T)

     in which X, XDOT, and RES are vectors, and T is a scalar.

     If FCN is a two-element string array or a two-element cell array of
     strings, inline functions, or function handles, the first element
     names the function f described above, and the second element names
     a function to compute the modified Jacobian

                df       df
          jac = -- + c ------
                dx     d xdot

     The modified Jacobian function must have the form


          JAC = j (X, XDOT, T, C)


     The second and third arguments to ‘daspk’ specify the initial
     condition of the states and their derivatives, and the fourth
     argument specifies a vector of output times at which the solution
     is desired, including the time corresponding to the initial
     condition.

     The set of initial states and derivatives are not strictly required
     to be consistent.  If they are not consistent, you must use the
     ‘daspk_options’ function to provide additional information so that
     ‘daspk’ can compute a consistent starting point.

     The fifth argument is optional, and may be used to specify a set of
     times that the DAE solver should not integrate past.  It is useful
     for avoiding difficulties with singularities and points where there
     is a discontinuity in the derivative.

     After a successful computation, the value of ISTATE will be greater
     than zero (consistent with the Fortran version of DASPK).

     If the computation is not successful, the value of ISTATE will be
     less than zero and MSG will contain additional information.

     You can use the function ‘daspk_options’ to set optional parameters
     for ‘daspk’.

     See also: *note dassl: XREFdassl.

 -- : daspk_options ()
 -- : val = daspk_options (OPT)
 -- : daspk_options (OPT, VAL)
     Query or set options for the function ‘daspk’.

     When called with no arguments, the names of all available options
     and their current values are displayed.

     Given one argument, return the value of the option OPT.

     When called with two arguments, ‘daspk_options’ sets the option OPT
     to value VAL.

     Options include

     "absolute tolerance"
          Absolute tolerance.  May be either vector or scalar.  If a
          vector, it must match the dimension of the state vector, and
          the relative tolerance must also be a vector of the same
          length.

     "relative tolerance"
          Relative tolerance.  May be either vector or scalar.  If a
          vector, it must match the dimension of the state vector, and
          the absolute tolerance must also be a vector of the same
          length.

          The local error test applied at each integration step is

                 abs (local error in x(i))
                      <= rtol(i) * abs (Y(i)) + atol(i)

     "compute consistent initial condition"
          Denoting the differential variables in the state vector by
          ‘Y_d’ and the algebraic variables by ‘Y_a’, ‘ddaspk’ can solve
          one of two initialization problems:

            1. Given Y_d, calculate Y_a and Y’_d

            2. Given Y’, calculate Y.

          In either case, initial values for the given components are
          input, and initial guesses for the unknown components must
          also be provided as input.  Set this option to 1 to solve the
          first problem, or 2 to solve the second (the default is 0, so
          you must provide a set of initial conditions that are
          consistent).

          If this option is set to a nonzero value, you must also set
          the "algebraic variables" option to declare which variables in
          the problem are algebraic.

     "use initial condition heuristics"
          Set to a nonzero value to use the initial condition heuristics
          options described below.

     "initial condition heuristics"
          A vector of the following parameters that can be used to
          control the initial condition calculation.

          ‘MXNIT’
               Maximum number of Newton iterations (default is 5).

          ‘MXNJ’
               Maximum number of Jacobian evaluations (default is 6).

          ‘MXNH’
               Maximum number of values of the artificial stepsize
               parameter to be tried if the "compute consistent initial
               condition" option has been set to 1 (default is 5).

               Note that the maximum total number of Newton iterations
               allowed is ‘MXNIT*MXNJ*MXNH’ if the "compute consistent
               initial condition" option has been set to 1 and
               ‘MXNIT*MXNJ’ if it is set to 2.

          ‘LSOFF’
               Set to a nonzero value to disable the linesearch
               algorithm (default is 0).

          ‘STPTOL’
               Minimum scaled step in linesearch algorithm (default is
               eps^(2/3)).

          ‘EPINIT’
               Swing factor in the Newton iteration convergence test.
               The test is applied to the residual vector, premultiplied
               by the approximate Jacobian.  For convergence, the
               weighted RMS norm of this vector (scaled by the error
               weights) must be less than ‘EPINIT*EPCON’, where ‘EPCON’
               = 0.33 is the analogous test constant used in the time
               steps.  The default is ‘EPINIT’ = 0.01.

     "print initial condition info"
          Set this option to a nonzero value to display detailed
          information about the initial condition calculation (default
          is 0).

     "exclude algebraic variables from error test"
          Set to a nonzero value to exclude algebraic variables from the
          error test.  You must also set the "algebraic variables"
          option to declare which variables in the problem are algebraic
          (default is 0).

     "algebraic variables"
          A vector of the same length as the state vector.  A nonzero
          element indicates that the corresponding element of the state
          vector is an algebraic variable (i.e., its derivative does not
          appear explicitly in the equation set).

          This option is required by the "compute consistent initial
          condition" and "exclude algebraic variables from error test"
          options.

     "enforce inequality constraints"
          Set to one of the following values to enforce the inequality
          constraints specified by the "inequality constraint types"
          option (default is 0).

            1. To have constraint checking only in the initial condition
               calculation.

            2. To enforce constraint checking during the integration.

            3. To enforce both options 1 and 2.

     "inequality constraint types"
          A vector of the same length as the state specifying the type
          of inequality constraint.  Each element of the vector
          corresponds to an element of the state and should be assigned
          one of the following codes

          -2
               Less than zero.

          -1
               Less than or equal to zero.

          0
               Not constrained.

          1
               Greater than or equal to zero.

          2
               Greater than zero.

          This option only has an effect if the "enforce inequality
          constraints" option is nonzero.

     "initial step size"
          Differential-algebraic problems may occasionally suffer from
          severe scaling difficulties on the first step.  If you know a
          great deal about the scaling of your problem, you can help to
          alleviate this problem by specifying an initial stepsize
          (default is computed automatically).

     "maximum order"
          Restrict the maximum order of the solution method.  This
          option must be between 1 and 5, inclusive (default is 5).

     "maximum step size"
          Setting the maximum stepsize will avoid passing over very
          large regions (default is not specified).

   Octave also includes DASSL, an earlier version of DASPK, and DASRT,
which can be used to solve DAEs with constraints (stopping conditions).

 -- : [X, XDOT, ISTATE, MSG] = dassl (FCN, X_0, XDOT_0, T, T_CRIT)
     Solve the set of differential-algebraic equations

          0 = f (x, xdot, t)

     with

          x(t_0) = x_0, xdot(t_0) = xdot_0

     The solution is returned in the matrices X and XDOT, with each row
     in the result matrices corresponding to one of the elements in the
     vector T.  The first element of T should be t_0 and correspond to
     the initial state of the system X_0 and its derivative XDOT_0, so
     that the first row of the output X is X_0 and the first row of the
     output XDOT is XDOT_0.

     The first argument, FCN, is a string, inline, or function handle
     that names the function f to call to compute the vector of
     residuals for the set of equations.  It must have the form

          RES = f (X, XDOT, T)

     in which X, XDOT, and RES are vectors, and T is a scalar.

     If FCN is a two-element string array or a two-element cell array of
     strings, inline functions, or function handles, the first element
     names the function f described above, and the second element names
     a function to compute the modified Jacobian

                df       df
          jac = -- + c ------
                dx     d xdot

     The modified Jacobian function must have the form


          JAC = j (X, XDOT, T, C)


     The second and third arguments to ‘dassl’ specify the initial
     condition of the states and their derivatives, and the fourth
     argument specifies a vector of output times at which the solution
     is desired, including the time corresponding to the initial
     condition.

     The set of initial states and derivatives are not strictly required
     to be consistent.  In practice, however, DASSL is not very good at
     determining a consistent set for you, so it is best if you ensure
     that the initial values result in the function evaluating to zero.

     The fifth argument is optional, and may be used to specify a set of
     times that the DAE solver should not integrate past.  It is useful
     for avoiding difficulties with singularities and points where there
     is a discontinuity in the derivative.

     After a successful computation, the value of ISTATE will be greater
     than zero (consistent with the Fortran version of DASSL).

     If the computation is not successful, the value of ISTATE will be
     less than zero and MSG will contain additional information.

     You can use the function ‘dassl_options’ to set optional parameters
     for ‘dassl’.

     See also: *note daspk: XREFdaspk, *note dasrt: XREFdasrt, *note
     lsode: XREFlsode.

 -- : dassl_options ()
 -- : val = dassl_options (OPT)
 -- : dassl_options (OPT, VAL)
     Query or set options for the function ‘dassl’.

     When called with no arguments, the names of all available options
     and their current values are displayed.

     Given one argument, return the value of the option OPT.

     When called with two arguments, ‘dassl_options’ sets the option OPT
     to value VAL.

     Options include

     "absolute tolerance"
          Absolute tolerance.  May be either vector or scalar.  If a
          vector, it must match the dimension of the state vector, and
          the relative tolerance must also be a vector of the same
          length.

     "relative tolerance"
          Relative tolerance.  May be either vector or scalar.  If a
          vector, it must match the dimension of the state vector, and
          the absolute tolerance must also be a vector of the same
          length.

          The local error test applied at each integration step is

                 abs (local error in x(i))
                      <= rtol(i) * abs (Y(i)) + atol(i)

     "compute consistent initial condition"
          If nonzero, ‘dassl’ will attempt to compute a consistent set
          of initial conditions.  This is generally not reliable, so it
          is best to provide a consistent set and leave this option set
          to zero.

     "enforce nonnegativity constraints"
          If you know that the solutions to your equations will always
          be non-negative, it may help to set this parameter to a
          nonzero value.  However, it is probably best to try leaving
          this option set to zero first, and only setting it to a
          nonzero value if that doesn’t work very well.

     "initial step size"
          Differential-algebraic problems may occasionally suffer from
          severe scaling difficulties on the first step.  If you know a
          great deal about the scaling of your problem, you can help to
          alleviate this problem by specifying an initial stepsize.

     "maximum order"
          Restrict the maximum order of the solution method.  This
          option must be between 1 and 5, inclusive.

     "maximum step size"
          Setting the maximum stepsize will avoid passing over very
          large regions (default is not specified).

     "step limit"
          Maximum number of integration steps to attempt on a single
          call to the underlying Fortran code.

 -- : [X, XDOT, T_OUT, ISTAT, MSG] = dasrt (FCN, G, X_0, XDOT_0, T)
 -- : ... = dasrt (FCN, G, X_0, XDOT_0, T, T_CRIT)
 -- : ... = dasrt (FCN, X_0, XDOT_0, T)
 -- : ... = dasrt (FCN, X_0, XDOT_0, T, T_CRIT)
     Solve the set of differential-algebraic equations

          0 = f (x, xdot, t)

     with

          x(t_0) = x_0, xdot(t_0) = xdot_0

     with functional stopping criteria (root solving).

     The solution is returned in the matrices X and XDOT, with each row
     in the result matrices corresponding to one of the elements in the
     vector T_OUT.  The first element of T should be t_0 and correspond
     to the initial state of the system X_0 and its derivative XDOT_0,
     so that the first row of the output X is X_0 and the first row of
     the output XDOT is XDOT_0.

     The vector T provides an upper limit on the length of the
     integration.  If the stopping condition is met, the vector T_OUT
     will be shorter than T, and the final element of T_OUT will be the
     point at which the stopping condition was met, and may not
     correspond to any element of the vector T.

     The first argument, FCN, is a string, inline, or function handle
     that names the function f to call to compute the vector of
     residuals for the set of equations.  It must have the form

          RES = f (X, XDOT, T)

     in which X, XDOT, and RES are vectors, and T is a scalar.

     If FCN is a two-element string array or a two-element cell array of
     strings, inline functions, or function handles, the first element
     names the function f described above, and the second element names
     a function to compute the modified Jacobian

                df       df
          jac = -- + c ------
                dx     d xdot

     The modified Jacobian function must have the form


          JAC = j (X, XDOT, T, C)


     The optional second argument names a function that defines the
     constraint functions whose roots are desired during the
     integration.  This function must have the form

          G_OUT = g (X, T)

     and return a vector of the constraint function values.  If the
     value of any of the constraint functions changes sign, DASRT will
     attempt to stop the integration at the point of the sign change.

     If the name of the constraint function is omitted, ‘dasrt’ solves
     the same problem as ‘daspk’ or ‘dassl’.

     Note that because of numerical errors in the constraint functions
     due to round-off and integration error, DASRT may return false
     roots, or return the same root at two or more nearly equal values
     of T.  If such false roots are suspected, the user should consider
     smaller error tolerances or higher precision in the evaluation of
     the constraint functions.

     If a root of some constraint function defines the end of the
     problem, the input to DASRT should nevertheless allow integration
     to a point slightly past that root, so that DASRT can locate the
     root by interpolation.

     The third and fourth arguments to ‘dasrt’ specify the initial
     condition of the states and their derivatives, and the fourth
     argument specifies a vector of output times at which the solution
     is desired, including the time corresponding to the initial
     condition.

     The set of initial states and derivatives are not strictly required
     to be consistent.  In practice, however, DASSL is not very good at
     determining a consistent set for you, so it is best if you ensure
     that the initial values result in the function evaluating to zero.

     The sixth argument is optional, and may be used to specify a set of
     times that the DAE solver should not integrate past.  It is useful
     for avoiding difficulties with singularities and points where there
     is a discontinuity in the derivative.

     After a successful computation, the value of ISTATE will be greater
     than zero (consistent with the Fortran version of DASSL).

     If the computation is not successful, the value of ISTATE will be
     less than zero and MSG will contain additional information.

     You can use the function ‘dasrt_options’ to set optional parameters
     for ‘dasrt’.

     See also: *note dasrt_options: XREFdasrt_options, *note daspk:
     XREFdaspk, *note dasrt: XREFdasrt, *note lsode: XREFlsode.

 -- : dasrt_options ()
 -- : val = dasrt_options (OPT)
 -- : dasrt_options (OPT, VAL)
     Query or set options for the function ‘dasrt’.

     When called with no arguments, the names of all available options
     and their current values are displayed.

     Given one argument, return the value of the option OPT.

     When called with two arguments, ‘dasrt_options’ sets the option OPT
     to value VAL.

     Options include

     "absolute tolerance"
          Absolute tolerance.  May be either vector or scalar.  If a
          vector, it must match the dimension of the state vector, and
          the relative tolerance must also be a vector of the same
          length.

     "relative tolerance"
          Relative tolerance.  May be either vector or scalar.  If a
          vector, it must match the dimension of the state vector, and
          the absolute tolerance must also be a vector of the same
          length.

          The local error test applied at each integration step is

                 abs (local error in x(i)) <= ...
                     rtol(i) * abs (Y(i)) + atol(i)

     "initial step size"
          Differential-algebraic problems may occasionally suffer from
          severe scaling difficulties on the first step.  If you know a
          great deal about the scaling of your problem, you can help to
          alleviate this problem by specifying an initial stepsize.

     "maximum order"
          Restrict the maximum order of the solution method.  This
          option must be between 1 and 5, inclusive.

     "maximum step size"
          Setting the maximum stepsize will avoid passing over very
          large regions.

     "step limit"
          Maximum number of integration steps to attempt on a single
          call to the underlying Fortran code.

   See K. E. Brenan, et al., ‘Numerical Solution of Initial-Value
Problems in Differential-Algebraic Equations’, North-Holland (1989) for
more information about the implementation of DASSL.


File: octave.info,  Node: Optimization,  Next: Statistics,  Prev: Differential Equations,  Up: Top

25 Optimization
***************

Octave comes with support for solving various kinds of optimization
problems.  Specifically Octave can solve problems in Linear Programming,
Quadratic Programming, Nonlinear Programming, and Linear Least Squares
Minimization.

* Menu:

* Linear Programming::
* Quadratic Programming::
* Nonlinear Programming::
* Linear Least Squares::


File: octave.info,  Node: Linear Programming,  Next: Quadratic Programming,  Up: Optimization

25.1 Linear Programming
=======================

Octave can solve Linear Programming problems using the ‘glpk’ function.
That is, Octave can solve

     min C'*x

   subject to the linear constraints A*x = b where x ≥ 0.

The ‘glpk’ function also supports variations of this problem.

 -- : [XOPT, FMIN, ERRNUM, EXTRA] = glpk (C, A, B, LB, UB, CTYPE,
          VARTYPE, SENSE, PARAM)
     Solve a linear program using the GNU GLPK library.

     Given three arguments, ‘glpk’ solves the following standard LP:

          min C'*x

     subject to

          A*x  = b
            x >= 0

     but may also solve problems of the form

          [ min | max ] C'*x

     subject to

          A*x [ "=" | "<=" | ">=" ] b
            x >= LB
            x <= UB

     Input arguments:

     C
          A column array containing the objective function coefficients.

     A
          A matrix containing the constraints coefficients.

     B
          A column array containing the right-hand side value for each
          constraint in the constraint matrix.

     LB
          An array containing the lower bound on each of the variables.
          If LB is not supplied, the default lower bound for the
          variables is zero.

     UB
          An array containing the upper bound on each of the variables.
          If UB is not supplied, the default upper bound is assumed to
          be infinite.

     CTYPE
          An array of characters containing the sense of each constraint
          in the constraint matrix.  Each element of the array may be
          one of the following values

          "F"
               A free (unbounded) constraint (the constraint is
               ignored).

          "U"
               An inequality constraint with an upper bound (‘A(i,:)*x
               <= b(i)’).

          "S"
               An equality constraint (‘A(i,:)*x = b(i)’).

          "L"
               An inequality with a lower bound (‘A(i,:)*x >= b(i)’).

          "D"
               An inequality constraint with both upper and lower bounds
               (‘A(i,:)*x >= -b(i)’) _and_ (‘A(i,:)*x <= b(i)’).

     VARTYPE
          A column array containing the types of the variables.

          "C"
               A continuous variable.

          "I"
               An integer variable.

     SENSE
          If SENSE is 1, the problem is a minimization.  If SENSE is -1,
          the problem is a maximization.  The default value is 1.

     PARAM
          A structure containing the following parameters used to define
          the behavior of solver.  Missing elements in the structure
          take on default values, so you only need to set the elements
          that you wish to change from the default.

          Integer parameters:

          ‘msglev (default: 1)’
               Level of messages output by solver routines:

               0 (‘GLP_MSG_OFF’)
                    No output.

               1 (‘GLP_MSG_ERR’)
                    Error and warning messages only.

               2 (‘GLP_MSG_ON’)
                    Normal output.

               3 (‘GLP_MSG_ALL’)
                    Full output (includes informational messages).

          ‘scale (default: 16)’
               Scaling option.  The values can be combined with the
               bitwise OR operator and may be the following:

               1 (‘GLP_SF_GM’)
                    Geometric mean scaling.

               16 (‘GLP_SF_EQ’)
                    Equilibration scaling.

               32 (‘GLP_SF_2N’)
                    Round scale factors to power of two.

               64 (‘GLP_SF_SKIP’)
                    Skip if problem is well scaled.

               Alternatively, a value of 128 (‘GLP_SF_AUTO’) may be also
               specified, in which case the routine chooses the scaling
               options automatically.

          ‘dual (default: 1)’
               Simplex method option:

               1 (‘GLP_PRIMAL’)
                    Use two-phase primal simplex.

               2 (‘GLP_DUALP’)
                    Use two-phase dual simplex, and if it fails, switch
                    to the primal simplex.

               3 (‘GLP_DUAL’)
                    Use two-phase dual simplex.

          ‘price (default: 34)’
               Pricing option (for both primal and dual simplex):

               17 (‘GLP_PT_STD’)
                    Textbook pricing.

               34 (‘GLP_PT_PSE’)
                    Steepest edge pricing.

          ‘itlim (default: intmax)’
               Simplex iterations limit.  It is decreased by one each
               time when one simplex iteration has been performed, and
               reaching zero value signals the solver to stop the
               search.

          ‘outfrq (default: 200)’
               Output frequency, in iterations.  This parameter
               specifies how frequently the solver sends information
               about the solution to the standard output.

          ‘branch (default: 4)’
               Branching technique option (for MIP only):

               1 (‘GLP_BR_FFV’)
                    First fractional variable.

               2 (‘GLP_BR_LFV’)
                    Last fractional variable.

               3 (‘GLP_BR_MFV’)
                    Most fractional variable.

               4 (‘GLP_BR_DTH’)
                    Heuristic by Driebeck and Tomlin.

               5 (‘GLP_BR_PCH’)
                    Hybrid pseudocost heuristic.

          ‘btrack (default: 4)’
               Backtracking technique option (for MIP only):

               1 (‘GLP_BT_DFS’)
                    Depth first search.

               2 (‘GLP_BT_BFS’)
                    Breadth first search.

               3 (‘GLP_BT_BLB’)
                    Best local bound.

               4 (‘GLP_BT_BPH’)
                    Best projection heuristic.

          ‘presol (default: 1)’
               If this flag is set, the simplex solver uses the built-in
               LP presolver.  Otherwise the LP presolver is not used.

          ‘lpsolver (default: 1)’
               Select which solver to use.  If the problem is a MIP
               problem this flag will be ignored.

               1
                    Revised simplex method.

               2
                    Interior point method.

          ‘rtest (default: 34)’
               Ratio test technique:

               17 (‘GLP_RT_STD’)
                    Standard ("textbook").

               34 (‘GLP_RT_HAR’)
                    Harris’ two-pass ratio test.

          ‘tmlim (default: intmax)’
               Searching time limit, in milliseconds.

          ‘outdly (default: 0)’
               Output delay, in seconds.  This parameter specifies how
               long the solver should delay sending information about
               the solution to the standard output.

          ‘save (default: 0)’
               If this parameter is nonzero, save a copy of the problem
               in CPLEX LP format to the file ‘"outpb.lp"’.  There is
               currently no way to change the name of the output file.

          Real parameters:

          ‘tolbnd (default: 1e-7)’
               Relative tolerance used to check if the current basic
               solution is primal feasible.  It is not recommended that
               you change this parameter unless you have a detailed
               understanding of its purpose.

          ‘toldj (default: 1e-7)’
               Absolute tolerance used to check if the current basic
               solution is dual feasible.  It is not recommended that
               you change this parameter unless you have a detailed
               understanding of its purpose.

          ‘tolpiv (default: 1e-10)’
               Relative tolerance used to choose eligible pivotal
               elements of the simplex table.  It is not recommended
               that you change this parameter unless you have a detailed
               understanding of its purpose.

          ‘objll (default: -DBL_MAX)’
               Lower limit of the objective function.  If the objective
               function reaches this limit and continues decreasing, the
               solver stops the search.  This parameter is used in the
               dual simplex method only.

          ‘objul (default: +DBL_MAX)’
               Upper limit of the objective function.  If the objective
               function reaches this limit and continues increasing, the
               solver stops the search.  This parameter is used in the
               dual simplex only.

          ‘tolint (default: 1e-5)’
               Relative tolerance used to check if the current basic
               solution is integer feasible.  It is not recommended that
               you change this parameter unless you have a detailed
               understanding of its purpose.

          ‘tolobj (default: 1e-7)’
               Relative tolerance used to check if the value of the
               objective function is not better than in the best known
               integer feasible solution.  It is not recommended that
               you change this parameter unless you have a detailed
               understanding of its purpose.

     Output values:

     XOPT
          The optimizer (the value of the decision variables at the
          optimum).

     FOPT
          The optimum value of the objective function.

     ERRNUM
          Error code.

          0
               No error.

          1 (‘GLP_EBADB’)
               Invalid basis.

          2 (‘GLP_ESING’)
               Singular matrix.

          3 (‘GLP_ECOND’)
               Ill-conditioned matrix.

          4 (‘GLP_EBOUND’)
               Invalid bounds.

          5 (‘GLP_EFAIL’)
               Solver failed.

          6 (‘GLP_EOBJLL’)
               Objective function lower limit reached.

          7 (‘GLP_EOBJUL’)
               Objective function upper limit reached.

          8 (‘GLP_EITLIM’)
               Iterations limit exhausted.

          9 (‘GLP_ETMLIM’)
               Time limit exhausted.

          10 (‘GLP_ENOPFS’)
               No primal feasible solution.

          11 (‘GLP_ENODFS’)
               No dual feasible solution.

          12 (‘GLP_EROOT’)
               Root LP optimum not provided.

          13 (‘GLP_ESTOP’)
               Search terminated by application.

          14 (‘GLP_EMIPGAP’)
               Relative MIP gap tolerance reached.

          15 (‘GLP_ENOFEAS’)
               No primal/dual feasible solution.

          16 (‘GLP_ENOCVG’)
               No convergence.

          17 (‘GLP_EINSTAB’)
               Numerical instability.

          18 (‘GLP_EDATA’)
               Invalid data.

          19 (‘GLP_ERANGE’)
               Result out of range.

     EXTRA
          A data structure containing the following fields:

          ‘lambda’
               Dual variables.

          ‘redcosts’
               Reduced Costs.

          ‘time’
               Time (in seconds) used for solving LP/MIP problem.

          ‘status’
               Status of the optimization.

               1 (‘GLP_UNDEF’)
                    Solution status is undefined.

               2 (‘GLP_FEAS’)
                    Solution is feasible.

               3 (‘GLP_INFEAS’)
                    Solution is infeasible.

               4 (‘GLP_NOFEAS’)
                    Problem has no feasible solution.

               5 (‘GLP_OPT’)
                    Solution is optimal.

               6 (‘GLP_UNBND’)
                    Problem has no unbounded solution.

     Example:

          c = [10, 6, 4]';
          A = [ 1, 1, 1;
               10, 4, 5;
                2, 2, 6];
          b = [100, 600, 300]';
          lb = [0, 0, 0]';
          ub = [];
          ctype = "UUU";
          vartype = "CCC";
          s = -1;

          param.msglev = 1;
          param.itlim = 100;

          [xmin, fmin, status, extra] = ...
             glpk (c, A, b, lb, ub, ctype, vartype, s, param);

