cmlmt#

Purpose#

Solve the nonlinear programming problem.

Format#

out = cmlmt(&modelProc, par[, ..., c1])#

Parameters:

&modelProc (function pointer) – Pointer to a procedure that computes the function to be minimized.
par (struct) – An instance of a PV structure. The par instance is passed to the user-provided procedure pointed to by &fct. par is constructed using the “pack” functions.
... (various) – Optional input arguments. They can be any set of structures, matrices, arrays, strings required to compute the function. Can include GAUSS data types or a DS structure for dataset manipulation. Specific usage depends on the requirements of the modelProc.

c1 (struct) –

An instance of a cmlmtControl structure. It is an optional argument. If none is provided, default values will be set. Members of this instance can be set to other values to control various aspects of the optimization process.

c1.A	MxK matrix, linear equality constraint coefficients: `c1.A * p = c1.B` where `p` is a vector of the parameters.
c1.B	Mx1 vector, linear equality constraint constants: `c1.A * p = c1.B` where `p` is a vector of the parameters.
c1.C	MxK matrix, linear inequality constraint coefficients: `c1.C * p >= c1.D` where `p` is a vector of the parameters.
c1.D	Mx1 vector, linear inequality constraint constants: `c1.C * p >= c1.D` where `p` is a vector of the parameters.
c1.eqProc	Scalar, pointer to a procedure that computes the nonlinear equality constraints. It has two input arguments: an instance of a `PV` parameter structure, and an instance of a `DS` data structure; and one output argument, a vector of computed equality constraints. Default = {.}, i.e., no equality procedure.
c1.IneqProc	Scalar, pointer to a procedure that computes the nonlinear inequality constraints. It has two input arguments: an instance of a `PV` parameter structure, and an instance of a `DS` data structure; and one output argument, a vector of computed inequality constraints. Default = {.}, i.e., no inequality procedure.
c1.eqJacobian	Scalar, pointer to a procedure that computes the Jacobian of the equality constraints. It has two input arguments: an instance of a `PV` parameter structure, and an instance of a `DS` data structure; and one output argument, a matrix of derivatives of the equality constraints with respect to the parameters. Default = {.}, i.e., no equality Jacobian procedure.
c1.ineqJacobian	Scalar, pointer to a procedure that computes the Jacobian of the inequality constraints. It has two input arguments: an instance of a `PV` parameter structure, and an instance of a `DS` data structure; and one output argument, a matrix of derivatives of the inequality constraints with respect to the parameters. Default = {.}, i.e., no inequality Jacobian procedure.
c1.Bounds	1x2 or Kx2 matrix, bounds on parameters. If 1x2, all parameters have the same bounds. Default = `{-1e256, 1e256}`.
c1.bayesAlpha	Exponent of the Dirichlet random variates used in the weighted bootstrap. Default = 1.4.
c1.priorProc	Pointer to a procedure for computing the prior. Assumes a uniform prior if not provided.
c1.numSamples	Number of re-samples in the weighted likelihood bootstrap.
c1.BayesFname	Filename for the simulated posterior parameters dataset. Defaults to a unique “BAYESxxxx” pattern.
c1.maxBootTime	Maximum time allowed for resampling.
c1.algorithm	Scalar, descent algorithm. 0 - Modified BFGS (Default), 1 = BFGS, 2 = DFP, 3 = Newton, 4 = BHHH.
c1.useThreads	Scalar, if nonzero, threading is turned on; otherwise, off. Default = off.
c1.Switch	4x1 or 4x2 vector, controls algorithm switching. If 4x1, the details follow specific conditions. If 4x2, CMLMT switches between the algorithms in column 1 and column 2. Default = {1 3, .0001 .0001, 10 10, .0001 .0001}.
c1.lineSearch	Scalar, sets the line search method. 0 = augmented Lagrangian penalty method (requires constraints), 1 = STEPBT (quadratic and cubic curve fit) (default), 2 = Brent’s method, 3 = BHHHStep, 4 = half, 5 = Strong Wolfe’s condition.
c1.trustRadius	Scalar, radius of the trust region. If missing, trust region not applied. Sets a maximum amount of the direction at each iteration. Default = .001.
c1.penalty	Scalar, augmentation constant for augmented Lagrangian penalty line search method.
c1.active	Kx1 vector, set the K-th element to zero to fix it to start value. Use the GAUSS function `pvGetIndex()` to determine where parameters in the `PV` structure are in the vector of parameters. Default = all parameters are active.
c1.numObs	Scalar, number of observations, required if the log-likelihood
c1.maxIters	Scalar, maximum number of iterations. Default = 10000.
c1.dirTol	Scalar, convergence tolerance. Iterations cease when all elements of the direction vector are less than this value.
c1.weights	Vector, weights for objective function returning a vector. Default = 1.
c1.CovParType	Scalar, if 2, QML covariance matrix; else if 0, no covariance matrix is computed; else ML covariance matrix is computed. Default = 1.
c1.alpha	Scalar, probability level for statistical tests. Default = .05.
c1.FeasibleTest	Scalar, if nonzero, parameters are tested for feasibility before computing function in line search. If function is defined outside inequality boundaries, then this test can be turned off. Default = 1.
c1.MaxTries	Scalar, maximum number of attempts in random search. Default = 100.
c1.randRadius	Scalar, if zero, no random search is attempted. If nonzero, it is the radius of the random search. Default = .001.
c1.gradMethod	Scalar, method for computing numerical gradient. 0 = central difference, 1 = forward difference (default), 2 = backward difference, 3 = complex derivatives.
c1.hessMethod	Scalar, method for computing numerical Hessian. 0 = central difference, 1 = forward difference (default), 2 = backward difference.
c1.gradStep	Scalar or Kx1, increment size for computing numerical gradient. If scalar, stepsize will be value times parameter estimates for the numerical gradient. If Kx1, the step size for the gradient will be the elements of the vector.
c1.hessStep	Scalar or Kx1, increment size for computing numerical Hessian. If scalar, stepsize will be value times parameter estimates for the numerical Hessian. If Kx1, the step size for the Hessian will be the elements of the vector.
c1.gradCheck	Scalar, if nonzero and if analytical gradients and/or Hessian have been provided, numerical gradients and/or Hessian are computed and compared against the analytical versions.
c1.state	Scalar, seed for random number generator.
c1.title	String, title of run.
c1.PrintIters	Scalar, if nonzero, prints iteration information. Default = 0.
c1.disableKey	Scalar, if nonzero, keyboard input disabled.

Returns:

out (struct) –

An instance of a cmlmtResults structure containing the optimization results, including parameter estimates and diagnostics.

out.par	Instance of a PV structure containing the parameter estimates will be placed in the member matrix out.par.
out.fct	Scalar, function evaluated at parameters in par.
out.returnDescription	String, description of return values.
out.covPar	KxK matrix, covariance matrix of parameters.
out.covParDescription	String, description of covPar.
out.numObs	Scalar, number of observations.
out.hessian	KxK matrix, Hessian evaluated at parameters in par.
out.xproduct	KxK matrix, cross-product of NxK matrix of first derivatives evaluated at parameters in par. Not available if log-likelihood function returns a scalar.
out.waldLimits	Kx2 matrix, Wald confidence limits.
out.inverseWaldLimits	Kx2 matrix, confidence limits by inversion of Wald statistics. Available only if `cmlmtInverseWaldLimits()` has been called.
out.profileLimits	Kx2 matrix, profile likelihood confidence limits, i.e., by inversion of likelihood ratio statistics. Only available if `cmlmtProfileLimits()` has been called.
out.bayesLimits	Kx2 matrix, weighted likelihood Bayesian confidence limits. Only available if `cmlmtBayes()` has been called.
out.bootLimits	Kx2 Matrix, bootstrap confidence limits. Available only if `cmlmtBoot()` has been called.
out.gradient	Kx1 vector, gradient evaluated at the parameters in par.
out.numIterations	Scalar, number of iterations.
out.elapsedTime	Scalar, elapsed time of iterations.
out.alpha	Scalar, probability level of confidence limits. Default = .05.
out.title	String, title of run.
out.lagr	An instance of a `cmlmtLagrange` structure containing the Lagrangeans for the constraints. For an instance named lagr, the members are: out.lagr.lineq - Mx1 vector, Lagrangeans of linear equality constraints, out.lagr.nlineq - Nx1 vector, Lagrangeans of nonlinear equality constraints out.lagr.linineq - Px1 vector, Lagrangeans of linear inequality constraints out.lagr.nlinineq - Qx1 vector, Lagrangeans of nonlinear inequality constraints out.lagr.bounds - Kx2 matrix, Lagrangeans of bounds out.Lagr.EqCov - (M+N)x(M+N) matrix, covariance matrix of equality constraints out.Lagr.IneqCov - (P+Q)x(P+Q) matrix, covariance matrix of inequality constraints Whenever a constraint is active, its associated Lagrangean will be nonzero. For any constraint that is inactive throughout the iterations as well as at convergence, the corresponding Lagrangean matrix will be set to a scalar missing value.
out.retcode	Return code: 0 - Normal convergence 1 - Forced exit 2 - Maximum number of iterations exceeded 3 - Function calculation failed 4 - Gradient calculation failed 5 - Hessian calculation failed 6 - Line search failed 7 - Functional evaluation failed 8 - Error with initial gradient 9 - Error with constraints 10 - Second update failed 11 - Maximum time exceeded 12 - Error with weights 13 - Quadratic program failed 14 - Equality constraint Jacobian failed 15 - Inequality constraint Jacobian failed 16 - Function evaluated as complex 20 - Hessian failed to invert 34 - Data set could not be opened

Examples#

The following is a complete example. It applys the Biochemical Oxygen Demand model to data taken from Douglas M. Bates and Donald G. Watts, Nonlinear Regression Analysis and Its Applications, page 270.

new;
library cmlmt;

// Likelihood function
proc lnlk(struct PV p, y, x, ind);
local dev,s2,m,r,b0,b;

    // Declare 'mm' to be a modelResults
    // struct local to this procedure, 'lnlk'
    struct modelResults mm;

    // Unpack parameters
    b0 = pvUnpack(p, 1);
    b = pvUnpack(p, 2);

    // Function computations
    r = exp(-b*x);
    m = 1 - r;
    dev = y - b0*m;
    s2 = dev'dev/rows(dev);

    // If the first element of the indicator
    // vector is non-zero, compute function value
    // and assign it to the 'function' member
    // of the modelResults struct
    if ind[1];
        mm.function = lnpdfmvn(dev,s2);
    endif;

    // If the first element of the indicator
    // vector is non-zero, compute gradient value
    // and assign it to the 'gradient' member
    // of the modelResults struct
    if ind[2];
        mm.gradient = (dev/s2) .*
        (m ~ b0*x.*r);
    endif;
    retp(mm);
endp;

// Enter data
y = { 8.3,
10.3,
19.0,
16.0,
15.6,
19.8 };

x = { 1,
2,
3,
4,
5,
7 };

// Pack starting values into
// PV structure
struct PV p0;
p0 = pvPacki(pvCreate,19.143,"b0",1);
p0 = pvPacki(p0,0.5311,"b",2);

// Declare instance of cmlmtControl structure
struct cmlmtControl ctl;
ctl = cmlmtControlCreate();

// Set parameter bounds to
//   10 <= b0 <= 35
//   0 <= b <= 2
ctl.bounds = { 10 35,
               0 2 };

//Declare 'out' to be a 'cmlmtResults' structure
//to hold the estimation results
struct cmlmtResults out;

//Perform the estimation
out = cmlmt(&lnlk, p0, y, x, ctl);

//Print the results
call cmlmtPrt(out);

Remarks#

There is one required user-provided procedure, the one computing the log-likelihood function and optionally the first and/or second derivatives, and four other optional procedures, one each for computing the equality constraints, the inequality constraints, the Jacobian of the equality constraints, and the Jacobian of the inequality constraints.
The main procedure, computing the log-likelihood and optionally the first and/or second derivatives, has an instance of type PV struct containing the parameters, a second argument that is an instance of type struct DS containing the data, and a third argument that is a vector of zeros and ones indicating which of the results, the function, first derivatives, or second derivatives, to be computed.
The remaining optional procedures take just two arguments, the instance of the PV structure containing the parameters and the instance of the DS structure containing the data.
The instance of the PV structure is set up using the PV pack procedures, pvPack(), pvPackm(), pvPacks(), and pvPacksm(). These procedures allow for setting up a parameter vector in a variety of ways.
The instance of the DS structure containing the data is set up in two distinct ways depending on whether cmlmt() is to read the data in from a GAUSS data set.
For example, the following procedure computes the log-likelihood and the first derivatives for a tobit model:

proc lpr(struct PV p, x, y, ind);
   local s2,b0,b,yh,u,res,g1,g2;

   struct modelResults mm;

   b0 = pvUnpack(p,"b0");
   b = pvUnpack(p,"b");
   s2 = pvUnpack(p,"variance");

   yh = b0 + x * b;
   res = y - yh;
   u = y[.,1] ./= 0;

   if ind[1];
       mm.function = u.*lnpdfmvn(res,s2) + (1-u).*(ln(cdfnc(yh/sqrt(s2))));
   endif;

   if ind[2];
       yh = yh/sqrt(s2);
       g1 = ((res~x.*res)/s2)~((res.*res/s2)-1)/(2*s2);
       g2 = ( -( ones(rows(x),1)~x )/sqrt(s2) )~(yh/(2*s2));
       g2 = (pdfn(yh)./cdfnc(yh)).*g2;
       mm.gradient = u.*g1 + (1-u).*g2;
   endif;
   retp(mm);

endp;

The procedures for nonlinear equality and inequality constraints take two input arguments, an instance of a PV parameters structure and an instance of a DS data structure. For example, to constrain the sum of squares of the regression coefficients to be greater than one, provide the following procedure:

proc ineqConst(struct PV par1, x, y);
     local b;
     b = pvUnpack(p,"b");
     retp(sumc(b^2) - 1);
endp;