Fit a linear model with an L1 penalty.
ridgeFit(y, X, lambda)¶
y (Nx1 vector) – The target, or dependent variable.
X (NxP matrix) – The model features, or independent variables.
lambda (Scalar, or Kx1 vector) – The L1 penalty parameter(s).
mdl (struct) –
An instance of a
ridgeModelstructure. An instance named mdl will have the following members:
(1 x nlambdas vector) The estimated value for the intercept for each provided lambda.
(P x nlambdas matrix) The estimated parameter values for each provided lambda.
(nlambdas x 1 vector) The mean squared error for each set of parameters, computed on the training set.
(nlambdas x 1 vector) The lambda values used in the estimation.
Example 1: Basic Estimation and Prediction¶
new; library gml; // Specify dataset with full path dataset = getGAUSSHome() $+ "pkgs/gml/examples/qsar_fish_toxicity.csv"; // Load dependent and independent variable y = loadd(dataset, "LC50"); X = loadd(dataset, ". -LC50"); // Split data into training sets y_test = y[1:636]; X_test = X[1:636,.]; y_train = y[637:rows(y)]; X_train = X[637:rows(X),.]; // Declare 'mdl' to be an instance of a // ridgeModel structure to hold the estimation results struct ridgeModel mdl; lambda = seqm(90, 0.8, 60); // Estimate the model with default settings mdl = ridgeFit(y_train, X_train, lambda);
After the above code, mdl.beta_hat will be a \(6 \times 60\) matrix, where each column contains the estimates for a different lambda value. The graph below shows the path of the parameter values as the value of lambda changes.
Continuing with our example, we can make test predictions like this:
// Make predictions on the test set y_hat = X_test * mdl.beta_hat + mdl.alpha_hat;
After the above code, y_hat will be a matrix with the same number of observations as y_test. However, it will have one column for each value of lambda used in the estimation. We can compute the mean-squared error (MSE) for each of our predictions with the following code:
// Compute MSE for each prediction mse_test = meanc((y_test - y_hat).^2);
Below is a plot of the change in MSE with the changes in lambda.