Select optimal penalty parameter by approximate leave-one-out cross-validation

Source: R/rags2ridges.R

Function that selects the optimal penalty parameter for the ridgeP call by usage of approximate leave-one-out cross-validation. Its output includes (a.o.) the precision matrix under the optimal value of the penalty parameter.

optPenalty.aLOOCV(
  Y,
  lambdaMin,
  lambdaMax,
  step,
  type = "Alt",
  cor = FALSE,
  target = default.target(covML(Y)),
  output = "light",
  graph = TRUE,
  verbose = TRUE
)

Arguments

Y

Data matrix. Variables assumed to be represented by columns.

lambdaMin

A numeric giving the minimum value for the penalty parameter.

lambdaMax

A numeric giving the maximum value for the penalty parameter.

step

An integer determining the number of steps in moving through the grid [lambdaMin, lambdaMax].

type

A character indicating the type of ridge estimator to be used. Must be one of: "Alt", "ArchI", "ArchII".

cor

A logical indicating if the evaluation of the approximate LOOCV score should be performed on the correlation scale.

target

A target matrix (in precision terms) for Type I ridge estimators.

output

A character indicating if the output is either heavy or light. Must be one of: "all", "light".

graph

A logical indicating if the grid search for the optimal penalty parameter should be visualized.

verbose

A logical indicating if information on progress should be printed on screen.

Value

An object of class list:

optLambda

A numeric giving the optimal value of the penalty parameter.

optPrec

A matrix representing the precision matrix of the chosen type (see ridgeS) under the optimal value of the penalty parameter.

lambdas

A numeric vector representing all values of the penalty parameter for which approximate cross-validation was performed; Only given when output = "all".

aLOOCVs

A numeric vector representing the approximate cross-validated negative log-likelihoods for each value of the penalty parameter given in lambdas; Only given when output = "all".

Details

The function calculates an approximate leave-one-out cross-validated (aLOOCV) negative log-likelihood score (using a regularized ridge estimator for the precision matrix) for each value of the penalty parameter contained in the search grid. The utilized aLOOCV score was proposed by Lian (2011) and Vujacic et al. (2014). The aLOOCV negative log-likeliho od score is computationally more efficient than its non-approximate counterpart (see optPenalty.LOOCV). For details on the aLOOCV negative log-likelihood score see Lian (2011) and Vujacic et al (2014). For scalar matrix targets (see default.target) the complete solution path of the alternative Type I and II ridge estimators (see ridgeP) depends on only 1 eigendecomposition and 1 matrix inversion, making the determination of the optimal penalty value particularly efficient (see van Wieringen and Peeters, 2015).

The value of the penalty parameter that achieves the lowest aLOOCV negative log-likelihood score is deemed optimal. The penalty parameter must be positive such that lambdaMin must be a positive scalar. The maximum allowable value of lambdaMax depends on the type of ridge estimator employed. For details on the type of ridge estimator one may use (one of: "Alt", "ArchI", "ArchII") see ridgeP. The ouput consists of an object of class list (see below). When output = "light" (default) only the optLambda and optPrec elements of the list are given.

Note

When cor = TRUE correlation matrices are used in the computation of the approximate (cross-validated) negative log-likelihood score, i.e., the sample covariance matrix is a matrix on the correlation scale. When performing evaluation on the correlation scale the data are assumed to be standardized. If cor = TRUE and one wishes to used the default target specification one may consider using target = default.target(covML(Y, cor = TRUE)). This gives a default target under the assumption of standardized data.

References

Lian, H. (2011). Shrinkage tuning parameter selection in precision matrices estimation. Journal of Statistical Planning and Inference, 141: 2839-2848.

van Wieringen, W.N. & Peeters, C.F.W. (2016). Ridge Estimation of Inverse Covariance Matrices from High-Dimensional Data, Computational Statistics & Data Analysis, vol. 103: 284-303. Also available as arXiv:1403.0904v3 [stat.ME].

Vujacic, I., Abbruzzo, A., and Wit, E.C. (2014). A computationally fast alternative to cross-validation in penalized Gaussian graphical models. arXiv: 1309.6216v2 [stat.ME].

Author

Carel F.W. Peeters <carel.peeters@wur.nl>, Wessel N. van Wieringen

Examples


## Obtain some (high-dimensional) data
p = 25
n = 10
set.seed(333)
X = matrix(rnorm(n*p), nrow = n, ncol = p)
colnames(X)[1:25] = letters[1:25]

## Obtain regularized precision under optimal penalty
OPT  <- optPenalty.aLOOCV(X, lambdaMin = .001, lambdaMax = 30, step = 400); OPT
#> Perform input checks... 
#> Calculating approximate LOOCV scores... 
#> lambda = 0.001 done
#> lambda = 0.00102617364194111 done
#> lambda = 0.00105303234341469 done
#> lambda = 0.00108059403492363 done
#> lambda = 0.00110887711627742 done
#> lambda = 0.00113790046887556 done
#> lambda = 0.00116768346831253 done
#> lambda = 0.0011982459973127 done
#> lambda = 0.00122960845900374 done
#> lambda = 0.00126179179053746 done
#> lambda = 0.00129481747706722 done
#> lambda = 0.00132870756609107 done
#> lambda = 0.00136348468217039 done
#> lambda = 0.00139917204203371 done
#> lambda = 0.00143579347007591 done
#> lambda = 0.00147337341426307 done
#> lambda = 0.00151193696245354 done
#> lambda = 0.00155150985914633 done
#> lambda = 0.00159211852266773 done
#> lambda = 0.00163379006280785 done
#> lambda = 0.00167655229891873 done
#> lambda = 0.00172043377848618 done
#> lambda = 0.00176546379618767 done
#> lambda = 0.00181167241344908 done
#> lambda = 0.00185909047851328 done
#> lambda = 0.00190774964703402 done
#> lambda = 0.00195768240320877 done
#> lambda = 0.00200892208146477 done
#> lambda = 0.00206150288871263 done
#> lambda = 0.00211545992718236 done
#> lambda = 0.0021708292178572 done
#> lambda = 0.0022276477245207 done
#> lambda = 0.00228595337843324 done
#> lambda = 0.00234578510365442 done
#> lambda = 0.00240718284302827 done
#> lambda = 0.00247018758484848 done
#> lambda = 0.00253484139022168 done
#> lambda = 0.00260118742114685 done
#> lambda = 0.00266926996932968 done
#> lambda = 0.00273913448575107 done
#> lambda = 0.00281082761100967 done
#> lambda = 0.00288439720645843 done
#> lambda = 0.00295989238615622 done
#> lambda = 0.00303736354965569 done
#> lambda = 0.00311686241564937 done
#> lambda = 0.00319844205649628 done
#> lambda = 0.00328215693365241 done
#> lambda = 0.00336806293402837 done
#> lambda = 0.00345621740729875 done
#> lambda = 0.00354667920418803 done
#> lambda = 0.00363950871575844 done
#> lambda = 0.00373476791372625 done
#> lambda = 0.00383252039183328 done
#> lambda = 0.00393283140830113 done
#> lambda = 0.00403576792939676 done
#> lambda = 0.00414139867413822 done
#> lambda = 0.00424979416017051 done
#> lambda = 0.00436102675084223 done
#> lambda = 0.00447517070351439 done
#> lambda = 0.00459230221913353 done
#> lambda = 0.00471249949310251 done
#> lambda = 0.00483584276748264 done
#> lambda = 0.00496241438456225 done
#> lambda = 0.0050922988418272 done
#> lambda = 0.00522558284837033 done
#> lambda = 0.00536235538277719 done
#> lambda = 0.00550270775252699 done
#> lambda = 0.00564673365494821 done
#> lambda = 0.00579452923976965 done
#> lambda = 0.00594619317330869 done
#> lambda = 0.00610182670433955 done
#> lambda = 0.00626153373168565 done
#> lambda = 0.00642542087358098 done
#> lambda = 0.00659359753884704 done
#> lambda = 0.00676617599993262 done
#> lambda = 0.0069432714678654 done
#> lambda = 0.00712500216916525 done
#> lambda = 0.00731148942477062 done
#> lambda = 0.0075028577310308 done
#> lambda = 0.0076992348428179 done
#> lambda = 0.00790075185881435 done
#> lambda = 0.00810754330903253 done
#> lambda = 0.0083197472446252 done
#> lambda = 0.00853750533004657 done
#> lambda = 0.00876096293762555 done
#> lambda = 0.00899026924461431 done
#> lambda = 0.00922557733277703 done
#> lambda = 0.00946704429058518 done
#> lambda = 0.0097148313180876 done
#> lambda = 0.00996910383452554 done
#> lambda = 0.0102300315887642 done
#> lambda = 0.0104977887726147 done
#> lambda = 0.0107725541371226 done
#> lambda = 0.0110545111118989 done
#> lambda = 0.0113438479275758 done
#> lambda = 0.0116407577414666 done
#> lambda = 0.0119454387665149 done
#> lambda = 0.0122580944036192 done
#> lambda = 0.0125789333774198 done
#> lambda = 0.0129081698756415 done
#> lambda = 0.0132460236920816 done
#> lambda = 0.0135927203733416 done
#> lambda = 0.0139484913693991 done
#> lambda = 0.0143135741881205 done
#> lambda = 0.0146882125538179 done
#> lambda = 0.0150726565699565 done
#> lambda = 0.0154671628861199 done
#> lambda = 0.015871994869346 done
#> lambda = 0.0162874227799474 done
#> lambda = 0.0167137239519333 done
#> lambda = 0.0171511829781538 done
#> lambda = 0.0176000919002905 done
#> lambda = 0.0180607504038193 done
#> lambda = 0.0185334660180767 done
#> lambda = 0.0190185543215616 done
#> lambda = 0.0195163391526117 done
#> lambda = 0.0200271528255935 done
#> lambda = 0.0205513363527505 done
#> lambda = 0.0210892396718587 done
#> lambda = 0.0216412218798403 done
#> lambda = 0.0222076514724914 done
#> lambda = 0.0227889065904854 done
#> lambda = 0.0233853752718142 done
#> lambda = 0.0239974557108372 done
#> lambda = 0.0246255565241103 done
#> lambda = 0.025270097023173 done
#> lambda = 0.0259315074944747 done
#> lambda = 0.0266102294866283 done
#> lambda = 0.0273067161051821 done
#> lambda = 0.0280214323151068 done
#> lambda = 0.0287548552511995 done
#> lambda = 0.0295074745366129 done
#> lambda = 0.0302797926097206 done
#> lambda = 0.0310723250595386 done
#> lambda = 0.0318856009699248 done
#> lambda = 0.0327201632727888 done
#> lambda = 0.0335765691105455 done
#> lambda = 0.0344553902080559 done
#> lambda = 0.0353572132543028 done
#> lambda = 0.0362826402940565 done
#> lambda = 0.0372322891297913 done
#> lambda = 0.0382067937341224 done
#> lambda = 0.0392068046730372 done
#> lambda = 0.0402329895402044 done
#> lambda = 0.0412860334026502 done
#> lambda = 0.0423666392581 done
#> lambda = 0.0434755285042897 done
#> lambda = 0.0446134414205616 done
#> lambda = 0.0457811376620642 done
#> lambda = 0.0469793967668878 done
#> lambda = 0.0482090186764737 done
#> lambda = 0.0494708242696441 done
#> lambda = 0.0507656559106095 done
#> lambda = 0.0520943780113195 done
#> lambda = 0.0534578776085326 done
#> lambda = 0.0548570649559902 done
#> lambda = 0.0562928741320886 done
#> lambda = 0.057766263663458 done
#> lambda = 0.0592782171648611 done
#> lambda = 0.0608297439958417 done
#> lambda = 0.0624218799345584 done
#> lambda = 0.0640556878692565 done
#> lambda = 0.0657322585078381 done
#> lambda = 0.0674527111060029 done
#> lambda = 0.0692181942144487 done
#> lambda = 0.0710298864456279 done
#> lambda = 0.0728889972605737 done
#> lambda = 0.0747967677763186 done
#> lambda = 0.0767544715944484 done
#> lambda = 0.0787634156513408 done
#> lambda = 0.080824941090658 done
#> lambda = 0.0829404241586763 done
#> lambda = 0.0851112771230494 done
#> lambda = 0.0873389492156189 done
#> lambda = 0.0896249275999015 done
#> lambda = 0.0919707383638993 done
#> lambda = 0.0943779475388957 done
#> lambda = 0.0968481621449158 done
#> lambda = 0.0993830312635516 done
#> lambda = 0.101984247138866 done
#> lambda = 0.104653546307113 done
#> lambda = 0.107392710756023 done
#> lambda = 0.110203569114436 done
#> lambda = 0.11308799787307 done
#> lambda = 0.116047922637237 done
#> lambda = 0.119085319412354 done
#> lambda = 0.122202215923096 done
#> lambda = 0.125400692967077 done
#> lambda = 0.128682885803965 done
#> lambda = 0.132050985580947 done
#> lambda = 0.135507240795513 done
#> lambda = 0.139053958796523 done
#> lambda = 0.142693507324557 done
#> lambda = 0.146428316092592 done
#> lambda = 0.150260878408039 done
#> lambda = 0.154193752837248 done
#> lambda = 0.158229564913567 done
#> lambda = 0.162371008890112 done
#> lambda = 0.166620849538419 done
#> lambda = 0.170981923994162 done
#> lambda = 0.175457143651187 done
#> lambda = 0.180049496105124 done
#> lambda = 0.184762047147857 done
#> lambda = 0.189597942814212 done
#> lambda = 0.194560411482202 done
#> lambda = 0.199652766028253 done
#> lambda = 0.204878406038829 done
#> lambda = 0.210240820079955 done
#> lambda = 0.215743588026133 done
#> lambda = 0.22139038345022 done
#> lambda = 0.227184976075852 done
#> lambda = 0.233131234294061 done
#> lambda = 0.239233127745763 done
#> lambda = 0.245494729971833 done
#> lambda = 0.251920221132546 done
#> lambda = 0.258513890798195 done
#> lambda = 0.265280140812751 done
#> lambda = 0.272223488232471 done
#> lambda = 0.279348568341428 done
#> lambda = 0.286660137745959 done
#> lambda = 0.294163077550112 done
#> lambda = 0.301862396614204 done
#> lambda = 0.30976323489867 done
#> lambda = 0.317870866895428 done
#> lambda = 0.32619070514906 done
#> lambda = 0.33472830387015 done
#> lambda = 0.343489362643203 done
#> lambda = 0.352479730231607 done
#> lambda = 0.361705408482189 done
#> lambda = 0.371172556331965 done
#> lambda = 0.380887493919766 done
#> lambda = 0.390856706805469 done
#> lambda = 0.401086850299677 done
#> lambda = 0.411584753906709 done
#> lambda = 0.422357425883884 done
#> lambda = 0.433412057920139 done
#> lambda = 0.444756029937101 done
#> lambda = 0.456396915015825 done
#> lambda = 0.468342484452477 done
#> lambda = 0.480600712946347 done
#> lambda = 0.493179783923648 done
#> lambda = 0.50608809500066 done
#> lambda = 0.519334263589867 done
#> lambda = 0.532927132652819 done
#> lambda = 0.546875776603577 done
#> lambda = 0.561189507366667 done
#> lambda = 0.575877880593591 done
#> lambda = 0.590950702042054 done
#> lambda = 0.606418034122151 done
#> lambda = 0.622290202613898 done
#> lambda = 0.638577803560576 done
#> lambda = 0.655291710342512 done
#> lambda = 0.672443080935995 done
#> lambda = 0.690043365362193 done
#> lambda = 0.708104313331023 done
#> lambda = 0.726637982085105 done
#> lambda = 0.745656744449013 done
#> lambda = 0.765173297089197 done
#> lambda = 0.785200668990109 done
#> lambda = 0.805752230152178 done
#> lambda = 0.826841700517433 done
#> lambda = 0.848483159128757 done
#> lambda = 0.870691053528856 done
#> lambda = 0.89348020940525 done
#> lambda = 0.916865840487693 done
#> lambda = 0.940863558704654 done
#> lambda = 0.96548938460563 done
#> lambda = 0.990759758056242 done
#> lambda = 1.01669154921327 done
#> lambda = 1.04330206978693 done
#> lambda = 1.07060908459796 done
#> lambda = 1.09863082343712 done
#> lambda = 1.12738599323524 done
#> lambda = 1.1568937905516 done
#> lambda = 1.18717391438939 done
#> lambda = 1.21824657934645 done
#> lambda = 1.25013252911025 done
#> lambda = 1.28285305030612 done
#> lambda = 1.31642998670789 done
#> lambda = 1.35088575382053 done
#> lambda = 1.38624335384437 done
#> lambda = 1.42252639103114 done
#> lambda = 1.45975908744177 done
#> lambda = 1.49796629911676 done
#> lambda = 1.53717353266969 done
#> lambda = 1.57740696231514 done
#> lambda = 1.6186934473422 done
#> lambda = 1.66106055004536 done
#> lambda = 1.70453655412475 done
#> lambda = 1.74915048356795 done
#> lambda = 1.79493212202598 done
#> lambda = 1.84191203269649 done
#> lambda = 1.89012157872731 done
#> lambda = 1.93959294415409 done
#> lambda = 1.99035915538588 done
#> lambda = 2.04245410325317 done
#> lambda = 2.09591256563287 done
#> lambda = 2.15077023066562 done
#> lambda = 2.20706372058067 done
#> lambda = 2.26483061614436 done
#> lambda = 2.32410948174859 done
#> lambda = 2.38493989115582 done
#> lambda = 2.44736245391801 done
#> lambda = 2.51141884248698 done
#> lambda = 2.5771518200344 done
#> lambda = 2.64460526899986 done
#> lambda = 2.71382422038624 done
#> lambda = 2.78485488382175 done
#> lambda = 2.85774467840886 done
#> lambda = 2.93254226438065 done
#> lambda = 3.00929757558572 done
#> lambda = 3.08806185282336 done
#> lambda = 3.16888767805117 done
#> lambda = 3.25182900948808 done
#> lambda = 3.33694121763614 done
#> lambda = 3.42428112224509 done
#> lambda = 3.51390703024444 done
#> lambda = 3.60587877466841 done
#> lambda = 3.70025775459964 done
#> lambda = 3.79710697615835 done
#> lambda = 3.89649109456442 done
#> lambda = 3.99847645730028 done
#> lambda = 4.10313114840362 done
#> lambda = 4.21052503391936 done
#> lambda = 4.32072980854125 done
#> lambda = 4.4338190434743 done
#> lambda = 4.54986823554988 done
#> lambda = 4.6689548576264 done
#> lambda = 4.79115841030913 done
#> lambda = 4.91656047502371 done
#> lambda = 5.0452447684788 done
#> lambda = 5.17729719855424 done
#> lambda = 5.31280592165191 done
#> lambda = 5.45186140154784 done
#> lambda = 5.59455646978453 done
#> lambda = 5.740986387644 done
#> lambda = 5.89124890974299 done
#> lambda = 6.04544434929257 done
#> lambda = 6.20367564506587 done
#> lambda = 6.36604843011863 done
#> lambda = 6.53267110230833 done
#> lambda = 6.70365489665919 done
#> lambda = 6.87911395962114 done
#> lambda = 7.05916542527236 done
#> lambda = 7.24392949351651 done
#> lambda = 7.43352951032648 done
#> lambda = 7.62809205008845 done
#> lambda = 7.8277470001013 done
#> lambda = 8.03262764728757 done
#> lambda = 8.24287076717395 done
#> lambda = 8.45861671520081 done
#> lambda = 8.68000952042159 done
#> lambda = 8.90719698165455 done
#> lambda = 9.14033076615132 done
#> lambda = 9.3795665108479 done
#> lambda = 9.62506392626567 done
#> lambda = 9.87698690313207 done
#> lambda = 10.1355036217917 done
#> lambda = 10.4007866644813 done
#> lambda = 10.6730131305433 done
#> lambda = 10.952364754655 done
#> lambda = 11.2390280281518 done
#> lambda = 11.5331943235267 done
#> lambda = 11.835060022188 done
#> lambda = 12.1448266455603 done
#> lambda = 12.4627009896181 done
#> lambda = 12.7888952629395 done
#> lambda = 13.123627228374 done
#> lambda = 13.4671203484181 done
#> lambda = 13.8196039343955 done
#> lambda = 14.1813132995423 done
#> lambda = 14.5524899160993 done
#> lambda = 14.9333815765149 done
#> lambda = 15.3242425588686 done
#> lambda = 15.7253337966232 done
#> lambda = 16.1369230528204 done
#> lambda = 16.5592850988362 done
#> lambda = 16.992701897814 done
#> lambda = 17.4374627928994 done
#> lambda = 17.8938647004022 done
#> lambda = 18.3622123080132 done
#> lambda = 18.8428182782098 done
#> lambda = 19.3360034569852 done
#> lambda = 19.8420970880404 done
#> lambda = 20.3614370325835 done
#> lambda = 20.8943699948809 done
#> lambda = 21.441251753712 done
#> lambda = 22.0024473998829 done
#> lambda = 22.5783315799556 done
#> lambda = 23.169288746357 done
#> lambda = 23.7757134140344 done
#> lambda = 24.3980104238278 done
#> lambda = 25.0365952127366 done
#> lambda = 25.6918940912594 done
#> lambda = 26.3643445279929 done
#> lambda = 27.0543954416807 done
#> lambda = 27.7625075009045 done
#> lambda = 28.4891534316206 done
#> lambda = 29.2348183327453 done
#> lambda = 30 done

#> $optLambda
#> [1] 0.1286829
#> 
#> $optPrec
#> A 25 x 25 ridge precision matrix estimate with lambda = 0.128683
#>             a           b           c           d           e           f …
#> a  1.83782515  0.08025733 -0.43284175  0.15507515  0.11307948 -0.22056357 …
#> b  0.08025733  2.36622537 -0.26243658 -0.28978846 -0.40798541 -0.11094129 …
#> c -0.43284175 -0.26243658  2.26556108 -0.37953290  0.15811288  0.08891277 …
#> d  0.15507515 -0.28978846 -0.37953290  2.49757467  0.02877713 -0.07618537 …
#> e  0.11307948 -0.40798541  0.15811288  0.02877713  2.24929629 -0.26810898 …
#> f -0.22056357 -0.11094129  0.08891277 -0.07618537 -0.26810898  2.57787744 …
#> … 19 more rows and 19 more columns
#> 
OPT$optLambda  # Optimal penalty
#> [1] 0.1286829
OPT$optPrec    # Regularized precision under optimal penalty
#> A 25 x 25 ridge precision matrix estimate with lambda = 0.128683
#>             a           b           c           d           e           f …
#> a  1.83782515  0.08025733 -0.43284175  0.15507515  0.11307948 -0.22056357 …
#> b  0.08025733  2.36622537 -0.26243658 -0.28978846 -0.40798541 -0.11094129 …
#> c -0.43284175 -0.26243658  2.26556108 -0.37953290  0.15811288  0.08891277 …
#> d  0.15507515 -0.28978846 -0.37953290  2.49757467  0.02877713 -0.07618537 …
#> e  0.11307948 -0.40798541  0.15811288  0.02877713  2.24929629 -0.26810898 …
#> f -0.22056357 -0.11094129  0.08891277 -0.07618537 -0.26810898  2.57787744 …
#> … 19 more rows and 19 more columns

## Another example with standardized data
X <- scale(X, center = TRUE, scale = TRUE)
OPT  <- optPenalty.aLOOCV(X, lambdaMin = .001, lambdaMax = 30,
                          step = 400, cor = TRUE,
                          target = default.target(covML(X, cor = TRUE))); OPT
#> Perform input checks... 
#> Calculating approximate LOOCV scores... 
#> lambda = 0.001 done
#> lambda = 0.00102617364194111 done
#> lambda = 0.00105303234341469 done
#> lambda = 0.00108059403492363 done
#> lambda = 0.00110887711627742 done
#> lambda = 0.00113790046887556 done
#> lambda = 0.00116768346831253 done
#> lambda = 0.0011982459973127 done
#> lambda = 0.00122960845900374 done
#> lambda = 0.00126179179053746 done
#> lambda = 0.00129481747706722 done
#> lambda = 0.00132870756609107 done
#> lambda = 0.00136348468217039 done
#> lambda = 0.00139917204203371 done
#> lambda = 0.00143579347007591 done
#> lambda = 0.00147337341426307 done
#> lambda = 0.00151193696245354 done
#> lambda = 0.00155150985914633 done
#> lambda = 0.00159211852266773 done
#> lambda = 0.00163379006280785 done
#> lambda = 0.00167655229891873 done
#> lambda = 0.00172043377848618 done
#> lambda = 0.00176546379618767 done
#> lambda = 0.00181167241344908 done
#> lambda = 0.00185909047851328 done
#> lambda = 0.00190774964703402 done
#> lambda = 0.00195768240320877 done
#> lambda = 0.00200892208146477 done
#> lambda = 0.00206150288871263 done
#> lambda = 0.00211545992718236 done
#> lambda = 0.0021708292178572 done
#> lambda = 0.0022276477245207 done
#> lambda = 0.00228595337843324 done
#> lambda = 0.00234578510365442 done
#> lambda = 0.00240718284302827 done
#> lambda = 0.00247018758484848 done
#> lambda = 0.00253484139022168 done
#> lambda = 0.00260118742114685 done
#> lambda = 0.00266926996932968 done
#> lambda = 0.00273913448575107 done
#> lambda = 0.00281082761100967 done
#> lambda = 0.00288439720645843 done
#> lambda = 0.00295989238615622 done
#> lambda = 0.00303736354965569 done
#> lambda = 0.00311686241564937 done
#> lambda = 0.00319844205649628 done
#> lambda = 0.00328215693365241 done
#> lambda = 0.00336806293402837 done
#> lambda = 0.00345621740729875 done
#> lambda = 0.00354667920418803 done
#> lambda = 0.00363950871575844 done
#> lambda = 0.00373476791372625 done
#> lambda = 0.00383252039183328 done
#> lambda = 0.00393283140830113 done
#> lambda = 0.00403576792939676 done
#> lambda = 0.00414139867413822 done
#> lambda = 0.00424979416017051 done
#> lambda = 0.00436102675084223 done
#> lambda = 0.00447517070351439 done
#> lambda = 0.00459230221913353 done
#> lambda = 0.00471249949310251 done
#> lambda = 0.00483584276748264 done
#> lambda = 0.00496241438456225 done
#> lambda = 0.0050922988418272 done
#> lambda = 0.00522558284837033 done
#> lambda = 0.00536235538277719 done
#> lambda = 0.00550270775252699 done
#> lambda = 0.00564673365494821 done
#> lambda = 0.00579452923976965 done
#> lambda = 0.00594619317330869 done
#> lambda = 0.00610182670433955 done
#> lambda = 0.00626153373168565 done
#> lambda = 0.00642542087358098 done
#> lambda = 0.00659359753884704 done
#> lambda = 0.00676617599993262 done
#> lambda = 0.0069432714678654 done
#> lambda = 0.00712500216916525 done
#> lambda = 0.00731148942477062 done
#> lambda = 0.0075028577310308 done
#> lambda = 0.0076992348428179 done
#> lambda = 0.00790075185881435 done
#> lambda = 0.00810754330903253 done
#> lambda = 0.0083197472446252 done
#> lambda = 0.00853750533004657 done
#> lambda = 0.00876096293762555 done
#> lambda = 0.00899026924461431 done
#> lambda = 0.00922557733277703 done
#> lambda = 0.00946704429058518 done
#> lambda = 0.0097148313180876 done
#> lambda = 0.00996910383452554 done
#> lambda = 0.0102300315887642 done
#> lambda = 0.0104977887726147 done
#> lambda = 0.0107725541371226 done
#> lambda = 0.0110545111118989 done
#> lambda = 0.0113438479275758 done
#> lambda = 0.0116407577414666 done
#> lambda = 0.0119454387665149 done
#> lambda = 0.0122580944036192 done
#> lambda = 0.0125789333774198 done
#> lambda = 0.0129081698756415 done
#> lambda = 0.0132460236920816 done
#> lambda = 0.0135927203733416 done
#> lambda = 0.0139484913693991 done
#> lambda = 0.0143135741881205 done
#> lambda = 0.0146882125538179 done
#> lambda = 0.0150726565699565 done
#> lambda = 0.0154671628861199 done
#> lambda = 0.015871994869346 done
#> lambda = 0.0162874227799474 done
#> lambda = 0.0167137239519333 done
#> lambda = 0.0171511829781538 done
#> lambda = 0.0176000919002905 done
#> lambda = 0.0180607504038193 done
#> lambda = 0.0185334660180767 done
#> lambda = 0.0190185543215616 done
#> lambda = 0.0195163391526117 done
#> lambda = 0.0200271528255935 done
#> lambda = 0.0205513363527505 done
#> lambda = 0.0210892396718587 done
#> lambda = 0.0216412218798403 done
#> lambda = 0.0222076514724914 done
#> lambda = 0.0227889065904854 done
#> lambda = 0.0233853752718142 done
#> lambda = 0.0239974557108372 done
#> lambda = 0.0246255565241103 done
#> lambda = 0.025270097023173 done
#> lambda = 0.0259315074944747 done
#> lambda = 0.0266102294866283 done
#> lambda = 0.0273067161051821 done
#> lambda = 0.0280214323151068 done
#> lambda = 0.0287548552511995 done
#> lambda = 0.0295074745366129 done
#> lambda = 0.0302797926097206 done
#> lambda = 0.0310723250595386 done
#> lambda = 0.0318856009699248 done
#> lambda = 0.0327201632727888 done
#> lambda = 0.0335765691105455 done
#> lambda = 0.0344553902080559 done
#> lambda = 0.0353572132543028 done
#> lambda = 0.0362826402940565 done
#> lambda = 0.0372322891297913 done
#> lambda = 0.0382067937341224 done
#> lambda = 0.0392068046730372 done
#> lambda = 0.0402329895402044 done
#> lambda = 0.0412860334026502 done
#> lambda = 0.0423666392581 done
#> lambda = 0.0434755285042897 done
#> lambda = 0.0446134414205616 done
#> lambda = 0.0457811376620642 done
#> lambda = 0.0469793967668878 done
#> lambda = 0.0482090186764737 done
#> lambda = 0.0494708242696441 done
#> lambda = 0.0507656559106095 done
#> lambda = 0.0520943780113195 done
#> lambda = 0.0534578776085326 done
#> lambda = 0.0548570649559902 done
#> lambda = 0.0562928741320886 done
#> lambda = 0.057766263663458 done
#> lambda = 0.0592782171648611 done
#> lambda = 0.0608297439958417 done
#> lambda = 0.0624218799345584 done
#> lambda = 0.0640556878692565 done
#> lambda = 0.0657322585078381 done
#> lambda = 0.0674527111060029 done
#> lambda = 0.0692181942144487 done
#> lambda = 0.0710298864456279 done
#> lambda = 0.0728889972605737 done
#> lambda = 0.0747967677763186 done
#> lambda = 0.0767544715944484 done
#> lambda = 0.0787634156513408 done
#> lambda = 0.080824941090658 done
#> lambda = 0.0829404241586763 done
#> lambda = 0.0851112771230494 done
#> lambda = 0.0873389492156189 done
#> lambda = 0.0896249275999015 done
#> lambda = 0.0919707383638993 done
#> lambda = 0.0943779475388957 done
#> lambda = 0.0968481621449158 done
#> lambda = 0.0993830312635516 done
#> lambda = 0.101984247138866 done
#> lambda = 0.104653546307113 done
#> lambda = 0.107392710756023 done
#> lambda = 0.110203569114436 done
#> lambda = 0.11308799787307 done
#> lambda = 0.116047922637237 done
#> lambda = 0.119085319412354 done
#> lambda = 0.122202215923096 done
#> lambda = 0.125400692967077 done
#> lambda = 0.128682885803965 done
#> lambda = 0.132050985580947 done
#> lambda = 0.135507240795513 done
#> lambda = 0.139053958796523 done
#> lambda = 0.142693507324557 done
#> lambda = 0.146428316092592 done
#> lambda = 0.150260878408039 done
#> lambda = 0.154193752837248 done
#> lambda = 0.158229564913567 done
#> lambda = 0.162371008890112 done
#> lambda = 0.166620849538419 done
#> lambda = 0.170981923994162 done
#> lambda = 0.175457143651187 done
#> lambda = 0.180049496105124 done
#> lambda = 0.184762047147857 done
#> lambda = 0.189597942814212 done
#> lambda = 0.194560411482202 done
#> lambda = 0.199652766028253 done
#> lambda = 0.204878406038829 done
#> lambda = 0.210240820079955 done
#> lambda = 0.215743588026133 done
#> lambda = 0.22139038345022 done
#> lambda = 0.227184976075852 done
#> lambda = 0.233131234294061 done
#> lambda = 0.239233127745763 done
#> lambda = 0.245494729971833 done
#> lambda = 0.251920221132546 done
#> lambda = 0.258513890798195 done
#> lambda = 0.265280140812751 done
#> lambda = 0.272223488232471 done
#> lambda = 0.279348568341428 done
#> lambda = 0.286660137745959 done
#> lambda = 0.294163077550112 done
#> lambda = 0.301862396614204 done
#> lambda = 0.30976323489867 done
#> lambda = 0.317870866895428 done
#> lambda = 0.32619070514906 done
#> lambda = 0.33472830387015 done
#> lambda = 0.343489362643203 done
#> lambda = 0.352479730231607 done
#> lambda = 0.361705408482189 done
#> lambda = 0.371172556331965 done
#> lambda = 0.380887493919766 done
#> lambda = 0.390856706805469 done
#> lambda = 0.401086850299677 done
#> lambda = 0.411584753906709 done
#> lambda = 0.422357425883884 done
#> lambda = 0.433412057920139 done
#> lambda = 0.444756029937101 done
#> lambda = 0.456396915015825 done
#> lambda = 0.468342484452477 done
#> lambda = 0.480600712946347 done
#> lambda = 0.493179783923648 done
#> lambda = 0.50608809500066 done
#> lambda = 0.519334263589867 done
#> lambda = 0.532927132652819 done
#> lambda = 0.546875776603577 done
#> lambda = 0.561189507366667 done
#> lambda = 0.575877880593591 done
#> lambda = 0.590950702042054 done
#> lambda = 0.606418034122151 done
#> lambda = 0.622290202613898 done
#> lambda = 0.638577803560576 done
#> lambda = 0.655291710342512 done
#> lambda = 0.672443080935995 done
#> lambda = 0.690043365362193 done
#> lambda = 0.708104313331023 done
#> lambda = 0.726637982085105 done
#> lambda = 0.745656744449013 done
#> lambda = 0.765173297089197 done
#> lambda = 0.785200668990109 done
#> lambda = 0.805752230152178 done
#> lambda = 0.826841700517433 done
#> lambda = 0.848483159128757 done
#> lambda = 0.870691053528856 done
#> lambda = 0.89348020940525 done
#> lambda = 0.916865840487693 done
#> lambda = 0.940863558704654 done
#> lambda = 0.96548938460563 done
#> lambda = 0.990759758056242 done
#> lambda = 1.01669154921327 done
#> lambda = 1.04330206978693 done
#> lambda = 1.07060908459796 done
#> lambda = 1.09863082343712 done
#> lambda = 1.12738599323524 done
#> lambda = 1.1568937905516 done
#> lambda = 1.18717391438939 done
#> lambda = 1.21824657934645 done
#> lambda = 1.25013252911025 done
#> lambda = 1.28285305030612 done
#> lambda = 1.31642998670789 done
#> lambda = 1.35088575382053 done
#> lambda = 1.38624335384437 done
#> lambda = 1.42252639103114 done
#> lambda = 1.45975908744177 done
#> lambda = 1.49796629911676 done
#> lambda = 1.53717353266969 done
#> lambda = 1.57740696231514 done
#> lambda = 1.6186934473422 done
#> lambda = 1.66106055004536 done
#> lambda = 1.70453655412475 done
#> lambda = 1.74915048356795 done
#> lambda = 1.79493212202598 done
#> lambda = 1.84191203269649 done
#> lambda = 1.89012157872731 done
#> lambda = 1.93959294415409 done
#> lambda = 1.99035915538588 done
#> lambda = 2.04245410325317 done
#> lambda = 2.09591256563287 done
#> lambda = 2.15077023066562 done
#> lambda = 2.20706372058067 done
#> lambda = 2.26483061614436 done
#> lambda = 2.32410948174859 done
#> lambda = 2.38493989115582 done
#> lambda = 2.44736245391801 done
#> lambda = 2.51141884248698 done
#> lambda = 2.5771518200344 done
#> lambda = 2.64460526899986 done
#> lambda = 2.71382422038624 done
#> lambda = 2.78485488382175 done
#> lambda = 2.85774467840886 done
#> lambda = 2.93254226438065 done
#> lambda = 3.00929757558572 done
#> lambda = 3.08806185282336 done
#> lambda = 3.16888767805117 done
#> lambda = 3.25182900948808 done
#> lambda = 3.33694121763614 done
#> lambda = 3.42428112224509 done
#> lambda = 3.51390703024444 done
#> lambda = 3.60587877466841 done
#> lambda = 3.70025775459964 done
#> lambda = 3.79710697615835 done
#> lambda = 3.89649109456442 done
#> lambda = 3.99847645730028 done
#> lambda = 4.10313114840362 done
#> lambda = 4.21052503391936 done
#> lambda = 4.32072980854125 done
#> lambda = 4.4338190434743 done
#> lambda = 4.54986823554988 done
#> lambda = 4.6689548576264 done
#> lambda = 4.79115841030913 done
#> lambda = 4.91656047502371 done
#> lambda = 5.0452447684788 done
#> lambda = 5.17729719855424 done
#> lambda = 5.31280592165191 done
#> lambda = 5.45186140154784 done
#> lambda = 5.59455646978453 done
#> lambda = 5.740986387644 done
#> lambda = 5.89124890974299 done
#> lambda = 6.04544434929257 done
#> lambda = 6.20367564506587 done
#> lambda = 6.36604843011863 done
#> lambda = 6.53267110230833 done
#> lambda = 6.70365489665919 done
#> lambda = 6.87911395962114 done
#> lambda = 7.05916542527236 done
#> lambda = 7.24392949351651 done
#> lambda = 7.43352951032648 done
#> lambda = 7.62809205008845 done
#> lambda = 7.8277470001013 done
#> lambda = 8.03262764728757 done
#> lambda = 8.24287076717395 done
#> lambda = 8.45861671520081 done
#> lambda = 8.68000952042159 done
#> lambda = 8.90719698165455 done
#> lambda = 9.14033076615132 done
#> lambda = 9.3795665108479 done
#> lambda = 9.62506392626567 done
#> lambda = 9.87698690313207 done
#> lambda = 10.1355036217917 done
#> lambda = 10.4007866644813 done
#> lambda = 10.6730131305433 done
#> lambda = 10.952364754655 done
#> lambda = 11.2390280281518 done
#> lambda = 11.5331943235267 done
#> lambda = 11.835060022188 done
#> lambda = 12.1448266455603 done
#> lambda = 12.4627009896181 done
#> lambda = 12.7888952629395 done
#> lambda = 13.123627228374 done
#> lambda = 13.4671203484181 done
#> lambda = 13.8196039343955 done
#> lambda = 14.1813132995423 done
#> lambda = 14.5524899160993 done
#> lambda = 14.9333815765149 done
#> lambda = 15.3242425588686 done
#> lambda = 15.7253337966232 done
#> lambda = 16.1369230528204 done
#> lambda = 16.5592850988362 done
#> lambda = 16.992701897814 done
#> lambda = 17.4374627928994 done
#> lambda = 17.8938647004022 done
#> lambda = 18.3622123080132 done
#> lambda = 18.8428182782098 done
#> lambda = 19.3360034569852 done
#> lambda = 19.8420970880404 done
#> lambda = 20.3614370325835 done
#> lambda = 20.8943699948809 done
#> lambda = 21.441251753712 done
#> lambda = 22.0024473998829 done
#> lambda = 22.5783315799556 done
#> lambda = 23.169288746357 done
#> lambda = 23.7757134140344 done
#> lambda = 24.3980104238278 done
#> lambda = 25.0365952127366 done
#> lambda = 25.6918940912594 done
#> lambda = 26.3643445279929 done
#> lambda = 27.0543954416807 done
#> lambda = 27.7625075009045 done
#> lambda = 28.4891534316206 done
#> lambda = 29.2348183327453 done
#> lambda = 30 done

#> $optLambda
#> [1] 0.001
#> 
#> $optPrec
#> A 25 x 25 ridge precision matrix estimate with lambda = 0.001000
#>            a          b         c          d          e         f …
#> a 19.6306358  0.4298834 -3.551785  1.8076633  0.4663132 -3.489980 …
#> b  0.4298834 14.3342848 -2.783266 -4.7566129 -5.4642169 -1.944148 …
#> c -3.5517850 -2.7832659 24.553653 -4.2398993  1.6340205  1.249596 …
#> d  1.8076633 -4.7566129 -4.239899 20.1589526  0.7106687 -1.938613 …
#> e  0.4663132 -5.4642169  1.634021  0.7106687 23.2040989 -3.120445 …
#> f -3.4899799 -1.9441482  1.249596 -1.9386126 -3.1204454 18.426248 …
#> … 19 more rows and 19 more columns
#> 
OPT$optLambda  # Optimal penalty
#> [1] 0.001
OPT$optPrec    # Regularized precision under optimal penalty
#> A 25 x 25 ridge precision matrix estimate with lambda = 0.001000
#>            a          b         c          d          e         f …
#> a 19.6306358  0.4298834 -3.551785  1.8076633  0.4663132 -3.489980 …
#> b  0.4298834 14.3342848 -2.783266 -4.7566129 -5.4642169 -1.944148 …
#> c -3.5517850 -2.7832659 24.553653 -4.2398993  1.6340205  1.249596 …
#> d  1.8076633 -4.7566129 -4.239899 20.1589526  0.7106687 -1.938613 …
#> e  0.4663132 -5.4642169  1.634021  0.7106687 23.2040989 -3.120445 …
#> f -3.4899799 -1.9441482  1.249596 -1.9386126 -3.1204454 18.426248 …
#> … 19 more rows and 19 more columns