topic Re: How to perform cross validation with replication in Statistical Procedures

How to perform cross validation with replication

Dcicantab5 — Tue, 11 Oct 2016 04:07:05 GMT

How to perform cross validation (cv) with replication in proc logistic? Anyone has views on cv with replication vs cv without replication?

Re: How to perform cross validation with replication

Ksharp — Tue, 11 Oct 2016 12:22:57 GMT

INFLUENCE option of model can do Regress Diagnoise. Especially check DFBETAS diagnostic .

proc glmselect can do CV, but I don't know if it could do some logistic regression.

Re: How to perform cross validation with replication

Dcicantab5 — Wed, 12 Oct 2016 00:51:16 GMT

Sorry, maybe i got the terminology wrong, what i meant was how to do, say 10-fold cross validation with resampling with proc logistic (which apparently is as good as bootstrap with replacements, "..We also carried out cross-validation with replication. Here the cross-validation was replicated r times, with a different random split into k groups each time..." - this is the article, http://m.aje.oxfordjournals.org/content/early/2014/06/24/aje.kwu140.full.pdf ). Help most appreciated!

Re: How to perform cross validation with replication

Ksharp — Wed, 12 Oct 2016 02:25:13 GMT

HoHo. That is really a big problem.

proc glmselect/hpgenselect can do "10-fold cross validation with resampling" .

I think your best choice is leveraging the power of SAS/IML and mixed PROC like:

proc iml;

x=sample().

.........

submit;

proc logistic.....

......

endsubmit;

......

quit;

Post it at IML forum, maybe Rick can give you something.

Re: How to perform cross validation with replication

Dcicantab5 — Wed, 12 Oct 2016 04:43:20 GMT

ok, will do, thanks!