topic K folds cross validation in SAS Data Science

K folds cross validation

Muo — Mon, 21 Mar 2022 18:20:44 GMT

Hello
may I have question how I do folds cross validation by SAS cane you share some cod’s and Explain the difference
Thanks

Re: K folds cross validation

PaigeMiller — Mon, 21 Mar 2022 19:19:59 GMT

10-FOLD CROSS VALIDATION & BOOTSTRAPPING

Re: K folds cross validation

sbxkoenk — Mon, 21 Mar 2022 20:09:48 GMT

Hello @Muo ,

It would help if you tell us what kind of model you want to build and what SAS-procedure (PROC) you intend to use.
Some procedures have built-in Cross-Validation (CV).

Also, what do you want to achieve with your cross-validation? Probably n°1 in the list below (?).

1. 'cross-validation' as a method to estimate the generalization error (in case you do NOT have a VALIDATION set)
2. 'cross-validation' as a so-called ensemble-strategy
3. 'cross-validation' for model selection
4. 'cross-validation' for feature selection

Since you are posting in 'data mining' - board, have a look at the crossValidateML action.

Here's an example :

SAS® 9.4 and SAS® Viya® 3.5 Programming Documentation | SAS 9.4 / Viya 3.5
SAS Visual Data Mining and Machine Learning Programming Guide
Cross Validation of a Forest Model
https://go.documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/casactml/casactml_mltools_example01.htm?homeOnFail

Or are you still using SAS Enterprise Miner under SAS 9.4? In that case the latter link is not useful to you.

Thanks,

Koen

Re: K folds cross validation

Muo — Sat, 26 Mar 2022 19:20:27 GMT

Hi I am not sure if that what I want, I have small data set and I want to divide into 10 fold in every time one for test contain 10 entities and other for training around ten times also I want applying logistics and SVM !!

Re: K folds cross validation

sbxkoenk — Sat, 26 Mar 2022 20:12:39 GMT

Hello,

What version of SAS are you using?

Submit

%PUT &=sysvlong4;

to find out.

I mainly want to know if you are on SAS 9.4 (with Enterprise Miner) or on SAS VIYA (with Visual Machine learning and Model Studio).

Also, how many observations do you have
(I am not sure if the 10 in your reply is an absolute number 10 or a relative number, 10%)?

Cross-validation is not going to help if there is an absolute mismatch between number of records and number of variables in the model (or n° of candidate variables for inclusion in the model).

Thanks,

Koen

Re: K folds cross validation

Muo — Sun, 27 Mar 2022 12:45:45 GMT

Hello,
I am on SAS academic for students The total observations equal 120
The Total number of columns are 45
I want to apply 2 models after dividing my data set