10-FOLD CROSS VALIDATION & BOOTSTRAPPING
Hello @Muo ,
It would help if you tell us what kind of model you want to build and what SAS-procedure (PROC) you intend to use.
Some procedures have built-in Cross-Validation (CV).
Also, what do you want to achieve with your cross-validation? Probably n°1 in the list below (?).
1. 'cross-validation' as a method to estimate the generalization error (in case you do NOT have a VALIDATION set)
2. 'cross-validation' as a so-called ensemble-strategy
3. 'cross-validation' for model selection
4. 'cross-validation' for feature selection
Since you are posting in 'data mining' - board, have a look at the crossValidateML action.
Here's an example :
SAS® 9.4 and SAS® Viya® 3.5 Programming Documentation | SAS 9.4 / Viya 3.5
SAS Visual Data Mining and Machine Learning Programming Guide
Cross Validation of a Forest Model
https://go.documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/casactml/casactml_mltools_example01.htm?ho...
Or are you still using SAS Enterprise Miner under SAS 9.4? In that case the latter link is not useful to you.
Thanks,
Koen
Hello,
What version of SAS are you using?
Submit
%PUT &=sysvlong4;
to find out.
I mainly want to know if you are on SAS 9.4 (with Enterprise Miner) or on SAS VIYA (with Visual Machine learning and Model Studio).
Also, how many observations do you have
(I am not sure if the 10 in your reply is an absolute number 10 or a relative number, 10%)?
Cross-validation is not going to help if there is an absolute mismatch between number of records and number of variables in the model (or n° of candidate variables for inclusion in the model).
Thanks,
Koen
Are you ready for the spotlight? We're accepting content ideas for SAS Innovate 2025 to be held May 6-9 in Orlando, FL. The call is open until September 25. Read more here about why you should contribute and what is in it for you!
Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.
Find more tutorials on the SAS Users YouTube channel.