User-friendly SAS application for performing all possible mixed model selection Step1: Prescreening

[#1]

I developed a user-friendly SAS macro application to perform all possible mixed model selection of fixed effects including quadratic and cross products within a user-specified subset range in the presence of random and repeated measures effects using SAS PROC MIXED (Fernandez, 2007). This macro application, ALLMIXED will complement the model selection option currently available in the SAS PROC REG for multiple linear regressions and the SAS Proc GLMSELECT that focuses on the standard independently and identically distributed general linear model for univariate responses. Options are also included in this macro to select the best covariance structure associated with the user-specified fully saturated repeated measures model; to graphically explore and to detect statistical significance of user specified linear, quadratic, interaction terms for fixed effects; and to diagnose multicollinearity, via the VIF statistic for each continuous predictor, involved in each model selection step. Two model selection criteria, AICC (corrected Akaike Information Criterion) and MDL (minimal description length) are used in all possible model selection and summaries of the best model selection are compared graphically. In this community posting, I will describe the prescreening step of ALLMIXED model selection steps.

ALL POSSIBLE MODEL SELECTION STEPS

The recommended selection steps for performing the model selection in MIXED model is illustrated in Figure1. Although the recommended sequence of the steps is identified in the figure 1, it is not a requirement to follow the same sequence. Users are free to choose to run any model selection steps in any order they desire. However, before running these model selection steps the data format must be suitable for running the SAS PROC Mixed procedure. The following types of PC data formats can be used with the ALLMIXED macro: SAS temporary and permanent data files, Microsoft excel, COMMA or TAB delimited text file.

SAS 9.4 Modules required to run this macro:

• SAS/STAT: PROC MIXED, PROC CORR, PROC REG, PROC GLMSELECT
• SAS/GRAPH: PROC GCHART, PROC GPLOT, PROC G3D
• Base SAS ODS (RTF, HTML, PDF)
• SAS/ACCESS: PC FILES – PROC IMPORT and PROC EXPORT

Improved ALLMIXED SAS macro application

The original SAS macro application, I developed (Fernandez, 2007) is not compatible in SAS enterprise guide (SAS EG) or in SAS studio. Therefore, I am presenting an improved version of the ALLMIXED macro in this post. By using this improved ALLMIXED macro application, SAS users can effectively perform complete mixed model analysis in SAS studio or in SAS EG. First download and unzip the ALLMIXED.zip file specified in this post and save the contents to a custom folder such as C:\temp\allmixed. The extracted ALLMIXED zip file should include, compiled ALLMIXED macro catalog, five macro call files corresponding to six ALLMIXED model selection steps, and sample demo data used in the demo. In this article, I will present the steps needed to perform Step1-Prescreening.

STEP1: PRE-SCREENING

 If the number of fixed effects exceeds 10, running all possible models will take very long time to complete. Therefore, under these circumstances, pre-screening is recommended to discard least contributing model terms. In pre-screening, the repeated measures covariance structure is ignored, and the random effects could be treated as fixed. To discard the least important model terms and to select the user-specified number of effects, the LASSO method implemented in the GLMSELECT SAS procedure is used.

Pre-Screening: Suggested Steps

• Number of fixed effects terms: <= 10
  • Run all possible subset model selection.
• Number of fixed effects terms: > 10
  • Pre-screen: Run ALLMIXED using GLMSELECT (treat random and repeated measure terms as fixed effects) and select up to 10 fixed effects and identify the potential subset range.               
  • Run all possible subset model selection within the user-specified subset range in the ALLMIXED macro.

The LASSO model selection options - CHOOSE=NONE and SELECT=SBC are used in pre-screening. The ‘FIT CRITERIA" and the ‘COEFFICIENT EVALUATION' plots generated by the SAS ODS GRAPHICS features were utilized in the pre-screening evaluation to identify the potential subset ranges and to discard potential insignificant covariate and to select less than or equal to 10 potentially significant covariates. The LASSO selection method adds or drop an effect and compute several information criteria (IC) statistics in each step.

ALLMIXED SAS macro help – Step 1 Pre-screening

1. Input the Excel file name or SAS data set name?

Descriptions and Explanation: Include the data type name (XLS, TAB, TXT, SAS, TMP) and name of the data set on which you would like to perform pre-screening.

Options / Examples:

•  xls_SIMDATA1: Data type is EXCEL, and the file name is SIMDATA1. Make sure to include the separator character '_'.

SAS_SIMDATA1: Data type is permanent SAS (SD7SAS) and the SAS permanent data name is SIMDATA1

 TMP_SIMDATA1: Data type is temporary SAS data and file name is SIMDATA1.

2. Input required Response variable or variables

Descriptions and Explanation: Input the continuous response (dependent) variable name (or names). The name should match the variable names in the data. You can include multiple responses

Options / Examples:

• Y Y1 Y2

3. Pre-screening fixed variables using GLMSELECT

Descriptions and Explanation: To start the variable pre-screening mode, enter yes. Pre-screening based on the LASSO method using the selection method SBC and the stop criterion NONE will be performed. This field should be left blank to run all other model selection steps.

Options / Examples:

• yes  

4. Input optional CLASS terms

Descriptions and Explanation: Input the names of the categorical variables that will be included in the CLASS statement in PROC GLMSELECT.

Options / Examples:

TRT 

5. Input ith analysis (a counter) to attach to the saved output file name

Descriptions and Explanation: Input any numeric and categorical character to track the number of the analysis that you are running using this data. For example, if you input 1A, the output file created in this step would be called SIMDATA11A.ext.

Options / Examples:

Z = -1

Z = 1A

8. Input list of class (line1) and continuous fixed effects (line2)

Descriptions and Explanation: Input the list of model terms that will be used in the LASSO variable selection step in PROC GLMSELECT.

From this list of model terms, variables are selected in the pre-screening. If you cannot fit all model terms in the first line, then fill it in the second line.

Options / Examples:

Line1: trt 

Line2: x1-x15

11. Input the subject variable name

Descriptions and Explanation: In case of repeated measures data, input the subject variable name. This forces the pre-screening to do initial selection at the subject level.

Options:

•  Id, subject, or blank

14. Display or save the Graphs/output? choose one

Descriptions and Explanation: Option for viewing and saving all output files in a folder specified in input number 17.

WORD: Output and all SAS graphics are saved together in the user-specified folder as a single RTF format.

WEB: Output and graphics are saved in the user-specified folder as a single HTML file.

PDF: Output and graphics are saved in the user-specified folder and as a single PDF file.

TXT: Output is saved as a TXT file in all SAS versions. No output is displayed in the OUTPUT window. All graphic files are saved as PNG format in the user-specified folder.

15. Folder containing the PC data files

Descriptions and Explanation: Input the full path of the folder containing the source data file.

Options / Examples:

• 😧\allmixed\sasdata\ - folder name SASDATA on drive D

Make sure that you include the backslash (\) at the end of the folder name.

 OUTPUT= c:\temp\allmixed\

17. Folder to save the output/graphics?

Descriptions and Explanation: To save the SAS graphics, data, and output files, input the output folder name. If the 14 field is left blank, the output files are saved in the default folder.

Options / Examples:

Dir2= C:\temp\allmixed\

ALLMIXED macro results of pre-screening step

The FIT CRITERIA plots display the trend of 6 information criteria (IC) statistics in each step and the best subset is identified by a STAR symbol.  Because the STOP=SBC option was used, the FIT CRITERIA plot highlights the best subset based on the SBC criterion. The results of best subset selected by the different IC were not consistent and ranged between 3 to 11 subsets. The SBC criterion selected the parsimonious subset (subset 3) and AIC selected subset 11. Because all six IC failed to select the same step as the best subset, it is not advisable to rely only one IC statistic when performing model selection. Therefore, a subset range (2 and 6) was identified and used in the subsequent all possible model selection. The ‘COEFFICIENT EVALUATION’ plots displayed, shows the magnitude of the standardized regression coefficients of the selected model effects in each step along with the SBC criterion. This plot could help us to discard the insignificant effects and to select less than or equal 10 covariates which can be used in all possible mixed model selection in the next step. Based on coefficient evaluation plot, the following 4 covariates, x15, x5, x14, x6 and  TRT were selected and used in all possible mixed model.

 Reference

Fernandez, G. (2007) Model Selection in PROC MIXED - A User-friendly SAS® Macro Application SAS Global Forum proceedings 191-2007

https://support.sas.com/resources/papers/proceedings/proceedings/forum2007/191-2007.pdf