Little's MCAR Macro

[BharatKoti]

Hi, 

Could you please provide working macro for Little MCAR Test in SAS. The Macro which is already there , contains stack of errors. 

I am attaching snippet of error and code i have used to run Base Sas.

Thanks,

Bharat

[ballardw]

When dealing with macros you want to set:

Options MPRINT SYMBOLGEN;

and then run the macro.

Post the LOG with code and the resulting diagnostics into a code box opened using the forum {I} menu icon to preserve plain text formatting.

You mentioned something about a "snippet of error" but that wasn't included in the attachment.

You may also need to describe what this is actually supposed to do. I'm not interested in running what a "Little MCAR Test" may require in terms of your data set or calculations.

Did your source for this macro include any information about the data requirements? Did you verify that your data met those requirements?

[BharatKoti]

Sorry for missing the attachment. To check whether missingness in dataset is due to MCAR , ran the macro code which was there on the net. I have the data set in the required format to run the macro. I am new to running macros, it would be of great help if you could me on the same. Please find attached data set which is an input data for macro. Thanks in advance. Mcar Error

%macro mcartest;
/* PROVIDE VALUES FOR THE FOLLOWING MACRO VARIABLES */
%let datafile = "C:\Users\Bharat\Desktop\7037 Project\hbat_missing_new"; * FILE PATH FOR RAW DATA;
%let numvars = 26; * NUMBER OF VARS IN DATA FILE;
%let misscode = .; * SPECIFY MISSING VALUE CODE;
%let varnames = v1_miss v2_miss v3_miss v4_miss v5_miss; * SPECIFY VARIABLE NAMES;
%let seednum = 564321; * SPECIFY RANDOM SEED;
/* DO NOT ALTER THE CODE BELOW */
data one;
infile &datafile ;
input &varnames ;
array m[&numvars] &varnames ; array r[&numvars] r1 - r&numvars ;
do i = 1 to &numvars;
if m[i] = &misscode then m[i] = .;
end; drop i;
do i = 1 to &numvars;
r[i] = 1;
if m[i] = . then r[i] = 0;
21
end; drop i;
proc sort;
by r1-r&numvars;
proc mi data = one seed = &seednum nimpute = 0 noprint;
var &varnames;
em outem = emcov;
proc iml;
use one; read all var {&varnames} into y;
read all var {%do i = 1 %to &numvars; r&i %end;} into r;
use emcov; read all var {&varnames} into em;
mu = em[1,]; sigma = em[2:nrow(em),];
/* ASSIGN AN INDEX VARIABLE DENOTING EACH CASE’S PATTERN */
jcol = j(nrow(y), 1 , 1);
do i = 2 to nrow(y);
rdiff = r[i,] - r[i - 1,];
if max(rdiff) = 0 & min(rdiff) = 0 then jcol[i,] = jcol[i - 1,];
else jcol[i,] = jcol[i - 1,] + 1;
end;
/* NUMBER OF DISTINCT MISSING DATA PATTERNS */
j = max(jcol);
/* PUT THE NUMBER OF CASES IN EACH PATTERN IN A COL VECTOR M */ /*
PUT THE MISSING DATA INDICATORS FOR EACH PATTERN IN A MATRIX RJ */
m = j(j, 1, 0); rj = j(j, ncol(r), 0);
do i = 1 to j;
count = 0;
do k = 1 to nrow(y);
if jcol[k,] = i then do;
count = count + 1;
end;
22
if jcol[k,] = i & count = 1 then rj[i,] = r[k,];
m[i,] = count;
end;
end;
/* COMPUTE D^2 STATISTIC FOR EACH J PATTERN */
d2j = j(j, 1, 0);
do i = 1 to j;
/* OBSERVED VALUES FOR PATTERN J */
yj = y[loc(jcol = i),loc(rj[i,] = 1)];
/* VARIABLE MEANS FOR PATTERN J */
ybarobsj = yj[+,]/nrow(yj);
/* D = P X Pj MATRIX OF INDICATORS (SEE P. 1199) */
Dj = j(ncol(y), rj[i,+], 0);
count = 1; do k = 1 to ncol(rj);
if rj[i,k] = 1 then do;
Dj[k, count] = 1;
count = count + 1;
end;
end;
/* REDUCE EM ESTIMATES TO CONTAIN OBSERVED ELEMENTS */
muobsj = mu * Dj; sigmaobsj = t(Dj) * sigma * Dj;
/* THE CONTRIBUTION TO THE D^2 STATISTIC FOR EACH OF THE J PATTERNS
*/
d2j[i,] = m[i,] * (ybarobsj - muobsj) * inv(sigmaobsj) * t(ybarobsj
- muobsj);
end;
23
/* THE D^2 STATISTIC */
d2 = d2j[+,];
/* DF FOR D^2 */
df = rj[+,+] - ncol(rj); p = 1 - probchi(d2,df);
/* PRINT ANALYSIS RESULTS */
file print; put "Number of Observed Variables = " (ncol(rj)) 3.0;
put "Number of Missing Data Patterns = " (j) 3.0; put; put "Summary
of Missing Data Patterns (0 = Missing, 1 = Observed)"; put; put
"Frequency | Pattern | d2j"; put; do i = 1 to nrow(rj);
put (m[i,]) 6.0 " | " @;
do j = 1 to ncol(rj);
put (rj[i,j]) 2.0 @;
end;
put " | " (d2j[i,]) 8.6;
end; put; put "Sum of the Number of Observed Variables Across
Patterns (Sigma psubj) = " (rj[+,+]) 5.0; put; put "Little’s (1988)
Chi-Square Test of MCAR"; put; put "Chi-Square (d2) = " (d2)
10.3; put "df (Sigma psubj - p) = " (df) 7.0; put "p-value
= " (p) 10.3;
%mend mcartest;
%mcartest;
run;

. 

[BharatKoti]

I wasn't able to upload the data set file , as it threw an error and said data type isn't supported

[ballardw]

---

@BharatKoti wrote:
I wasn't able to upload the data set file , as it threw an error and said data type isn't supported

---

Instructions here: https://communities.sas.com/t5/SAS-Communities-Library/How-to-create-a-data-step-version-of-your-dat... will show how to turn an existing SAS data set into data step code that can be pasted into a forum code box using the {i} icon or attached as text to show exactly what you have and that we can test code against.

[ballardw]

In the body of the code:

r[i] = 1;
if m[i] = . then r[i] = 0;
21                                  /*<= this is not valid code, likely a line number from somewhere, remove the 21*/
end; drop i;

You also apparently have incomplete data, the error about "too few variables" says that something is expecting more variables than you supplied.

[Reeza]

Where ever you got that macro from (copy and paste from PDF???) garbled the text somehow, messing up the macro code.

Find a better source.

http://www.appliedmissingdata.com/littles-mcar-test.sas

It's also old, you may want to look at PROC MI.

https://stats.idre.ucla.edu/sas/seminars/multiple-imputation-in-sas/mi_new_1/

[dhahs]

I'm trying to run the code from this site to test Little's MCAR: http://www.appliedmissingdata.com/littles-mcar-test.sas 

However, I am continuing to get errors.  I have tried redefining the datafile name based on other suggestions I have seen online, however nothing seems to be solving the problem. I'm pasting the code and output I get below. Any suggestions?

1 /**********************************************************************************************
1 ! ********************/
2 *
2 ! *
3 * This SAS macro implements the chi-square test for a missing completely at random (MCAR)
3 ! mechanism, as *
4 * outlined in Little's (1998) JASA article. Note that the macro requires SAS version 8.2 (or
4 ! higher) because *
5 * PROC MI is used to obtain ML estimates of the covariance matrix and mean vector.
5 ! * *
6 *
6 ! *
7 /**********************************************************************************************
7 ! ********************/;
8
9 %macro mcartest;
10
11 /* SPECIFY FILE PATH FOR THE INPUT DATA */
12 DATA = WORK.Newtwo;
13
14 /* SPECIFY INPUT DATA VARIABLE LIST */
15
16 %let varlist = Age
17 Began
18 Track
19 BioGender
20 Minority
21 GRE
22 GPA
23 CSES_Total_Ori
24 CSES_Total_Prac
25 CSES_Total_Intern
26 mnrty_2
27 mnrty_3
28 mnrty_1
29 mnrty_4
30 mnrty_5
31 mnrty_6;
32
33 /* SPECIFY VARIABLE SET FOR THE MCAR TEST */
34
35 %let testvars =
36 GRE
37 GPA
38 CSES_Total_Ori
39 CSES_Total_Prac
40 CSES_Total_Intern;
41
42 /* SPECIFY THE MISSING VALUE CODE */
43
44 %let misscode = .;
45
46 /*******************************/
47 /* DO NOT ALTER THE CODE BELOW */
48 /*******************************/
49
50 data one;
51 infile &datafile ;
52 input &varlist;
53
54 %let numvars = %sysfunc(countw(&testvars));
55
56 array m[&numvars] &testvars ;
57 array r[&numvars] r1 - r&numvars ;
58
59 do i = 1 to &numvars;
60 if m[i] = &misscode then m[i] = .;
61 end;
62 drop i;
63
64 do i = 1 to &numvars;
65 r[i] = 1;
66 if m[i] = . then r[i] = 0;
67 end;
68 drop i;
69
70 proc sort;
71 by r1-r&numvars;
72
73 proc mi data = one nimpute = 0 noprint;
74 var &testvars;
75 em outem = emcov;
76
77 proc iml;
78
79 use one;
80 read all var {&testvars} into y;
81 read all var {%do i = 1 %to &numvars; r&i %end;} into r;
82 use emcov;
83 read all var {&testvars} into em;
84
85 mu = em[1,];
86 sigma = em[2:nrow(em),];
87
88 /* ASSIGN AN INDEX VARIABLE DENOTING EACH CASE'S PATTERN */
89
90 jcol = j(nrow(y), 1 , 1);
91
92 do i = 2 to nrow(y);
93 rdiff = r[i,] - r[i - 1,];
94 if max(rdiff) = 0 & min(rdiff) = 0 then jcol[i,] = jcol[i - 1,];
95 else jcol[i,] = jcol[i - 1,] + 1;
96 end;
97
98 /* NUMBER OF DISTINCT MISSING DATA PATTERNS */
99
100 j = max(jcol);
101
102 /* PUT THE NUMBER OF CASES IN EACH PATTERN IN A COL VECTOR M */
103 /* PUT THE MISSING DATA INDICATORS FOR EACH PATTERN IN A MATRIX RJ */
104
105 m = j(j, 1, 0);
106 rj = j(j, ncol(r), 0);
107
108 do i = 1 to j;
109 count = 0;
110 do k = 1 to nrow(y);
111 if jcol[k,] = i then do;
112 count = count + 1;
113 end;
114 if jcol[k,] = i & count = 1 then rj[i,] = r[k,];
115 m[i,] = count;
116 end;
117 end;
118
119 /* COMPUTE D^2 STATISTIC FOR EACH J PATTERN */
120
121 d2j = j(j, 1, 0);
122
123 do i = 1 to j;
124
125 /* OBSERVED VALUES FOR PATTERN J */
126
127 yj = y[loc(jcol = i),loc(rj[i,] = 1)];
128
129 /* VARIABLE MEANS FOR PATTERN J */
130
131 ybarobsj = yj[+,]/nrow(yj);
132
133 /* D = P X Pj MATRIX OF INDICATORS (SEE P. 1199) */
134
135 Dj = j(ncol(y), rj[i,+], 0);
136
137 count = 1;
138 do k = 1 to ncol(rj);
139 if rj[i,k] = 1 then do;
140 Dj[k, count] = 1;
141 count = count + 1;
142 end;
143 end;
144
145 /* REDUCE EM ESTIMATES TO CONTAIN OBSERVED ELEMENTS */
146
147 muobsj = mu * Dj;
148 sigmaobsj = t(Dj) * sigma * Dj;
149
150 /* THE CONTRIBUTION TO THE D^2 STATISTIC FOR EACH OF THE J PATTERNS */
151
152 d2j[i,] = m[i,] * (ybarobsj - muobsj) * inv(sigmaobsj) * t(ybarobsj - muobsj);
153
154 end;
155
156 /* THE D^2 STATISTIC */
157
158 d2 = d2j[+,];
159
160 /* DF FOR D^2 */
161
162 df = rj[+,+] - ncol(rj);
163 p = 1 - probchi(d2,df);
164
165 /* PRINT ANALYSIS RESULTS */
166
167 file print;
168 put "Number of Observed Variables = " (ncol(rj)) 3.0;
169 put "Number of Missing Data Patterns = " (j) 3.0; put;
170 put "Summary of Missing Data Patterns (0 = Missing, 1 = Observed)"; put;
171 put "Frequency | Pattern | d2j"; put;
172 do i = 1 to nrow(rj);
173 put (m[i,]) 6.0 " | " @;
174 do j = 1 to ncol(rj);
175 put (rj[i,j]) 2.0 @;
176 end;
177 put " | " (d2j[i,]) 8.6;
178 end;
179 put;
180 put "Sum of the Number of Observed Variables Across Patterns (Sigma psubj) = " (rj[+,+]) 5.0;
180! put;
181 put "Little's (1988) Chi-Square Test of MCAR"; put;
182 put "Chi-Square (d2) = " (d2) 10.3;
183 put "df (Sigma psubj - p) = " (df) 7.0;
184 put "p-value = " (p) 10.3;
185
186 %mend mcartest;
187 %mcartest;
NOTE: Line generated by the invoked macro "MCARTEST".
1 DATA = WORK.Newtwo;
----
180

ERROR 180-322: Statement is not valid or it is used out of proper order.

WARNING: Apparent symbolic reference DATAFILE not resolved.
NOTE 137-205: Line generated by the invoked macro "MCARTEST".
3 data one; infile &datafile ; input &varlist;
-
22
ERROR 22-322: Syntax error, expecting one of the following: a name, a quoted string.

NOTE: Line generated by the invoked macro "MCARTEST".
3 data one; infile &datafile ; input &varlist;
-
200
ERROR 200-322: The symbol is not recognized and will be ignored.

WARNING: Apparent symbolic reference DATAFILE not resolved.

NOTE: The SAS System stopped processing this step because of errors.
WARNING: The data set WORK.ONE may be incomplete. When this step was stopped there were 0
observations and 21 variables.
NOTE: DATA statement used (Total process time):
real time 0.25 seconds
cpu time 0.03 seconds

NOTE: Input data set is empty.
NOTE: The data set WORK.ONE has 0 observations and 21 variables.
NOTE: PROCEDURE SORT used (Total process time):
real time 0.07 seconds
cpu time 0.00 seconds

NOTE: Writing HTML Body file: sashtml.htm
NOTE: No observations in data set WORK.ONE.
NOTE: The data set WORK.EMCOV has 0 observations and 7 variables.
NOTE: PROCEDURE MI used (Total process time):
real time 0.86 seconds
cpu time 0.28 seconds

NOTE: IML Ready
WARNING: Data set WORK.ONE is empty.

statement : USE at line 187 column 1
WARNING: End of File reached.

statement : READ at line 187 column 1
WARNING: End of File reached.

statement : READ at line 187 column 1
WARNING: Data set WORK.EMCOV is empty.

statement : USE at line 187 column 1
WARNING: End of File reached.

statement : READ at line 187 column 1
ERROR: (execution) Matrix has not been set to a value.

operation : [ at line 187 column 1
operands : em, *LIT1004,

em 0 row 0 col (type ?, size 0)

*LIT1004 1 row 1 col (numeric)

1

statement : ASSIGN at line 187 column 1
ERROR: (execution) Matrix has not been set to a value.

operation : [ at line 187 column 1
operands : em, *LIT1005, _TEM1001,

em 0 row 0 col (type ?, size 0)

*LIT1005 1 row 1 col (numeric)

2

_TEM1001 1 row 1 col (numeric)

0

statement : ASSIGN at line 187 column 1
ERROR: (execution) Invalid operand to operation.

operation : J at line 187 column 1
operands : _TEM1001, *LIT1006, *LIT1007

_TEM1001 1 row 1 col (numeric)

0

*LIT1006 1 row 1 col (numeric)

1

*LIT1007 1 row 1 col (numeric)

1

statement : ASSIGN at line 187 column 1
ERROR: (execution) Matrix has not been set to a value.

operation : MAX at line 187 column 1
operands : jcol

jcol 0 row 0 col (type ?, size 0)

statement : ASSIGN at line 187 column 1
ERROR: (execution) Matrix has not been set to a value.

operation : J at line 187 column 1
operands : j, *LIT1015, *LIT1016

j 0 row 0 col (type ?, size 0)

*LIT1015 1 row 1 col (numeric)

1

*LIT1016 1 row 1 col (numeric)

0

statement : ASSIGN at line 187 column 1
ERROR: (execution) Matrix has not been set to a value.

operation : J at line 187 column 1
operands : j, _TEM1001, *LIT1017

j 0 row 0 col (type ?, size 0)

_TEM1001 1 row 1 col (numeric)

0

*LIT1017 1 row 1 col (numeric)

0

statement : ASSIGN at line 187 column 1
ERROR: DO expression not given value.

statement : DO at line 187 column 1
ERROR: (execution) Matrix has not been set to a value.

operation : J at line 187 column 1
operands : j, *LIT1023, *LIT1024

j 0 row 0 col (type ?, size 0)

*LIT1023 1 row 1 col (numeric)

1

*LIT1024 1 row 1 col (numeric)

0

statement : ASSIGN at line 187 column 1
ERROR: DO expression not given value.

statement : DO at line 187 column 1
ERROR: (execution) Matrix has not been set to a value.

operation : [ at line 187 column 1
operands : d2j, $SUB0001,

d2j 0 row 0 col (type ?, size 0)

statement : ASSIGN at line 187 column 1
ERROR: (execution) Matrix has not been set to a value.

operation : [ at line 187 column 1
operands : rj, $SUB0001, $SUB0001

rj 0 row 0 col (type ?, size 0)

statement : ASSIGN at line 187 column 1
ERROR: (execution) Matrix has not been set to a value.

operation : PROBCHI at line 187 column 1
operands : d2, df

d2 0 row 0 col (type ?, size 0)

df 0 row 0 col (type ?, size 0)

statement : ASSIGN at line 187 column 1
NOTE: Non-portable document will be produced. The current settings of FORMCHAR use nonstandard
line-drawing characters and the resulting output file will not render correctly unless all
readers of the document have the SAS Monospace font installed. To make your document
portable, issue the following command:
OPTIONS FORMCHAR="|----|+|---+=|-/\<>*";

ERROR: Operand j does not have a value.

statement : PUT at line 187 column 1
ERROR: (execution) Matrix has not been set to a value.

operation : [ at line 187 column 1
operands : rj, $SUB0001, $SUB0001

rj 0 row 0 col (type ?, size 0)

statement : PUT at line 187 column 1
ERROR: Operand d2 does not have a value.

statement : PUT at line 187 column 1
ERROR: Operand df does not have a value.

statement : PUT at line 187 column 1
ERROR: Operand p does not have a value.

statement : PUT at line 187 column 1
188
189 run;
NOTE: Module MAIN is undefined in IML; cannot be RUN.

[Reeza]

Where did you get the macro from and can you please include the code you've used and how you called the macro. Yes, its in the log, but harder to read from there.  

[dhahs]

The macro came from here:  http://www.appliedmissingdata.com/macro-programs.html and I've pasted it below.

I've  using various syntax for calling in the data as the code in the macro for that, specifically

%let datafile = "c:\data\eatingrisk.dat"; 

created errors.  Thank you for your help!

/******************************************************************************************************************/
* *
* This SAS macro implements the chi-square test for a missing completely at random (MCAR) mechanism, as *
* outlined in Little's (1998) JASA article. Note that the macro requires SAS version 8.2 (or higher) because *
* PROC MI is used to obtain ML estimates of the covariance matrix and mean vector. * *
* *
/******************************************************************************************************************/;

%macro mcartest;

/* SPECIFY FILE PATH FOR THE INPUT DATA */
%let DATA = WORK.Newtwo;

/* SPECIFY INPUT DATA VARIABLE LIST */

%let varlist = Age
Began
Track
BioGender
Minority
GRE
GPA
CSES_Total_Ori
CSES_Total_Prac
CSES_Total_Intern
mnrty_2
mnrty_3
mnrty_1
mnrty_4
mnrty_5
mnrty_6;

/* SPECIFY VARIABLE SET FOR THE MCAR TEST */

%let testvars =
GRE
GPA
CSES_Total_Ori
CSES_Total_Prac
CSES_Total_Intern;

/* SPECIFY THE MISSING VALUE CODE */

%let misscode = .;

/*******************************/
/* DO NOT ALTER THE CODE BELOW */
/*******************************/

data one;
infile &datafile ;
input &varlist;

%let numvars = %sysfunc(countw(&testvars));

array m[&numvars] &testvars ;
array r[&numvars] r1 - r&numvars ;

do i = 1 to &numvars;
if m[i] = &misscode then m[i] = .;
end;
drop i;

do i = 1 to &numvars;
r[i] = 1;
if m[i] = . then r[i] = 0;
end;
drop i;

proc sort;
by r1-r&numvars;

proc mi data = one nimpute = 0 noprint;
var &testvars;
em outem = emcov;

proc iml;

use one;
read all var {&testvars} into y;
read all var {%do i = 1 %to &numvars; r&i %end;} into r;
use emcov;
read all var {&testvars} into em;

mu = em[1,];
sigma = em[2:nrow(em),];

/* ASSIGN AN INDEX VARIABLE DENOTING EACH CASE'S PATTERN */

jcol = j(nrow(y), 1 , 1);

do i = 2 to nrow(y);
rdiff = r[i,] - r[i - 1,];
if max(rdiff) = 0 & min(rdiff) = 0 then jcol[i,] = jcol[i - 1,];
else jcol[i,] = jcol[i - 1,] + 1;
end;

/* NUMBER OF DISTINCT MISSING DATA PATTERNS */

j = max(jcol);

/* PUT THE NUMBER OF CASES IN EACH PATTERN IN A COL VECTOR M */
/* PUT THE MISSING DATA INDICATORS FOR EACH PATTERN IN A MATRIX RJ */

m = j(j, 1, 0);
rj = j(j, ncol(r), 0);

do i = 1 to j;
count = 0;
do k = 1 to nrow(y);
if jcol[k,] = i then do;
count = count + 1;
end;
if jcol[k,] = i & count = 1 then rj[i,] = r[k,];
m[i,] = count;
end;
end;

/* COMPUTE D^2 STATISTIC FOR EACH J PATTERN */

d2j = j(j, 1, 0);

do i = 1 to j;

/* OBSERVED VALUES FOR PATTERN J */

yj = y[loc(jcol = i),loc(rj[i,] = 1)];

/* VARIABLE MEANS FOR PATTERN J */

ybarobsj = yj[+,]/nrow(yj);

/* D = P X Pj MATRIX OF INDICATORS (SEE P. 1199) */

Dj = j(ncol(y), rj[i,+], 0);

count = 1;
do k = 1 to ncol(rj);
if rj[i,k] = 1 then do;
Dj[k, count] = 1;
count = count + 1;
end;
end;

/* REDUCE EM ESTIMATES TO CONTAIN OBSERVED ELEMENTS */

muobsj = mu * Dj;
sigmaobsj = t(Dj) * sigma * Dj;

/* THE CONTRIBUTION TO THE D^2 STATISTIC FOR EACH OF THE J PATTERNS */

d2j[i,] = m[i,] * (ybarobsj - muobsj) * inv(sigmaobsj) * t(ybarobsj - muobsj);

end;

/* THE D^2 STATISTIC */

d2 = d2j[+,];

/* DF FOR D^2 */

df = rj[+,+] - ncol(rj);
p = 1 - probchi(d2,df);

/* PRINT ANALYSIS RESULTS */

file print;
put "Number of Observed Variables = " (ncol(rj)) 3.0;
put "Number of Missing Data Patterns = " (j) 3.0; put;
put "Summary of Missing Data Patterns (0 = Missing, 1 = Observed)"; put;
put "Frequency | Pattern | d2j"; put;
do i = 1 to nrow(rj);
put (m[i,]) 6.0 " | " @;
do j = 1 to ncol(rj);
put (rj[i,j]) 2.0 @;
end;
put " | " (d2j[i,]) 8.6;
end;
put;
put "Sum of the Number of Observed Variables Across Patterns (Sigma psubj) = " (rj[+,+]) 5.0; put;
put "Little's (1988) Chi-Square Test of MCAR"; put;
put "Chi-Square (d2) = " (d2) 10.3;
put "df (Sigma psubj - p) = " (df) 7.0;
put "p-value = " (p) 10.3;

%mend mcartest;
%mcartest;

run;

[sarah4]

i'm in the same case, please let me know if you succeeded.