topic Re: how to find the removed records by proc glm in SAS Procedures

how to find the removed records by proc glm

grace999 — Tue, 17 Jul 2018 15:27:39 GMT

I was using Proc GLM. the model removed 35 records when running the model. I checked the data, and no missing in either Y nor Xs. How do I find out those removed records? Thanks.

Re: how to find the removed records by proc glm

Reeza — Tue, 17 Jul 2018 16:02:02 GMT

@grace999 wrote:

I was using Proc GLM. the model removed 35 records when running the model. I checked the data, and no missing in either Y nor Xs. How do I find out those removed records? Thanks.

There has to be missing.

You need to check all variables in your CLASS, BY and/or MODEL statements. Replace the data set name with your data set and run it. If you still don't see the missing values please post your PROC GLM code, the log and the output from the code below as well.

proc format ;
	value $ missfmt ' '="Missing" other="Not Missing";
	value nmissfmt .="Missing" other="Not Missing";
run;

*Proc freq to count missing/non missing;
ods table onewayfreqs=temp;

proc freq data=YOUR_DATA_SET_NAME;
	table _all_ / missing;
	format _numeric_ nmissfmt. _character_ $missfmt.;
run;

Re: how to find the removed records by proc glm

grace999 — Tue, 17 Jul 2018 17:03:12 GMT

SAS Output

The FREQ Procedure

pct_num Frequency Percent CumulativeFrequency CumulativePercentNot Missing

1393

100.00

1393

100.00

cat_3lvl Frequency Percent CumulativeFrequency CumulativePercentNot Missing

1393

100.00

1393

100.00

weight_num Frequency Percent CumulativeFrequency CumulativePercentNot Missing

1393

100.00

1393

100.00

The GLM Procedure

Class Level InformationClass Levels Valuescat_3lvl

3	grp1 grp2 grp3

Number of Observations ReadNumber of Observations Used

1393

1350

The GLM Procedure

Dependent Variable: pct_num

Weight: weight_num

Source DF Sum of Squares Mean Square F Value Pr > FModelErrorCorrected Total

2	536.8515	268.4258	1.11	0.3287
1347	324685.3574	241.0433
1349	325222.2089

R-Square Coeff Var Root MSE pct_num Mean

0.001651

-17716.39

15.52557

-0.087634

Source DF Type I SS Mean Square F Value Pr > Fcat_3lvl

536.8515293

268.4257646

1.11

0.3287

Source DF Type III SS Mean Square F Value Pr > Fcat_3lvl

536.8515293

268.4257646

1.11

0.3287

The GLM Procedure

Level ofcat_3lvl N Sum ofWeights pct_numMean Std Devgrp1 664 367834grp2 97 260940grp3 589 306783

-0.08937308	13.9091106
-0.05322111	22.1748667
-0.11481910	15.9299903

Re: how to find the removed records by proc glm

grace999 — Tue, 17 Jul 2018 17:04:42 GMT

proc glm data=new;
class cat_3lvl;
model pct_num=cat_3lvl;
weight weight_num;
means cat_3lvl;
run;

Log:

NOTE: PROCEDURE GLM used (Total process time):
real time 0.04 seconds
cpu time 0.05 seconds

Re: how to find the removed records by proc glm

Reeza — Tue, 17 Jul 2018 17:23:00 GMT

Output from the PROC FREQ?

Re: how to find the removed records by proc glm

ballardw — Tue, 17 Jul 2018 17:25:06 GMT

@grace999 wrote:
proc glm data=new;
class cat_3lvl;
model pct_num=cat_3lvl;
weight weight_num;
means cat_3lvl;
run;

Log:

NOTE: PROCEDURE GLM used (Total process time):
real time 0.04 seconds
cpu time 0.05 seconds

I suspect the issue may be your weight variable. From documentation for Proc GLM:

An observation is used in the analysis only if the value of the WEIGHT statement variable is nonmissing and greater than zero.

So if weight_num is missing, 0 or negative the record is excluded.

Re: how to find the removed records by proc glm

grace999 — Tue, 17 Jul 2018 17:54:26 GMT

Yes. The weight variable includes negative. What shall I do if the weight variable is the one I really need to use since negative is valid values?

Re: how to find the removed records by proc glm

PaigeMiller — Tue, 17 Jul 2018 18:09:21 GMT

I don't know how a negative value for a weight can be "valid". The procedure can't use a negative weight. Explain how it can be valid.