topic Re: Output Dataset with Number of Observations Used from Proc Logistic in Statistical Procedures

Output Dataset with Number of Observations Used from Proc Logistic

chester2018 — Tue, 21 Jan 2025 21:07:15 GMT

Hi everyone! I'm running into some trouble with making a dataset from proc logistic. My number of observations used (2913) is smaller than the number of observations read (4457), and I want to create a dataset with only the number of observations used. Below is my code:

proc logistic data=mydata;
    class education_cat (ref="0") income_cat (ref="0")  work_cat (ref="0");
    model PASB_Cat (ref="0") = education_cat income_cat
	 work_cat / link=glogit;
	output out=used_obs (keep=PASB_Cat education_cat income_cat 
	 work_cat);
run;

data used_obs_clean;
    set used_obs;
    if nmiss(of PASB_Cat education_cat income_cat
	 work_cat) = 0;
run;

Below is the log of the code, which basically says "Invalid Numeric Data"

NOTE: Character values have been converted to numeric values at the places given by: (Line):(Column).
       71:17   72:46   
 NOTE: Invalid numeric data, PASB_Cat='0' , at line 71 column 17.

And of course since I have error messages, my output dataset has no data! I appreciate any help!

Re: Output Dataset with Number of Observations Used from Proc Logistic

StatDave — Tue, 21 Jan 2025 21:24:21 GMT

Use an ODS OUTPUT statement for this, not the OUTPUT statement. The ODS OUTPUT statement can save any displayed table in a data set. See this note on using it. The table you want is named NObs, so use this statement to keep just the number of observations used (dropping the number read):

ods output nobs=myobsused(where=(label ? "Used"));

Re: Output Dataset with Number of Observations Used from Proc Logistic

FreelanceReinh — Tue, 21 Jan 2025 22:36:18 GMT

Hello @chester2018,

Just to explain the issue with your DATA step: The NMISS function requires numeric arguments, but some of your variables are character. Use the CMISS function instead. It works for both numeric and character arguments. Since you restricted dataset USED_OBS to the variables of interest, you can also simplify the IF condition:

if cmiss(of _all_)=0;

Re: Output Dataset with Number of Observations Used from Proc Logistic

coder1234 — Tue, 21 Jan 2025 22:45:24 GMT

debugged?

Re: Output Dataset with Number of Observations Used from Proc Logistic

chester2018 — Tue, 21 Jan 2025 23:19:10 GMT

Thanks for the reply! But I'm struggling with the variables not being in the smaller dataset. This is my new code!

proc logistic data=mydata;
    class education_cat (ref="0") income_cat (ref="0")  work_cat (ref="0");
    model PASB_Cat (ref="0") = education_cat income_cat
	 work_cat / link=glogit;
	ods output nobs=myobsused (keep=PASB_Cat education_cat income_cat 
	 work_cat);
run;

Based on the Number of Observations Read (2913), I should have the variables that were read in the smaller dataset.

Re: Output Dataset with Number of Observations Used from Proc Logistic

StatDave — Tue, 21 Jan 2025 23:41:41 GMT

Your wording is confusing, but I now have to guess that you mean you want a data set of all of the observations that were used in the analysis. What I showed with the ODS OUTPUT statement just provides the actual count of those observations, not the observations themselves. If you want the used observations themselves, then use the OUTPUT statement with the PREDPROBS=I option to create a copy of your data set along with the predicted probabilities of your response levels. In the OUTPUT statement, you can use a WHERE clause to exclude any observations which have a missing predicted probability variable value or a missing response variable value. Such observations did not contribute to fitting the model. For example, if the numeric response is Y with levels 1,2, or 3:

output out=myusedobs(where=(IP_1 ne . and Y ne .)) predprobs=i;

Re: Output Dataset with Number of Observations Used from Proc Logistic

Ksharp — Wed, 22 Jan 2025 00:50:36 GMT

As FreelanceReinh said, using CMISS() instead of NMISS().
CMISS() could take into account of both character and numeric variable, while NMISS() is only suited for numeric variable:

if nmiss(PASB_Cat , education_cat ,income_cat ,work_cat) = 0;
-->
if cmiss(PASB_Cat , education_cat ,income_cat ,work_cat) = 0;

Re: Output Dataset with Number of Observations Used from Proc Logistic

chester2018 — Wed, 22 Jan 2025 03:40:08 GMT

This worked! Thank you so much for your help! I truly appreciate it! My smaller dataset has all the variables/observations I needed and there are no missing values!