topic Re: Missing observations in scored data, no missing data in SAS Data Science

Missing observations in scored data, no missing data

Xamius32 — Mon, 24 Oct 2016 18:16:07 GMT

I am building 4 different logistic models based on 4 different datasets, and then scoring 1 validation dataset with all 4 models to compare the scores.

But for some reason, the scored data is getting different # of observations, even though I know there are no missing values in training or validation data. Is there a reason this would occur?

Re: Missing observations in scored data, no missing data

Reeza — Mon, 24 Oct 2016 18:36:47 GMT

When you say missing data, do you mean that all categories are covered in scored data are also covered in training data?

Re: Missing observations in scored data, no missing data

Xamius32 — Mon, 24 Oct 2016 19:07:28 GMT

Well I guess the scored data set will have more categories than the training data. Is that a problem?

Re: Missing observations in scored data, no missing data

Reeza — Mon, 24 Oct 2016 19:37:00 GMT

If the category isn't in the training data, then yes it would be. It's equivalent to a missing value/category.

If the model is designed for sex=F or sex=M and sex = Unknown appears the model doesn't have a method to score the data and you'll end up with missing values.

Re: Missing observations in scored data, no missing data

Xamius32 — Mon, 24 Oct 2016 20:09:23 GMT

Well I am not sure that is the problem. Some of the scored data match the # of obs in the training data, and some match the # of obs in the validation data.I cant figure it out.

Re: Missing observations in scored data, no missing data

Xamius32 — Mon, 24 Oct 2016 21:35:18 GMT

So, I see that my socre node has different inputted data. One has the regression train data and one has the validation data, just not sure how that has happened.