Building models with SAS Enterprise Miner, SAS Factory Miner, SAS Visual Data Mining and Machine Learning or just with programming

Enterprise Miner missing observations

Reply
Contributor
Posts: 22

Enterprise Miner missing observations

I have a data source with 14000 obs. But I noticed today that my regression only had 1400 observations in it. I had partitioned it, so it should have 50% for Train, and thus be 7500. But no matter what I did - even removing all analytical and partition nodes and just running a descriptive node, gave me about 2900 obs. 

 

In desperation, I tried creating a new datasource, selected the same source. This finally solved the problem, but both data sources say the same - 14000 obs. But the first one only yields 2900 in any analysis or count. The new data source does not have the problem. What is going on? Tried refreshing meta data, but to no help. Very strange bug. 

 

Has anyone else encountered this?

SAS Employee
Posts: 122

Re: Enterprise Miner missing observations

Hi, In EM, like in many other SAS analytics procedures, if one input variable has missing values on a group of observations, that group of observations will be excluded from the procedure execution. So check the missingness of your input variables. Collectively, missingness can delete the observations by the % you are reporting here. In EM, please consider using Impute Node before regression. Hope this helps. Jason Xin
Ask a Question
Discussion stats
  • 1 reply
  • 219 views
  • 0 likes
  • 2 in conversation