topic Re: Evaluation problem using Start group, End Group nodes - Cross Validation in SAS Data Science

Evaluation problem using Start group, End Group nodes - Cross Validation

baabed — Tue, 10 Mar 2020 17:08:54 GMT

Hello everyone,

I'm running the below flow in SAS Enterprise Miner Version 15.1 with the purpose of running a semi-automatic hyper-parameter tuning using cross-validation and comparing the CV error of each parameters combination to select the best model within the logistic regression model for example.

I encounter two problems:

First: running each Model Import seperately runs smoothly as seen through the green ticks.

Running however the model comparison leads to the issue of the first model import flow not running as it stops at the stepwise node.

Second: The results of End Group do not include the appropiate score ranking measures like cumulative lift or gain etc but instead the predicted mean. Does anyone know why this is the case?

Any help is appreciated!

Thank you.

Re: Evaluation problem using Start group, End Group nodes - Cross Validation

WendyCzika — Wed, 11 Mar 2020 13:29:48 GMT

For logistic regression and lift/gain charts, you need to have a categorical target (binary or nominal) and it seems like your target might have level of Interval if you are getting those predicted plots instead of the lift/gain plots. So you might want to check in your Input Data node that the target has the correct level set.

Re: Evaluation problem using Start group, End Group nodes - Cross Validation

baabed — Sun, 15 Mar 2020 09:04:55 GMT

Perfect! My target's role was set to interval indeed (although it is binary). Thank you! 🙂