topic Re: SAM EM: confusion matrix on test/validation data? in SAS Data Science

SAM EM: confusion matrix on test/validation data?

ycenycute — Sat, 06 Nov 2021 03:18:31 GMT

I have a dataset where the Y variable is a binary variable. I have partitioned the data into training and testing, and connect a logistic regression node to the partition node. I am wondering the is confusion matrix in the results only for the training data only? Cuz I see the role = TRAIN (see below). If so, is there a way to get the confusion matrix for testing data to do the evaluation?

Re: SAM EM: confusion matrix on test/validation data?

sbxkoenk — Sat, 06 Nov 2021 13:07:22 GMT

Hello @ycenycute ,

The Logistic Regression node does not show the confusion matrix for the test data set. You will need to write your own SAS code in order to get this confusion matrix. If you add a SAS Code node after the Logistic Regression node, you can write the following code to get the confusion matrix:

*** I am using the target variable named target. If your target variable is named 'bad', then use f_bad and i_bad.

proc freq data=&_test;
tables f_target*i_target; /* f_ = from ; i_ = into */
run;

The code is the same for any modeling node. It can be Neural Networks, Tree, Regression, etc.

Good luck,

Koen

Re: SAM EM: confusion matrix on test/validation data?

ycenycute — Tue, 09 Nov 2021 07:48:09 GMT

I followed your code, I created a SAS code and connected to the regression node. I opened code editor in the property panel and wrote the following code in Training code.

proc freq data=&_test;
tables f_Status*i_Status; /* f_ = from ; i_ = into */
run;

But it threw run time error. The system error returned was 3000.

Re: SAM EM: confusion matrix on test/validation data?

ycenycute — Tue, 09 Nov 2021 07:55:44 GMT

Looks like the error is related to this "tables f_Status*i_Status; /* f_ = from ; i_ = into */". symbol not recognized.