topic Re: How to get a Classification Chart for the TEST SET (not just train and validation)? in SAS Data Science

How to get a Classification Chart for the TEST SET (not just train and validation)?

Mike90 — Tue, 10 Oct 2017 19:36:02 GMT

My Data Partition node is set at 40:30:30 (Train:Validate:Test)

My model is set up like this:

Data Source -> Data Partition -> Decision Tree -> Model Comparison

-> Decision Tree /

I can get a Classification Chart for the Train and Validation parts when looking at the individual Decision Tree results and in the Model Comparison results.

How do a get a Classification Chart for the test results?

Thanks

(In case it is somehow relevant, there are interval and nominal inputs, and the target is binary.

I used the option for Default with Inverse Prior Weights as there is a 95 / 5 distribution, and I'm

interested in improving the ability to predict the rare event, which of course cause a reduction

in the ability to predict the common event.)

Re: How to get a Classification Chart for the TEST SET (not just train and validation)?

AnujaNagpal — Tue, 10 Oct 2017 20:06:38 GMT

You will see charts like ROC, score ranking overlays and other fit statistics for test data in model comparison node output.

Re: How to get a Classification Chart for the TEST SET (not just train and validation)?

CraigDeVault — Tue, 10 Oct 2017 20:10:34 GMT

You will need to write your own SAS code in order to get this classification matrix. If you add a SAS Code node after the Model Comparison node, you can write the following code to get the classification matrix:

***I am using the target variable named target. If my target variable is bad, then I use f_bad and I_bad (assuming no decisions or prior probabilities set).

proc freq data=&em_import_test;
tables f_target*I_target;
run;

***I am using the target variable named target. If my target variable is bad, then I use f_bad and I_bad (assuming decisions are set).

proc freq data=&em_import_test;
tables f_target*d_target;
run;