Hi,
I am SAS beginner. I need your help with simple question. I need to use my own metric - Log Loss Metric. If I want to use the Model Comparison node, I can't see this type of metric in the options, but I need it.
I would like to implement the following:
First:
Second:
So far I know how to compute Log Loss Metric in SAS Code node. Based on this article An Overview of Machine Learning with SAS® Enterprise Miner™ I implemented this code in Kaggle competition San Francisco Crime classification
DATA &EM_EXPORT_TRAIN; set &EM_IMPORT_DATA end=eof; array posteriorProbs(1:39) P_CategoryARSON P_CategoryASSAULT P_CategoryBAD_CHECKS P_CategoryBRIBERY P_CategoryBURGLARY P_CategoryDISORDERLY_CONDUCT P_CategoryDRIVING_UNDER_THE_INFL P_CategoryDRUG_NARCOTIC P_CategoryDRUNKENNESS P_CategoryEMBEZZLEMENT P_CategoryEXTORTION P_CategoryFAMILY_OFFENSES P_CategoryFORGERY_COUNTERFEITING P_CategoryFRAUD P_CategoryGAMBLING P_CategoryKIDNAPPING P_CategoryLARCENY_THEFT P_CategoryLIQUOR_LAWS P_CategoryLOITERING P_CategoryMISSING_PERSON P_CategoryNON_CRIMINAL P_CategoryOTHER_OFFENSES P_CategoryPORNOGRAPHY_OBSCENE_MA P_CategoryPROSTITUTION P_CategoryRECOVERED_VEHICLE P_CategoryROBBERY P_CategoryRUNAWAY P_CategorySECONDARY_CODES P_CategorySEX_OFFENSES_FORCIBLE P_CategorySEX_OFFENSES_NON_FORCI P_CategorySTOLEN_PROPERTY P_CategorySUICIDE P_CategorySUSPICIOUS_OCC P_CategoryTREA P_CategoryTRESPASS P_CategoryVANDALISM P_CategoryVEHICLE_THEFT P_CategoryWARRANTS P_CategoryWEAPON_LAWS; if F_Category = "ARSON" then t=1; if F_Category = "ASSAULT" then t=2; if F_Category = "BAD CHECKS" then t=3; if F_Category = "BRIBERY" then t=4; if F_Category = "BURGLARY" then t=5; if F_Category = "DISORDERLY CONDUCT" then t=6; if F_Category = "DRIVING UNDER THE INFLUENCE" then t=7; if F_Category = "DRUG/NARCOTIC" then t=8; if F_Category = "DRUNKENNESS" then t=9; if F_Category = "EMBEZZLEMENT" then t=10; if F_Category = "EXTORTION" then t=11; if F_Category = "FAMILY OFFENSES" then t=12; if F_Category = "FORGERY/COUNTERFEITING" then t=13; if F_Category = "FRAUD" then t=14; if F_Category = "GAMBLING" then t=15; if F_Category = "KIDNAPPING" then t=16; if F_Category = "LARCENY/THEFT" then t=17; if F_Category = "LIQUOR LAWS" then t=18; if F_Category = "LOITERING" then t=19; if F_Category = "MISSING PERSON" then t=20; if F_Category = "NON-CRIMINAL" then t=21; if F_Category = "OTHER OFFENSES" then t=22; if F_Category = "PORNOGRAPHY/OBSCENE MAT" then t=23; if F_Category = "PROSTITUTION" then t=24; if F_Category = "RECOVERED VEHICLE" then t=25; if F_Category = "ROBBERY" then t=26; if F_Category = "RUNAWAY" then t=27; if F_Category = "SECONDARY CODES" then t=28; if F_Category = "SEX OFFENSES FORCIBLE" then t=29; if F_Category = "SEX OFFENSES NON FORCIBLE" then t=30; if F_Category = "STOLEN PROPERTY" then t=31; if F_Category = "SUICIDE" then t=32; if F_Category = "SUSPICIOUS OCC" then t=33; if F_Category = "TREA" then t=34; if F_Category = "TRESPASS" then t=35; if F_Category = "VANDALISM" then t=36; if F_Category = "VEHICLE THEFT" then t=37; if F_Category = "WARRANTS" then t=38; if F_Category = "WEAPON LAWS" then t=39; retain logloss 0; logloss+log(posteriorProbs[t]); if eof then do; logloss=(-1*logloss)/_N_; PUT "result " logloss; end; RUN;
But I don't know how to transmit this result to the Model Comparison Node or how to put it into the Model Node. Maybe it is not possible. This metric is the basic/necessary in modeling and predictions.
Is there any other solution to the problem except my approach? I hope that there is more sofisticated approach to the problem than mine.
Thanks for help.
Are you ready for the spotlight? We're accepting content ideas for SAS Innovate 2025 to be held May 6-9 in Orlando, FL. The call is open until September 25. Read more here about why you should contribute and what is in it for you!
Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.
Find more tutorials on the SAS Users YouTube channel.