When scoring data, can I not only score the probability, but also the ...

xzhang · Posted 09-28-2017 04:49 PM

When I score the cases in the new dataset, could I not only assgin the probability for each case, but also the lower limit and upper limit of the probability for each case? Thanks!

Norman21 · Posted 09-29-2017 01:50 AM

What procedure are you using? PROC SCORE? PROC LOGISTIC? PROC TPSPLINE? PROC PLM?

Do you have some code that you can share?

Norman.
SAS 9.4 (TS1M6) X64_10PRO WIN 10.0.17763 Workstation

Rick_SAS · Posted 09-29-2017 08:20 AM

Sure you can. For example, the SCORE statement in PROC PLM supports the LCLM and UCLM for the lower and upper limits of the mean prediction. If you've use the STORE statement to save the model from PROC LOGISTIC, the code looks something like this:

proc logistic data=Have plots=none;
model Y = X1;
store out=LogiModel;
run;

proc plm restore=LogiModel;
score data=ScoreData out=Scores predicted lclm uclm;
run;

xzhang · Posted 09-29-2017 08:25 AM

Thank you for your reply! I am using survival data mining node in SAS E Miner. I used the score node to socre the data after developing the model. The score code is the following:

*************************************;
*** begin scoring code for regression;
*************************************;

length _WARN_ $4;
label _WARN_ = 'Warnings' ;

length I__g_ $ 12;
label I__g_ = 'Into: _g_' ;
*** Target Values;
array SURV4DRF [2] $12 _temporary_ ('1' '0' );
label U__g_ = 'Unnormalized Into: _g_' ;
*** Unnormalized target values;
ARRAY SURV4DRU[2] _TEMPORARY_ (1 0);

drop _DM_BAD;
_DM_BAD=0;

/*-------------------------------------------------*/
/*Survival Score Code*/
/*-------------------------------------------------*/

label EM_SURVIVAL = "Survival Probability at Censoring Time";
label EM_SURVFCST = "Survival Probability at Future Time";
label EM_SURVEVENT = "Event Probability before or at the Future Time";
label EM_HAZARD = "Hazard Function at Censoring Time";
label EM_HZRDFCST = "Hazard Function at Future Time";

BadObs = 0;
if _T_ ne . and BadObs=0 then do;
T_FCST=_T_+4 ;

/*----------Generate Cubic Spline Basis Functions-------------*/
if _T_ > 10 then
_csb1 =(_T_-10)**3 - _T_**3 + 30*_T_**2 - 300*_T_;
else
_csb1=-_T_**3 + 30*_T_**2 - 300*_T_;

if _T_ > 20 then
_csb2 =(_T_-20)**3 - _T_**3 + 60*_T_**2 - 1200*_T_;
else
_csb2=-_T_**3 + 60*_T_**2 - 1200*_T_;

if _T_ > 30 then
_csb3 =(_T_-30)**3 - _T_**3 + 90*_T_**2 - 2700*_T_;
else
_csb3=-_T_**3 + 90*_T_**2 - 2700*_T_;

if _T_ > 40 then
_csb4 =(_T_-40)**3 - _T_**3 + 120*_T_**2 - 4800*_T_;
else
_csb4=-_T_**3 + 120*_T_**2 - 4800*_T_;

if _T_ > 50 then
_csb5 =(_T_-50)**3 - _T_**3 + 150*_T_**2 - 7500*_T_;
else
_csb5=-_T_**3 + 150*_T_**2 - 7500*_T_;

*** Check _t_ for missing values ;
if missing( _t_ ) then do;
substr(_warn_,1,1) = 'M';
_DM_BAD = 1;
end;

*** Check _csb1 for missing values ;
if missing( _csb1 ) then do;
substr(_warn_,1,1) = 'M';
_DM_BAD = 1;
end;

*** Check _csb2 for missing values ;
if missing( _csb2 ) then do;
substr(_warn_,1,1) = 'M';
_DM_BAD = 1;
end;

*** Check _csb3 for missing values ;
if missing( _csb3 ) then do;
substr(_warn_,1,1) = 'M';
_DM_BAD = 1;
end;

*** Check _csb4 for missing values ;
if missing( _csb4 ) then do;
substr(_warn_,1,1) = 'M';
_DM_BAD = 1;
end;

*** Check _csb5 for missing values ;
if missing( _csb5 ) then do;
substr(_warn_,1,1) = 'M';
_DM_BAD = 1;
end;

*** Check Age for missing values ;
if missing( Age ) then do;
substr(_warn_,1,1) = 'M';
_DM_BAD = 1;
end;

*** Check CAP for missing values ;
if missing( CAP ) then do;
substr(_warn_,1,1) = 'M';
_DM_BAD = 1;
end;

*** Generate dummy variables for Agey_c ;
drop _1_0 _1_1 _1_2 _1_3 _1_4 ;
*** encoding is sparse, initialize to zero;
_1_0 = 0;
_1_1 = 0;
_1_2 = 0;
_1_3 = 0;
_1_4 = 0;
if missing( Agey_c ) then do;
   _1_0 = .;
   _1_1 = .;
   _1_2 = .;
   _1_3 = .;
   _1_4 = .;
   substr(_warn_,1,1) = 'M';
   _DM_BAD = 1;
end;
else do;
   length _dm12 $ 12; drop _dm12 ;
   _dm12 = put( Agey_c , BEST12. );
   %DMNORMIP( _dm12 )
   if _dm12 = '0' then do;
      _1_0 = 1;
   end;
   else if _dm12 = '1' then do;
      _1_1 = 1;
   end;
   else if _dm12 = '3' then do;
      _1_3 = 1;
   end;
   else if _dm12 = '2' then do;
      _1_2 = 1;
   end;
   else if _dm12 = '4' then do;
      _1_4 = 1;
   end;
   else do;
      _1_0 = .;
      _1_1 = .;
      _1_2 = .;
      _1_3 = .;
      _1_4 = .;
      substr(_warn_,2,1) = 'U';
      _DM_BAD = 1;
   end;
end;

*** Generate dummy variables for EduLevel ;
drop _3_0 _3_1 _3_2 _3_3 _3_4 ;
*** encoding is sparse, initialize to zero;
_3_0 = 0;
_3_1 = 0;
_3_2 = 0;
_3_3 = 0;
_3_4 = 0;
if missing( EduLevel ) then do;
   _3_0 = .;
   _3_1 = .;
   _3_2 = .;
   _3_3 = .;
   _3_4 = .;
   substr(_warn_,1,1) = 'M';
   _DM_BAD = 1;
end;
else do;
   length _dm12 $ 12; drop _dm12 ;
   _dm12 = put( EduLevel , BEST12. );
   %DMNORMIP( _dm12 )
   _dm_find = 0; drop _dm_find;
   if _dm12 <= '3' then do;
      if _dm12 <= '2' then do;
         if _dm12 = '1' then do;
            _3_0 = 1;
            _dm_find = 1;
         end;
         else do;
            if _dm12 = '2' then do;
               _3_1 = 1;
               _dm_find = 1;
            end;
         end;
      end;
      else do;
         if _dm12 = '3' then do;
            _3_2 = 1;
            _dm_find = 1;
         end;
      end;
   end;
   else do;
      if _dm12 = '4' then do;
         _3_3 = 1;
         _dm_find = 1;
      end;
      else do;
         if _dm12 = '5' then do;
            _3_4 = 1;
            _dm_find = 1;
         end;
      end;
   end;
   if not _dm_find then do;
      _3_0 = .;
      _3_1 = .;
      _3_2 = .;
      _3_3 = .;
      _3_4 = .;
      substr(_warn_,2,1) = 'U';
      _DM_BAD = 1;
   end;
end;

*** Generate dummy variables for Emlevel ;
drop _4_0 _4_1 _4_2 _4_3 ;
*** encoding is sparse, initialize to zero;
_4_0 = 0;
_4_1 = 0;
_4_2 = 0;
_4_3 = 0;
if missing( Emlevel ) then do;
   _4_0 = .;
   _4_1 = .;
   _4_2 = .;
   _4_3 = .;
   substr(_warn_,1,1) = 'M';
   _DM_BAD = 1;
end;
else do;
   length _dm12 $ 12; drop _dm12 ;
   _dm12 = put( Emlevel , BEST12. );
   %DMNORMIP( _dm12 )
   if _dm12 = '1' then do;
      _4_0 = 1;
   end;
   else if _dm12 = '2' then do;
      _4_1 = 1;
   end;
   else if _dm12 = '3' then do;
      _4_2 = 1;
   end;
   else if _dm12 = '4' then do;
      _4_3 = 1;
   end;
   else do;
      _4_0 = .;
      _4_1 = .;
      _4_2 = .;
      _4_3 = .;
      substr(_warn_,2,1) = 'U';
      _DM_BAD = 1;
   end;
end;

*** Generate dummy variables for Gender ;
drop _6_0 _6_1 ;
if missing( Gender ) then do;
   _6_0 = .;
   _6_1 = .;
   substr(_warn_,1,1) = 'M';
   _DM_BAD = 1;
end;
else do;
   length _dm12 $ 12; drop _dm12 ;
   _dm12 = put( Gender , BEST12. );
   %DMNORMIP( _dm12 )
   if _dm12 = '0' then do;
      _6_0 = 1;
      _6_1 = 0;
   end;
   else if _dm12 = '1' then do;
      _6_0 = 0;
      _6_1 = 1;
   end;
   else do;
      _6_0 = .;
      _6_1 = .;
      substr(_warn_,2,1) = 'U';
      _DM_BAD = 1;
   end;
end;

*** Generate dummy variables for HiPo ;
drop _7_0 _7_1 _7_2 _7_3 _7_4 ;
*** encoding is sparse, initialize to zero;
_7_0 = 0;
_7_1 = 0;
_7_2 = 0;
_7_3 = 0;
_7_4 = 0;
if missing( HiPo ) then do;
   _7_0 = .;
   _7_1 = .;
   _7_2 = .;
   _7_3 = .;
   _7_4 = .;
   substr(_warn_,1,1) = 'M';
   _DM_BAD = 1;
end;
else do;
   length _dm12 $ 12; drop _dm12 ;
   _dm12 = put( HiPo , BEST12. );
   %DMNORMIP( _dm12 )
   if _dm12 = '0' then do;
      _7_0 = 1;
   end;
   else if _dm12 = '2' then do;
      _7_2 = 1;
   end;
   else if _dm12 = '3' then do;
      _7_3 = 1;
   end;
   else if _dm12 = '1' then do;
      _7_1 = 1;
   end;
   else if _dm12 = '4' then do;
      _7_4 = 1;
   end;
   else do;
      _7_0 = .;
      _7_1 = .;
      _7_2 = .;
      _7_3 = .;
      _7_4 = .;
      substr(_warn_,2,1) = 'U';
      _DM_BAD = 1;
   end;
end;

*** Generate dummy variables for Promoted ;
drop _9_0 _9_1 ;
if missing( Promoted ) then do;
   _9_0 = .;
   _9_1 = .;
   substr(_warn_,1,1) = 'M';
   _DM_BAD = 1;
end;
else do;
   length _dm12 $ 12; drop _dm12 ;
   _dm12 = put( Promoted , BEST12. );
   %DMNORMIP( _dm12 )
   if _dm12 = '0' then do;
      _9_0 = 1;
      _9_1 = 0;
   end;
   else if _dm12 = '1' then do;
      _9_0 = 0;
      _9_1 = 1;
   end;
   else do;
      _9_0 = .;
      _9_1 = .;
      substr(_warn_,2,1) = 'U';
      _DM_BAD = 1;
   end;
end;

*** Generate dummy variables for Spouse ;
drop _10_0 _10_1 ;
if missing( Spouse ) then do;
   _10_0 = .;
   _10_1 = .;
   substr(_warn_,1,1) = 'M';
   _DM_BAD = 1;
end;
else do;
   length _dm12 $ 12; drop _dm12 ;
   _dm12 = put( Spouse , BEST12. );
   %DMNORMIP( _dm12 )
   if _dm12 = '1' then do;
      _10_0 = 0;
      _10_1 = 1;
   end;
   else if _dm12 = '0' then do;
      _10_0 = 1;
      _10_1 = 0;
   end;
   else do;
      _10_0 = .;
      _10_1 = .;
      substr(_warn_,2,1) = 'U';
      _DM_BAD = 1;
   end;
end;

*** If missing inputs, use averages;
if _DM_BAD > 0 then do;
   _P0 = 0.0226759359;
   _P1 = 0.9773240641;
   goto SURV4DR1;
end;

*** Compute Linear Predictor;
drop _TEMP;
drop _LP0 ;
_LP0 = 0;

*** Effect: _t_ ;
_TEMP = _t_ ;
_LP0 = _LP0 + ( -1.42842396283266 * _TEMP);

*** Effect: _csb1 ;
_TEMP = _csb1 ;
_LP0 = _LP0 + ( -0.01535754286207 * _TEMP);

*** Effect: _csb2 ;
_TEMP = _csb2 ;
_LP0 = _LP0 + ( 0.01118630965477 * _TEMP);

*** Effect: _csb3 ;
_TEMP = _csb3 ;
_LP0 = _LP0 + ( -0.01194256527271 * _TEMP);

*** Effect: _csb4 ;
_TEMP = _csb4 ;
_LP0 = _LP0 + ( 0.01022061466094 * _TEMP);

*** Effect: _csb5 ;
_TEMP = _csb5 ;
_LP0 = _LP0 + ( -0.00371669297902 * _TEMP);

*** Effect: Agey_c ;
_TEMP = 1;
_LP0 = _LP0 + (                 4.0275846578654) * _TEMP * _1_0;
_LP0 = _LP0 + (                4.22606526886279) * _TEMP * _1_1;
_LP0 = _LP0 + (                3.64271459200542) * _TEMP * _1_2;
_LP0 = _LP0 + (                3.36781118290354) * _TEMP * _1_3;
_LP0 = _LP0 + (                               0) * _TEMP * _1_4;

*** Effect: EduLevel ;
_TEMP = 1;
_LP0 = _LP0 + (                0.08857144518881) * _TEMP * _3_0;
_LP0 = _LP0 + (                0.14704421842113) * _TEMP * _3_1;
_LP0 = _LP0 + (                0.34104973636078) * _TEMP * _3_2;
_LP0 = _LP0 + (                0.66474244705976) * _TEMP * _3_3;
_LP0 = _LP0 + (                               0) * _TEMP * _3_4;

*** Effect: Emlevel ;
_TEMP = 1;
_LP0 = _LP0 + (                0.68145919313593) * _TEMP * _4_0;
_LP0 = _LP0 + (                0.29754298008235) * _TEMP * _4_1;
_LP0 = _LP0 + (               -0.66171686655645) * _TEMP * _4_2;
_LP0 = _LP0 + (                               0) * _TEMP * _4_3;

*** Effect: Gender ;
_TEMP = 1;
_LP0 = _LP0 + ( 0.19470623601398) * _TEMP * _6_0;
_LP0 = _LP0 + ( 0) * _TEMP * _6_1;

*** Effect: HiPo ;
_TEMP = 1;
_LP0 = _LP0 + (                4.16393327855287) * _TEMP * _7_0;
_LP0 = _LP0 + (               -1.58016075023945) * _TEMP * _7_1;
_LP0 = _LP0 + (                2.34369735819832) * _TEMP * _7_2;
_LP0 = _LP0 + (                2.73316541811577) * _TEMP * _7_3;
_LP0 = _LP0 + (                               0) * _TEMP * _7_4;

*** Effect: Promoted ;
_TEMP = 1;
_LP0 = _LP0 + ( 1.55628770969706) * _TEMP * _9_0;
_LP0 = _LP0 + ( 0) * _TEMP * _9_1;

*** Effect: Spouse ;
_TEMP = 1;
_LP0 = _LP0 + ( 0.66291205484103) * _TEMP * _10_0;
_LP0 = _LP0 + ( 0) * _TEMP * _10_1;

*** Effect: Age ;
_TEMP = Age ;
_LP0 = _LP0 + ( -0.03816383026071 * _TEMP);

*** Effect: CAP ;
_TEMP = CAP ;
_LP0 = _LP0 + ( 0.16072692688469 * _TEMP);

*** Naive Posterior Probabilities;
drop _MAXP _IY _P0 _P1;
drop _LPMAX;
_LPMAX= 0;
_LP0 = -13.5255493090752 + _LP0;
if _LPMAX < _LP0 then _LPMAX = _LP0;
_LP0 = exp(_LP0 - _LPMAX);
_LPMAX = exp(-_LPMAX);
_P1 = 1 / (_LPMAX + _LP0);
_P0 = _LP0 * _P1;
_P1 = _LPMAX * _P1;

SURV4DR1:

*** Posterior Probabilities and Predicted Level;
label P__g_1 = 'Predicted: _g_=1' ;
label P__g_0 = 'Predicted: _g_=0' ;
P__g_1 = _P0;
_MAXP = _P0;
_IY = 1;
P__g_0 = _P1;
if (_P1 > _MAXP + 1E-8) then do;
_MAXP = _P1;
_IY = 2;
end;
I__g_ = SURV4DRF[_IY];
U__g_ = SURV4DRU[_IY];

*************************************;
***** end scoring code for regression;
*************************************;

EM_SUBHZRD1 = P__g_1;
EM_SUBHZRD0 =1-(EM_SUBHZRD1);
EM_HAZARD=EM_SUBHZRD1;

/*---------- Survival Function Estimation -------------*/
EM_SURVFCST=1;
_T0_ = _T_;
t0_fcst = t_fcst;
_t_ = 0;
do while (_t_ <= t0_fcst);