topic Re: Options to optimize a score mapping table in Mathematical Optimization, Discrete-Event Simulation, and OR

Options to optimize a score mapping table

aaaaawe2 — Mon, 28 Jan 2019 19:38:01 GMT

Hi,

I am looking to optimize a score mapping table but am not sure if there is a way in SAS. Following is the problem set up:

Score 1 is a score calculated by regression model. Score 2 is provided to us. And the predicted score is calculated based on score 1 and the value of score 2. Below is the mapping table:

Score 1 Score 2 adjustment

+1 0 -1

21 | Range 1 | Range 2 | Range 3|

22 | Range 4 | Range 5 | Range 6|

23 | Range 7 | Range 8 | Range 9|

To illustrate, if Score 1 from the model is 21 and score 2 is within Range 1 then predicted score is 22 (21+1). The goal is to minimize absolute(Predicted score - Actual score).

Would appreciate any help on this. Thanks!

Re: Options to optimize a score mapping table

RobPratt — Tue, 29 Jan 2019 00:49:15 GMT

The following documentation example illustrates minimizing absolute deviation:

https://go.documentation.sas.com/?docsetId=ormpex&docsetTarget=ormpex_ex11_toc.htm&docsetVersion=15.1&locale=en

If you are still stuck, please post the complete data for your small example.

Re: Options to optimize a score mapping table

aaaaawe2 — Tue, 29 Jan 2019 17:15:29 GMT

Thank you Rob for your reply. Following is more detail on my data:

Score_1 | Score_2 | Predicted Output (based on mapping table) | Target Output

21 55 23 22

21 20 18 20

22 58 24 24

22 40 22 23

23 69 25 24

Mapping table (that needs optimization):

Adjustment to Score 1

-2 | -1 | 0 | 1 | 2

Score1 / Score2 Range->

21 0-18 19-30 31-40 41-50 51+

22 0-20 21-34 35-48 49-58 59+

23 0-24 25-40 41-52 53-62 63+

I am looking to calibrate the above score2 ranges in the mapping table so the sum(abs(target-predicted output)) is minimized.

Hope this clarifies the optimization problem. Please let me know if further clarity is required.

Thanks!

Re: Options to optimize a score mapping table

RobPratt — Tue, 29 Jan 2019 22:19:41 GMT

It looks to me like the problem can be solved separately for each score 1 value (three values in your example), but please confirm.

Are the allowed adjustments -2 to 2 part of the input?

Is score 2 always a nonnegative integer?

In your example solution, each range is nonempty and increasing with respect to adjustment. Are those requirements, or is the solver allowed to return empty or nonincreasing ranges?

Re: Options to optimize a score mapping table

aaaaawe2 — Tue, 29 Jan 2019 22:29:40 GMT

Please find my response inline in blue font below:

Are the allowed adjustments -2 to 2 part of the input?

No they are not. The adjustments are a function of 1) Score 1 and 2) Score 2 values. The only inputs are Score 1 and Score 2, and the Target output is the desired output after the adjustment is applied to Score 1.

Is score 2 always a nonnegative integer?

Yes that's true

In your example solution, each range is nonempty and increasing with respect to adjustment. Are those requirements, or is the solver allowed to return empty or nonincreasing ranges?

Yes that's a strict requirement, the range value is increasing both horizontally and vertically.

Thanks!

Re: Options to optimize a score mapping table

RobPratt — Tue, 29 Jan 2019 23:32:37 GMT

OK, the vertical increase requirement implies that the problem cannot be solved one at a time for each score 1.

In your example, it looks like the second and third predicted scores should be 20 and 23 instead of 18 and 24. Please confirm.

Re: Options to optimize a score mapping table

aaaaawe2 — Wed, 30 Jan 2019 14:35:13 GMT

Yes, you are right, in the example the second and third predicted scores should be 20 and 23 respectively. Is there any way to solve this problem using optimization? thanks.

Re: Options to optimize a score mapping table

RobPratt — Wed, 30 Jan 2019 15:05:53 GMT

Here's a first crack, assuming adjustments -2, -1, 0, 1, 2, and ignoring vertical increase:

data indata;
   input Score_1 Score_2 Target;
   datalines;
21 55 22
21 20 20
22 58 24
22 40 23
23 69 24
;

proc optmodel;
   set OBS;
   num score_1 {OBS};
   num score_2 {OBS};
   num target {OBS};
   read data indata into OBS=[_N_] score_1 score_2 target;

   set SCORES1 = setof {i in OBS} score_1[i];
   num score_2_max {s in SCORES1} = max {i in OBS: score_1[i] = s} score_2[i];
   print score_2_max;

   num pmax = 5;
   set POSITIONS = 1..pmax+1;
   num source {s in SCORES1} = 0;
   num sink {s in SCORES1} = score_2_max[s] + 1 + pmax;
   set NODES {s in SCORES1} = (source[s]..sink[s]) cross POSITIONS;
   set ARCS {s in SCORES1} = setof {<i,p> in NODES[s], <j,p+1> in NODES[s]: i < j} <i,j,p>;
   var UseArc {s in SCORES1, ARCS[s]} binary;
   con Balance {s in SCORES1, <i,p> in NODES[s]}:
      sum {<(i),j,(p)> in ARCS[s]} UseArc[s,i,j,p] - sum {<j,(i),(p)-1> in ARCS[s]} UseArc[s,j,i,p-1]
    = (if i = source[s] and p = 1 then 1 else if i = sink[s] and p = pmax+1 then -1 else 0);
   set SUPPORT {s in SCORES1} = {<i,j,p> in ARCS[s]: UseArc[s,i,j,p].sol > 0.5};

   num adjustment {POSITIONS} = [-2 -1 0 1 2];
   var Prediction {OBS};
   var ErrorPlus {OBS} >= 0;
   var ErrorMinus {OBS} >= 0;
   impvar Error {i in OBS} = ErrorPlus[i] + ErrorMinus[i];
   min TotalError = sum {i in OBS} Error[i];
   con ErrorCon {i in OBS}:
      Prediction[i] - target[i] = ErrorPlus[i] - ErrorMinus[i];

   /* if score_2[o] in i..j-1 and UseArc[score_1[o],i,j,p] = 1 then Prediction[o] = score_1[o] + adjustment[p] */ 
   con PredictionCon {o in OBS}:
      Prediction[o] = score_1[o] + sum {<i,j,p> in ARCS[score_1[o]]: score_2[o] in i..j-1} adjustment[p] * UseArc[score_1[o],i,j,p];

   solve;
   num start {SCORES1, POSITIONS};
   for {s in SCORES1, <i,j,p> in SUPPORT[s]} start[s,p] = i;
   create data rangedata from [s]=SCORES1 {p in 1..pmax} <col('start'||p)=start[s,p]>;
   create data preddata(drop=o) from [o] score_1 score_2 Prediction target Error;
quit;

Re: Options to optimize a score mapping table

RobPratt — Wed, 30 Jan 2019 15:44:21 GMT

To enforce monotonicity, treat Start as a variable instead of a constant and introduce some additional constraints:

   var Start {SCORES1, POSITIONS};
   con StartCon {s in SCORES1, p in POSITIONS}:
      Start[s,p] = sum {<i,j,(p)> in ARCS[s]} i * UseArc[s,i,j,p];
   set SCORE1_PAIRS = {s1 in SCORES1, s2 in SCORES1: s1 < s2};
   con Monotonic {<s1,s2> in SCORE1_PAIRS, p in POSITIONS}:
      Start[s1,p] <= Start[s2,p];

   solve;
/*   num start {SCORES1, POSITIONS};*/
/*   for {s in SCORES1, <i,j,p> in SUPPORT[s]} start[s,p] = i;*/