Building models with SAS Enterprise Miner, SAS Factory Miner, SAS Visual Data Mining and Machine Learning or just with programming

Precision and recall scores in cutoff node

Accepted Solution Solved
Reply
Contributor
Posts: 45
Accepted Solution

Precision and recall scores in cutoff node

Hi, 

I am using Cutoff node and trying to see Recall score but cannot find it.

I have set the parameter of Cutoff method to Event Precision Equal Recall. In the results I get Overall precision rate and other various values but cannot figure out how would I get recall values and from that value how can I measure f-score/measure.

 

Regards

 


Accepted Solutions
Solution
a week ago
SAS Employee
Posts: 114

Re: Precision and recall scores in cutoff node

If you go to the SAS Enterprise Miner help available by opening SAS Enterprise Miner and then clicking on Help --> Contents, you can navigate to the Cutoff Node by navigating in the panel on the left to

 

Node Reference

    Assess Nodes

         Cutoff Node

 

and then navigate in the panel on the right to Cutoff Node Train Properties, you can scroll down until you see Event Precision Equal Recall where it says the following: 

    • Event Precision Equal Recall — With precision defined as % true predicted events / (true predicted + false predicted) and recall defined as the event classification rate, this method chooses the point at which precision and recall are equal.

 

There are two ways to find this in the output:  

(1) In the Overall Rates plot, a line is drawn at the requested point and hovering over the line with the mouse will show the cutoff  (see attached document for plot)

(2) In the Output section, you can see the point at which the first two columns are closest is when the cutoff is at 0.36 (I added the bold -- see attached document for partial table).  

 

--------------------------------------------------

|           |            |             |             |Overall |

|           | Event  |  True   | False    |Classif-|

|           |Precisi-|Positive|Positive|ication |

|           |on Rate|  Rate    |  Rate   |   Rate  |

|--------+--------+--------+---------+---------|
|Cutoff |            |            |             |             |
|--------|             |            |             |             |
|0.99    |  200.00|    8.30 |    0.00 |  161.78|

|--------+--------+--------+--------+----------|

|0.98    |  200.00|   13.26|    0.00|  162.77 |

|--------+--------+--------+--------+---------|

|0.97    |  196.67|   15.47|    0.07|  163.15 |

|--------+--------+--------+--------+----------|  

  .            .        .        .        .

  .            .        .        .        .

  .            .        .        .        .

|--------+--------+---------+--------+---------|

|0.38    |  134.11|  125.06|   15.31|  172.80|

|--------+---------+--------+--------+---------|

|0.37    |  132.51|  127.47|   16.18|  172.59|

|--------+---------+--------+--------+---------|

|0.36    |  131.01 | 129.67|  17.01 | 172.36|

|--------+---------+--------+--------+---------|

|0.35    |  128.45|  131.27|   18.22|  171.71|

|--------+---------+--------+--------+---------|

 

I hope this helps!

Doug

View solution in original post


All Replies
Solution
a week ago
SAS Employee
Posts: 114

Re: Precision and recall scores in cutoff node

If you go to the SAS Enterprise Miner help available by opening SAS Enterprise Miner and then clicking on Help --> Contents, you can navigate to the Cutoff Node by navigating in the panel on the left to

 

Node Reference

    Assess Nodes

         Cutoff Node

 

and then navigate in the panel on the right to Cutoff Node Train Properties, you can scroll down until you see Event Precision Equal Recall where it says the following: 

    • Event Precision Equal Recall — With precision defined as % true predicted events / (true predicted + false predicted) and recall defined as the event classification rate, this method chooses the point at which precision and recall are equal.

 

There are two ways to find this in the output:  

(1) In the Overall Rates plot, a line is drawn at the requested point and hovering over the line with the mouse will show the cutoff  (see attached document for plot)

(2) In the Output section, you can see the point at which the first two columns are closest is when the cutoff is at 0.36 (I added the bold -- see attached document for partial table).  

 

--------------------------------------------------

|           |            |             |             |Overall |

|           | Event  |  True   | False    |Classif-|

|           |Precisi-|Positive|Positive|ication |

|           |on Rate|  Rate    |  Rate   |   Rate  |

|--------+--------+--------+---------+---------|
|Cutoff |            |            |             |             |
|--------|             |            |             |             |
|0.99    |  200.00|    8.30 |    0.00 |  161.78|

|--------+--------+--------+--------+----------|

|0.98    |  200.00|   13.26|    0.00|  162.77 |

|--------+--------+--------+--------+---------|

|0.97    |  196.67|   15.47|    0.07|  163.15 |

|--------+--------+--------+--------+----------|  

  .            .        .        .        .

  .            .        .        .        .

  .            .        .        .        .

|--------+--------+---------+--------+---------|

|0.38    |  134.11|  125.06|   15.31|  172.80|

|--------+---------+--------+--------+---------|

|0.37    |  132.51|  127.47|   16.18|  172.59|

|--------+---------+--------+--------+---------|

|0.36    |  131.01 | 129.67|  17.01 | 172.36|

|--------+---------+--------+--------+---------|

|0.35    |  128.45|  131.27|   18.22|  171.71|

|--------+---------+--------+--------+---------|

 

I hope this helps!

Doug

☑ This topic is SOLVED.

Need further help from the community? Please ask a new question.

Discussion stats
  • 1 reply
  • 209 views
  • 1 like
  • 2 in conversation