In EM; Can I exceed the "Number to keep" on Association rules?

[bman1]

Hi,

I am using Enterprise miner Association rules. I want to explore more rules than the limit, the "number to keep" seems to be capped at 10,000. Is there a way to exceed this limit?

Many thanks

[DougWielenga]

There are settings for the number of rules generated (the maximum setting is 10,000) and for the number rules which are transposed (the maximum setting is 200).  There are practical limitations to how many rules can be transposed since each rule becomes a new column in the transposed data set. Note that you can choose which rules to transform since it is almost certainly unnecessary to transform all 10,000 rules.  In fact, searching through 10,000 rules is certainly an ominous task so consider choosing alternate node settings to mask trivial cases involving too few transactions to be meaningful.   If you still find you need more than 10,000 rules, contact SAS Technical Support who can provide you information on how to customize the number of rules available by modifying the files underlying the Association node which controls those settings.  Please note that doing so might require administrative permission in your environment. 

Hope this helps!

Doug   

[bman1]

Thanks for this, indeed I do not need all 10,000 rules. The reason I am asking to extend beyond the limit is that a good number of the rules I get do not fit my purpose. I.e. I have the requirement that there is only one item on the RHS. This means that of the 10k rules I get out about 8k aren't useful to me. Is there a way to play with the association rules properties to set this requirement and therefore not need 10k rules?

Many thanks for your help!

[DougWielenga · Accepted solution]

What you are experiencing is not uncommon.  From a software standpoint, there is not a reliable way to identify which rules are useful or important since this depends on the business scenario.   Commonly reported statistics such as Lift, Confidence, and Support do not necessarily do a good job of identifying useful rules by themselves.  Modifying your node settings to obtain fewer rules can help limit the total number of rules but there is no guarantee that you are not suppressing rules which you would be highly interested in.  Given that you are planning on filtering the rules (which is a great strategy), I would suggest that you contact SAS Tech Support to get some direction on how to modify the node .XML file to allow a larger number of rules.  

Hope this helps!

Doug