About PrakashSridharan

PrakashSridharan · ‎01-31-2012

That is correct. The goal is to count every possible 3 way/4 way and 5 way occurence i.e. in the above binary matrix all possibilities of 3 way, 4 way and 5 way frequencies of combinations of each item. Your idea was very interesting in terms of using the sum to determine the combinations. The way we should look to do it is to try and iteratively sum the binary variables within each row. So, to evaluate 3 way combinations you need to create 72C3 variables and scan for variables where we have sum = 3. Likewise for 4 and 5 way combinations. To my knowledge this can be accomplished using macros but thats where I feel the root cause of our problems exist - Memory. I think the iterations are prohibitively high for my system. Let us know your thoughts. Thanks, Prakash

PrakashSridharan · ‎01-31-2012

Hi Astounding, When I carefully reviewed your code i realized it doesn't produce what I exactly wanted. For Example, if you take a row with n_items = 5, it only produces the 5 items in the presented variables. However, there are 5C3 three way interactions and 5C4 4 way interations which do not get produced. Thats the whole idea of frequent item sets which we are trying to arrive at here. Let us know your thoughts. Thanks, Prakash

PrakashSridharan · ‎01-29-2012

Hi Astounding, Thanks for your response. I'm currently identifying certain specific products and producing association rules using proc freq. Its worked well for me and has not put too much constraint on the system resources. However, I plan to revisit the approach using your technique in the coming days as my end goal is to have a complete picture. I'll let you know how it works. Thanks once again. Prakash

PrakashSridharan · ‎01-24-2012

Hard Luck with this approach as well.:smileyconfused: I'm encountering the same lengthy runs followed by the out of memory error. I think this is a SAS environment issue versus the aproach we are taking. I'm going to talk to my admin to see if something can be done. Thanks, Prakash

PrakashSridharan · ‎01-23-2012

Hi MikeZDeb, Let me try this approach first thing in the morning and let you know how it worked. Thanks, Prakash

PrakashSridharan · ‎01-23-2012

Hi Doc, Thanks for your response. I did realize up front that the number of combinations that can be possible is pretty high. I actually had put in a control within the code where I filter for and add only those frequencies with a compulsary positive value i.e. something like this proc freq data = dataset; tables Item1*Item2*Item3/out = freqset; where Item1 = 1 and Item2 = 1 and Item3 = 1; run; The Item1, Item2 and Item3 will be controlled using a loop to ensure there are no repeats and scenarios where Item1 = Item2. From your comments above I think 3 way should have worked because 60K rows should have been handled by SAS, but it looks like the brute force method is too intensive for SAS. I do agree that 1+ million and 13 million was always going to be iffy. Thanks, Prakash

PrakashSridharan · ‎01-23-2012

Hi, I'm working with a sas dataset containing 72 Binary Variables (indicating occurence/non - occurence of an item in the transaction) and 1 ID variable similar to the following setup: ID Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Item 7 Item 8 Item 9 Item 10 1 1 0 0 0 1 0 0 0 0 0 2 0 0 0 0 1 1 0 0 0 0 I want to use PROC FREQ or PROC SQL to produce all possible 3, 4 and 5 way frequencies of these binary variables. I want the outputs is tables respectively designed for 3, 4 and 5 way possibilities as follows: Variable 1 Variable 2 Variable 3 Count Item 1 Item 2 Item 3 3 Item 1 Item 5 Item 6 8 I tried producing this using sas looping macros with a mix of PROC FREQ, DATA STEP and PROC APPEND. I'm unable to produce the code here due to confidentiality reasons. I keep getting an out of memory error (in my Unix Server based SAS EG with a Windows Frontend) beyond a certain point of time which typically runs for several hours (only to discover the error). I was able to get this done for 2 way frequencies beyond which I have been without any luck. My end goal is to produce a market basket analysis. Please let me know if there is a more efficient alternative to get this done. Thanks in advance for your help. Thanks, Prakash

Online Status	Offline
Date Last Visited	‎09-01-2015 07:11 AM

Producing all possible 3 way, 4 way and 5 way frequencies using PROC F...

Producing all possible 3 way, 4 way and 5 way frequencies using PROC F...

Producing all possible 3 way, 4 way and 5 way frequencies using PROC F...

Producing all possible 3 way, 4 way and 5 way frequencies using PROC F...

Producing all possible 3 way, 4 way and 5 way frequencies using PROC F...

Producing all possible 3 way, 4 way and 5 way frequencies using PROC F...

Producing all possible 3 way, 4 way and 5 way frequencies using PROC F...

Producing all possible 3 way, 4 way and 5 way frequencies using PROC F...

Producing all possible 3 way, 4 way and 5 way frequencies using PROC F...

Producing all possible 3 way, 4 way and 5 way frequencies using PROC F...

Producing all possible 3 way, 4 way and 5 way frequencies using PROC F...

Producing all possible 3 way, 4 way and 5 way frequencies using PROC F...

Producing all possible 3 way, 4 way and 5 way frequencies using PROC F...

Producing all possible 3 way, 4 way and 5 way frequencies using PROC F...