Building models with SAS Enterprise Miner, SAS Factory Miner, SAS Visual Data Mining and Machine Learning or just with programming

How to apply chracteristics of data on another data Set

Accepted Solution Solved
Reply
Occasional Contributor
Posts: 6
Accepted Solution

How to apply chracteristics of data on another data Set

Hi,

I have a data set which has its characteristics (cluster, Strat,...) and each has proporation. how can select from another data set data has the same characteristics same as the first one?

Accepted Solutions
Solution
Monday
SAS Employee
Posts: 120

Re: How to apply chracteristics of data on another data Set

In general, this would not typically be a concern of a data mining problem since the methods and software are intended for large data sets.  Sampling can be helpful when resources or limited or when the distributions in the data are not reflective of the population, but the goal is to have your training/validation/test sample be as representative as possible.  

 

If you are sampling, however, one approach would be to cluster the first data set, score the second data set using the cluster solution obtained on the first, and then sample proportionally from the second data set based on the distribution of clusters in the first data set.   You might consider stratifying on certain grouping variables (e.g. gender, location, etc...) to make the distribution as balanced as possible.

 

I hope this helps!

Doug 

View solution in original post


All Replies
Solution
Monday
SAS Employee
Posts: 120

Re: How to apply chracteristics of data on another data Set

In general, this would not typically be a concern of a data mining problem since the methods and software are intended for large data sets.  Sampling can be helpful when resources or limited or when the distributions in the data are not reflective of the population, but the goal is to have your training/validation/test sample be as representative as possible.  

 

If you are sampling, however, one approach would be to cluster the first data set, score the second data set using the cluster solution obtained on the first, and then sample proportionally from the second data set based on the distribution of clusters in the first data set.   You might consider stratifying on certain grouping variables (e.g. gender, location, etc...) to make the distribution as balanced as possible.

 

I hope this helps!

Doug 

☑ This topic is SOLVED.

Need further help from the community? Please ask a new question.

Discussion stats
  • 1 reply
  • 124 views
  • 0 likes
  • 2 in conversation