BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
Mohamed
Calcite | Level 5
Hi,

I have a data set which has its characteristics (cluster, Strat,...) and each has proporation. how can select from another data set data has the same characteristics same as the first one?
1 ACCEPTED SOLUTION

Accepted Solutions
DougWielenga
SAS Employee

In general, this would not typically be a concern of a data mining problem since the methods and software are intended for large data sets.  Sampling can be helpful when resources or limited or when the distributions in the data are not reflective of the population, but the goal is to have your training/validation/test sample be as representative as possible.  

 

If you are sampling, however, one approach would be to cluster the first data set, score the second data set using the cluster solution obtained on the first, and then sample proportionally from the second data set based on the distribution of clusters in the first data set.   You might consider stratifying on certain grouping variables (e.g. gender, location, etc...) to make the distribution as balanced as possible.

 

I hope this helps!

Doug 

View solution in original post

1 REPLY 1
DougWielenga
SAS Employee

In general, this would not typically be a concern of a data mining problem since the methods and software are intended for large data sets.  Sampling can be helpful when resources or limited or when the distributions in the data are not reflective of the population, but the goal is to have your training/validation/test sample be as representative as possible.  

 

If you are sampling, however, one approach would be to cluster the first data set, score the second data set using the cluster solution obtained on the first, and then sample proportionally from the second data set based on the distribution of clusters in the first data set.   You might consider stratifying on certain grouping variables (e.g. gender, location, etc...) to make the distribution as balanced as possible.

 

I hope this helps!

Doug 

sas-innovate-2024.png

Don't miss out on SAS Innovate - Register now for the FREE Livestream!

Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.

 

Register now!

How to choose a machine learning algorithm

Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 1 reply
  • 662 views
  • 0 likes
  • 2 in conversation