SAS Data Integration Studio, DataFlux Data Management Studio, SAS/ACCESS, SAS Data Loader for Hadoop and others

Question about Clustering in Data Management Studio 2.4

Accepted Solution Solved
Reply
Occasional Contributor
Posts: 5
Accepted Solution

Question about Clustering in Data Management Studio 2.4

Hi all,

I am fairly new to the advanced functionality of DMStudio and I am trying to set up a new data job that requires clustering of data and surviving record identification.

My input looks something like the below table.

e1a
e1a
e1a
e2b
e2b
e3c
e3c
e3c

I would like to have as output only the records that are repeated 3 times

e1a
e3c

Any ideas about what I could use to do so?

Thank you in advance,

Cristina


Accepted Solutions
Solution
‎03-17-2015 07:07 AM
SAS Employee
Posts: 85

Re: Question about Clustering in Data Management Studio 2.4

Posted in reply to cristinadumitru

Cristina,

If you are querying a database directly, SQL can do this for you. DM Studio you can create an expression to help with this. Once you cluster the records, add an expression node which will return only clusters containing 3 records:

Job:

cluster_dm_job.png

Clustering:

dm_step_2.png

Desired Output:

dm_step_3.png

Expression configuration (Grouping):

dm_step_4.png

Expression configuration (Expression Logic):

dm_step_5.png

-shawn

View solution in original post


All Replies
Solution
‎03-17-2015 07:07 AM
SAS Employee
Posts: 85

Re: Question about Clustering in Data Management Studio 2.4

Posted in reply to cristinadumitru

Cristina,

If you are querying a database directly, SQL can do this for you. DM Studio you can create an expression to help with this. Once you cluster the records, add an expression node which will return only clusters containing 3 records:

Job:

cluster_dm_job.png

Clustering:

dm_step_2.png

Desired Output:

dm_step_3.png

Expression configuration (Grouping):

dm_step_4.png

Expression configuration (Expression Logic):

dm_step_5.png

-shawn

Occasional Contributor
Posts: 5

Re: Question about Clustering in Data Management Studio 2.4

Hi Shawn,

Have you added any additional conditions?

My output is just:

scr.jpg

Thanks again,

Cristina

SAS Employee
Posts: 85

Re: Question about Clustering in Data Management Studio 2.4

Posted in reply to cristinadumitru

When you are clustering, are you sorting by the cluster id? Does your cluster output resemble my second screenshot? Also, make sure in the expression, that you have indicated that you are grouping by the sorted cluster id.

Occasional Contributor
Posts: 5

Re: Question about Clustering in Data Management Studio 2.4

My fault, I didn't see the grouping by cluster_id - never used the functionality before.

Thank you very much, it all works great now.

Best wishes,

Cristina

🔒 This topic is solved and locked.

Need further help from the community? Please ask a new question.

Discussion stats
  • 4 replies
  • 407 views
  • 4 likes
  • 2 in conversation