DATA Step, Macro, Functions and more

create design matrix from long dataset

Accepted Solution Solved
Reply
New Contributor
Posts: 3
Accepted Solution

create design matrix from long dataset

I have a long dataset like the example below (in my real dataset I have thousands of ID's and approx one thousand distinct Dx values).  

ID         Dx

Pat1     A

Pat1     B

Pat1     C

Pat2     B

Pat2     D

Pat3     A

Pat3     D

 

And I want to generate a design matrix like the one below (obviously the real design matrix will be high dimensional with a column for each Dx value).   

 

ID           A      B     C     D

Pat1       1       1     1      0

Pat2       0       1      0     1  

Pat3       1       0      0     1

 

 

My original plan was to transpose the long dataset, and then let proc glm generate the design matrix using a class statement; however I can't get the data to transpose how proc glm would need it with distinct dx values in each dx column.  

 

Using transpose I haven't been able to get anything better than the structure below. 

 

ID          Dx1      Dx2     Dx3

Pat1       A           B         C

Pat2       B           D

Pat3       A

 

I've thought about approaches using loops, arrays, and proc sql, but I haven't been able to put the pieces together to get to the end design matrix I need.  

 

Any help is appreciated, and while any answer that works is great, in my situation a proc sql solution would be optimal.  

 

Thanks much!


Accepted Solutions
Solution
‎09-21-2017 09:55 AM
Super User
Posts: 5,518

Re: create design matrix from long dataset

[ Edited ]
Posted in reply to bootstrap_armada

PROC TRANSPOSE can handle this.  You just need to use this statement:

 

id Dx;

 

Since Dx is no longer going to be in the VAR statement, you may need to take a couple of additional steps (try it and see if they are needed):

 

Before transposing, create a variable that is 1 on each observation, and use that in the VAR statement of PROC TRANSPOSE.

 

After the transpose go back and replace missing values with zeros.

 

One final consideration ... your incoming data set may may be to deduplicated.  You can't have multiple observations for the same ID with the same DX.

View solution in original post


All Replies
Solution
‎09-21-2017 09:55 AM
Super User
Posts: 5,518

Re: create design matrix from long dataset

[ Edited ]
Posted in reply to bootstrap_armada

PROC TRANSPOSE can handle this.  You just need to use this statement:

 

id Dx;

 

Since Dx is no longer going to be in the VAR statement, you may need to take a couple of additional steps (try it and see if they are needed):

 

Before transposing, create a variable that is 1 on each observation, and use that in the VAR statement of PROC TRANSPOSE.

 

After the transpose go back and replace missing values with zeros.

 

One final consideration ... your incoming data set may may be to deduplicated.  You can't have multiple observations for the same ID with the same DX.

New Contributor
Posts: 3

Re: create design matrix from long dataset

Posted in reply to Astounding

This is a good, simple solution - I hadn't thought of that approach with proc transpose.  Thanks!

Trusted Advisor
Posts: 1,231

Re: create design matrix from long dataset

Posted in reply to bootstrap_armada

Hi,

 

Or if just need to generate a matrix.

 

proc tabulate data=have;
class id dx;
table id,dx=""*N="";
run;

☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 3 replies
  • 82 views
  • 2 likes
  • 3 in conversation