topic Re: Subsetting a SAS dataset and saving it as an RDS with proc iml in SAS/IML Software and Matrix Computations

Subsetting a SAS dataset and saving it as an RDS with proc iml

Coreyw184 — Mon, 03 Feb 2025 22:27:32 GMT

I am struggling to understand the proper way to subset a dataset for use in proc iml. My goal is to take a large sas dataset (6,000,000x1,500) and save it as multiple smaller RDS datasets using proc iml based on a grouping variable. My issue is that submit cannot be used in a macro and the use and read statements only keep the numeric variables. Is there a way to keep all variables, numeric and character in the dataset? Below is a small example of the code I have at the moment.

%let names = J K L;

data heights_weights;

input Name $ Height Weight second_group $;

datalines;

J 70 180 D

J 65 140 D

K 68 150 S

K 72 200 D

K 66 130 S

L 74 210 S

L 64 135 S

L 69 175 D

J 67 145 D

K 71 190 D

L 63 125 D

J 73 205 D

L 62 120 D

K 75 220 D

K 70 160 D

;

run;

proc iml;

do i=1 to countw("&names");

letter_name = scan("&names",i);

use heights_weights where(Name = letter_name);

read all into nhw;

close heights_weights;

call exportmatrixtor(nhw, "nhw");

submit nhw letter_name/R;

print(paste("dataset","&letter_name"))

print(nhw)

# saveRDS(nhw, paste("nhw","&letter_name",".rds"))

endsubmit;

end;

quit;

Re: Subsetting a SAS dataset and saving it as an RDS with proc iml

Ksharp — Tue, 04 Feb 2025 01:53:51 GMT


data heights_weights;
    input Name $ Height Weight second_group $;
    datalines;
J 70 180 D
J 65 140 D
K 68 150 S
K 72 200 D
K 66 130 S
L 74 210 S
L 64 135 S
L 69 175 D
J 67 145 D
K 71 190 D
L 63 125 D
J 73 205 D
L 62 120 D
K 75 220 D
K 70 160 D
;
run;
 
proc iml;
use heights_weights;
read all var {Name} ;
close;
levels=unique(name); 
do i=1 to ncol(levels);
  use heights_weights;
  read all var _num_ where(name=(levels[i])) into num[c=vname1] ;
  read all var _char_ where(name=(levels[i])) into char[c=vname2] ;
  close;
  create (levels[i]) from num char[c=(vname1||vname2)];
  append from num char;
  close;
/*call exportmatrixtor(nhw, "nhw");
submit nhw letter_name/R;
...................
endsubmit
*/
end;
quit;

proc print data=J noobs;run;
proc print data=K noobs;run;
proc print data=L noobs;run;

Re: Subsetting a SAS dataset and saving it as an RDS with proc iml

Ksharp — Tue, 04 Feb 2025 02:08:45 GMT

You could try new data structure of IML:
https://documentation.sas.com/doc/en/imlcdc/14.2/imlug/imlug_tables_toc.htm
https://blogs.sas.com/content/iml/2017/03/29/lists-sasiml.html
https://blogs.sas.com/content/iml/2018/01/22/iml-lists-syntax.html
https://blogs.sas.com/content/iml/2017/03/22/data-tables-in-sasiml.html