Help using Base SAS procedures

SPDE indexing

Reply
N/A
Posts: 1

SPDE indexing

i am new to the SPD engine.

the following is test code i am running to try and get a feel for how SPDE works:

libname tst SPDE "c:/test/" partsize=1G;

data tst.test1;
do i=1 to 4000000;
output;
end;
run;

proc datasets library=tst;
modify test1(asyncindex=yes);
index create i;
run;

data tst.test2;
do i=1 to 2000000;
output;
end;
run;

proc append base=tst.test1 data=tst.test2;
run;

the first data step produces a file with size 24m
the proc datasets produces an hbx file with size of 133m
the second data step produces a file with size of 11m
the proc append increases the size of the first dataset and also creates an idx file with size of 140m

i am using SPDE to deal with large datasets in the first place and now i'm finding that the hbx/idx files combined are going to require 7 or 8 times the size of my original data?
can someone explain where i am going wrong or if there is a way around these large hbx/idx files?

thank you in advance.
Ask a Question
Discussion stats
  • 0 replies
  • 36 views
  • 0 likes
  • 1 in conversation