Hi, I have a SAS dataset with limited number of columns (~15). I usually and frequently have the need to search the relevant records from this dataset using Index() function. However due to size of dataset (>7gb, >25million records) it is taking very long. I created SAS Index on individual variable. Of course Index reduces time to retrieve records but it only works with an exact match in where clause, which is not what I need 99% of the time. 99% of the time I have to do keyword search (search not by full string value but part of string value, like _VAR Contains "searchtext") using Index function and in this case, as seen below, SAS doesn't use the available Index on the same variable. Can someone please help, point to relevant threads, material: I need to search (Index() or Find()) very fast and it seems SAS Index so far not helping. 26 data j__; 27 set adhc.codetrak (where=(program_name='t_hier_testing_diab_cv1')); INFO: Index program_name selected for WHERE clause optimization. 28 run; NOTE: There were 6404 observations read from the data set ADHC.CODETRAK. WHERE program_name='t_hier_testing_diab_cv1'; NOTE: The data set WORK.J__ has 6404 observations and 16 variables. NOTE: DATA statement used (Total process time): real time 0.16 seconds cpu time 0.03 seconds 27 data j__; 28 set adhc.codetrak (where=(index(lowcase(strip(program_name)), 't_hier_testing_diab_cv1'))); 29 run; NOTE: There were 6404 observations read from the data set ADHC.CODETRAK. WHERE INDEX(LOWCASE(STRIP(program_name)), 't_hier_testing_diab_cv1'); NOTE: The data set WORK.J__ has 6404 observations and 16 variables. NOTE: DATA statement used (Total process time): real time 2:13.92 cpu time 43.62 seconds
... View more