topic Re: Question about Proc sql in merging two big dataset in SAS Programming

Question about Proc sql in merging two big dataset

Seashore — Mon, 29 Jun 2015 01:45:06 GMT

I have a data a:

with a lot variables a b c d e...etc. for example, I want to combine with dataset b with variable named "X". but one row of b might have around 30,000 matched "X" in a.

my SAS code is:

proc sql noprint;

create table ds as

select a.*, b.momstatefips

from a left join b

on a.momstatefips1=b.momstatefips;

quit;

Is this code right? It always takes a long time to run this code and give me the note that "out of resource". Does that make sense???

Re: Question about Proc sql in merging two big dataset

Tom — Mon, 29 Jun 2015 02:50:40 GMT

What are you actually trying to do? Your current step is not adding any information since you are only selecting that variable that already matches the existing variable.

If you are doing a 1 to Many merge then use data step with a merge statement instead.

Below is the data step equivalent of your left join, plus I added the variable FOUND to indicate if there was a match.

Note that it will only work if your data is sorted.

data DS ;

merge MOMSTATEFIPS (in=in1 rename=(MOMSTATEFIPS1 = MOMSTATEFIPS)) B (keep=MOMSTATEFIPS in=in2) ;

by MOMSTATEFIPS;

if in1 ;

FOUND= in2;

run;

Re: Question about Proc sql in merging two big dataset

PGStats — Mon, 29 Jun 2015 03:17:11 GMT

Maybe you are trying to subset dataset a to extract the rows that are mentioned in dataset b? If so, use

proc sql;

create table ds as

select a.*

from a inner join b

on a.momstatefips1 = b.momstatefips;

quit;

Re: Question about Proc sql in merging two big dataset

AmitRathore — Mon, 29 Jun 2015 10:54:59 GMT

Hi Experts,

Using an index in table momstatefips1 is one of options to reduce the processing time.

An index can reduce the time required to locate a set of rows, especially for a large data file.

proc sql noprint;

create index x on work.momstatefips1(x);

create table ds as

select a.*

from a inner join b

on a.momstatefips1 = b.momstatefips;

quit;

Br, Amit