Question about Proc sql in merging two big dataset

[Seashore]

I have a data a:

with a lot variables a b c d e...etc. for example, I want to combine with dataset b with variable named "X". but one row of b might have around 30,000 matched "X" in a.

my SAS code is:

proc sql noprint;

create table ds as

select a.*, b.momstatefips

from a left join b

on a.momstatefips1=b.momstatefips;

quit;

Is this code right? It always takes a long time to run this code and give me the note that "out of resource". Does that make sense???

[Tom]

What are you actually trying to do?  Your current step is not adding any information since you are only selecting that variable that already matches the existing variable.

If you are doing a 1 to Many merge then use data step with a merge statement instead.

Below is the data step equivalent of your left join, plus I added the variable FOUND to indicate if there was a match. 

Note that it will only work if your data is sorted.

data DS ;

  merge MOMSTATEFIPS (in=in1 rename=(MOMSTATEFIPS1 =  MOMSTATEFIPS)) B (keep=MOMSTATEFIPS in=in2) ;

  by MOMSTATEFIPS;

  if in1 ;

  FOUND= in2;

run;

[PGStats · Accepted solution]

Maybe you are trying to subset dataset a to extract the rows that are mentioned in dataset b? If so, use

proc sql;

create table ds as

select a.*

from a inner join b

on a.momstatefips1 = b.momstatefips;

quit;

PG

[AmitRathore]

Hi Experts,

Using an index in table momstatefips1 is one of options to reduce the processing time.

An index can reduce the time required to locate a set of rows, especially for a large data file.

proc sql noprint;

  create index x on work.momstatefips1(x);

  create table ds as

  select a.*

  from a inner join b

  on a.momstatefips1 = b.momstatefips;

quit;

Br, Amit