topic Re: optimized code for cartesian product in SAS Programming

optimized code for cartesian product

Sathya3 — Thu, 11 May 2023 13:18:53 GMT

What is better way of doing this which takes less processing time? My actual data is huge and since below code produces cartesian product it is taking very large amount of time for processing.Below is sampel data

data one ;

datalines;

id name $;

100 sam

200 ram

;

run;

data two ;

datalines;

id name $;

100 sam

200 ram

300 paul

;

run;

Proc sql;

create table temp as

select b.id,b.name from

one a,

two b

where a.id NE b.id;

quit;

Re: optimized code for cartesian product

PaigeMiller — Thu, 11 May 2023 13:55:55 GMT

A very recent and long thread on Cartesian joins

https://communities.sas.com/t5/SAS-Programming/NOTE-The-execution-of-this-query-involves-performing-one-or-more/m-p/873367

Basically, if it has to be a Cartesian join, you can't optimize it and you can't speed it up. So the real question is ... can this be done without a Cartesian join? Please let us know.

Re: optimized code for cartesian product

LinusH — Thu, 11 May 2023 15:09:01 GMT

Are both tables huge?

If not, the smaller table may fit into memory and then you could use the hash object in the datastep/proc ds2.

Re: optimized code for cartesian product

PGStats — Thu, 11 May 2023 17:10:09 GMT

What operation are you trying to perform (in words).