BIG DATA - PROC COMPARE

gra_in_aus — Wed, 10 Feb 2016 23:13:37 GMT

Just wondering if there is anyway of doing a proc compare (SAS 9.3) for very large datasets > 144m obs between data that is held on DB2 and SAS SPDS via the SAS/ACCESS DB2 Engine? Or is there a specific SAS product or other technique that could be utilised.

Thanks

Re: BIG DATA - PROC COMPARE

SASKiwi — Thu, 11 Feb 2016 02:40:13 GMT

Its my understanding that the COMPARE procedure only runs in the SAS environment. What that means is when you reference a DB2 table with COMPARE the entire table (or subset if chosen) will be copied into SAS so that the comparison can happen.

I've used COMPARE with multi-million row tables easily without any problems but have not tried > 144m rows. In theory as long as you have the resources it should work OK. I suggest you give COMPARE a go first before trying anything else. Its a great tool that is not used as widely as it could be.

topic BIG DATA - PROC COMPARE in SAS Procedures

BIG DATA - PROC COMPARE

Re: BIG DATA - PROC COMPARE