Help using Base SAS procedures

BIG DATA - PROC COMPARE

Reply
Contributor
Posts: 41

BIG DATA - PROC COMPARE

Just wondering if there is anyway of doing a proc compare (SAS 9.3) for very large datasets > 144m obs between data that is held on DB2 and SAS SPDS via the SAS/ACCESS DB2 Engine? Or is there a specific SAS product or other technique that could be utilised.

 

Thanks

 

G.

Super User
Posts: 3,260

Re: BIG DATA - PROC COMPARE

Posted in reply to gra_in_aus

Its my understanding that the COMPARE procedure only runs in the SAS environment. What that means is when you reference a DB2 table with COMPARE the entire table (or subset if chosen) will be copied into SAS so that the comparison can happen.

 

I've used COMPARE with multi-million row tables easily without any problems but have not tried > 144m rows. In theory as long as you have the resources it should work OK. I suggest you give COMPARE a go first before trying anything else. Its a great tool that is not used as widely as it could be.

Ask a Question
Discussion stats
  • 1 reply
  • 261 views
  • 0 likes
  • 2 in conversation