I have 2 datasets with each 500 MB of size approx...when im left joining them the final dataset is coming around 7GB Can any one tell me the reason and how can i reduce that.Both of the input datasets are not sorted or donot have index. and i cant do a proc sort on them.and they contain duplicates values also...as they contain duplicate values i cant even create index on key column
Thanks & Regards
OK, having described the input side of your data model, it is understandable that proc sql might have some trouble. You can help proc sql to control that explosion of data with a variety of conditions in the join and in where clause(s). If these are unneccessary, you may have to accept that the output side of your problem will be a 7GB file.
When you offer code, it might become possible to help you more.