SAS Scalable Performance Data Server provides an engine to handle large data As it is not always available, you may find the smaller brother SPDE (a SAS library engine) helpful. SPDE achieves performance in several ways. I think the main 2 are partitioning and index optimisation. It is just great the way multi-gigabyte tables perform when partitioned and indexed well. Use the system option MSGLEVEL=i to see which indexes are used. the great thing about the (big brother) SPDS is that it is a separate server - increasing the capacity of the service to solve your query, and it provides further index handling optimisation. It is a bit like a database server optimised for SAS queries. Good luck peterC ... View more

I am not sure if this is what you are looking for, but there are several papers by LeRoy Bessler that address similar problems using DDE. Consider: http://www2.sas.com/proceedings/sugi31/022-31.pdf www.mwsug.org/proceedings/2010/excel_db/MWSUG-2010-166.pdf Message was edited by: ArtC ... View more

Many thanks Cynthia, i will apply these techniques and let you know how it goes. ... View more

Many thanks Linus and sbb, i have taken all the points into consideration, i am currently putting a document together and might be back for more details but thanks for the info supplied so far ... View more