12-03-2015 10:52 AM
I have a very large data set with over 17,000,000 records that I am trying to export to a bookmarked PDF. I would like to index and catalog that PDF so my users can do a search by loan for those records. Each time I have tried to export that via Enterprise Guide it runs for over 24 hours and then fails on a memory or not enough space error. I don't have an easy way to break it up into smaller files by loan number unless someone can make a suggestion on that as well.
Any ideas would be appreciated!
12-03-2015 11:02 AM
12-03-2015 11:55 AM
I had grouped it by Account Number and designated Enterprise Guide to start each group on a new page. When I tested this with the top 10 records, it seemed to work fine. The first account was grouped with its observations expanded below it. I can't really group it in SAS because each observation has the account number tied to it with data that is not unique.
Do you think it is better to add a program with SAS code to do this? Would you have an example you can show me?
12-03-2015 06:24 PM
Personally I don't think a single PDF will scale to 17M records.
I suggest you try experimenting with 1,000, 10,000, 100,000, 1M record PDFs first. Can you produce them? Can you open them and search for a loan in a reasonable timeframe?
I've found large ODS reports use a lot processing resources so I'm not surprised you are running into resource issues.
You really should consider a database-type solution for the volume of data you have.
12-03-2015 06:39 PM
You may also want to close the HTML (and/or LISTING) destination while doing such if you haven't already. Trying to build both the PDF and HTML tables could be using LOTS of resources.
12-03-2015 07:34 PM