Desktop productivity for business analysts and programmers

Exporting large data set to bookmarked PDF

Reply
Frequent Contributor
Posts: 138

Exporting large data set to bookmarked PDF

Good morning...

 

I have a very large data set with over 17,000,000 records that I am trying to export to a bookmarked PDF.  I would like to index and catalog that PDF so my users can do a search by loan for those records.  Each time I have tried to export that via Enterprise Guide it runs for over 24 hours and then fails on a memory or not enough space error.  I don't have an easy way to break it up into smaller files by loan number unless someone can make a suggestion on that as well.

 

Any ideas would be appreciated!

Grand Advisor
Posts: 17,313

Re: Exporting large data set to bookmarked PDF

Does your data have some ideal way to break it up? Ie some grouping system? There is a NEWFILE option in ODS PDF that you can use to separate the files automatically.

http://support.sas.com/documentation/cdl/en/odsug/61723/HTML/default/viewer.htm#a002217095.htm#a0029...
Frequent Contributor
Posts: 138

Re: Exporting large data set to bookmarked PDF

I had grouped it by Account Number and designated Enterprise Guide to start each group on a new page. When I tested this with the top 10 records, it seemed to work fine.  The first account was grouped with its observations expanded below it.  I can't really group it in SAS because each observation has the account number tied to it with data that is not unique.

 

Do you think it is better to add a program with SAS code to do this?  Would you have an example you can show me?

Respected Advisor
Posts: 3,060

Re: Exporting large data set to bookmarked PDF

Personally I don't think a single PDF will scale to 17M records.

 

I suggest you try experimenting with 1,000, 10,000, 100,000, 1M record PDFs first. Can you produce them? Can you open them and search for a loan in a reasonable timeframe?

 

I've found large ODS reports use a lot processing resources so I'm not surprised you are running into resource issues.

 

You really should consider a database-type solution for the volume of data you have.

Grand Advisor
Posts: 10,196

Re: Exporting large data set to bookmarked PDF

You may also want to close the HTML (and/or LISTING) destination while doing such if you haven't already. Trying to build both the PDF and HTML tables could be using LOTS of resources.

Frequent Contributor
Posts: 138

Re: Exporting large data set to bookmarked PDF

Thank you @SASKiwi and @ballardw...

@SASKiwi... I have not tried to break them down yet. I was trying to figure a way to easily split the datasets and then print individual pdfs so I am still looking at that.

@ballardw, I did ensure all output was off except for PDF but that was a great idea. The first few times I tested I realized I had that on.
Ask a Question
Discussion stats
  • 5 replies
  • 538 views
  • 0 likes
  • 4 in conversation