BookmarkSubscribeRSS Feed
elwayfan446
Barite | Level 11

Good morning...

 

I have a very large data set with over 17,000,000 records that I am trying to export to a bookmarked PDF.  I would like to index and catalog that PDF so my users can do a search by loan for those records.  Each time I have tried to export that via Enterprise Guide it runs for over 24 hours and then fails on a memory or not enough space error.  I don't have an easy way to break it up into smaller files by loan number unless someone can make a suggestion on that as well.

 

Any ideas would be appreciated!

5 REPLIES 5
Reeza
Super User
Does your data have some ideal way to break it up? Ie some grouping system? There is a NEWFILE option in ODS PDF that you can use to separate the files automatically.

http://support.sas.com/documentation/cdl/en/odsug/61723/HTML/default/viewer.htm#a002217095.htm#a0029...
elwayfan446
Barite | Level 11

I had grouped it by Account Number and designated Enterprise Guide to start each group on a new page. When I tested this with the top 10 records, it seemed to work fine.  The first account was grouped with its observations expanded below it.  I can't really group it in SAS because each observation has the account number tied to it with data that is not unique.

 

Do you think it is better to add a program with SAS code to do this?  Would you have an example you can show me?

SASKiwi
PROC Star

Personally I don't think a single PDF will scale to 17M records.

 

I suggest you try experimenting with 1,000, 10,000, 100,000, 1M record PDFs first. Can you produce them? Can you open them and search for a loan in a reasonable timeframe?

 

I've found large ODS reports use a lot processing resources so I'm not surprised you are running into resource issues.

 

You really should consider a database-type solution for the volume of data you have.

ballardw
Super User

You may also want to close the HTML (and/or LISTING) destination while doing such if you haven't already. Trying to build both the PDF and HTML tables could be using LOTS of resources.

elwayfan446
Barite | Level 11
Thank you @SASKiwi and @ballardw...

@SASKiwi... I have not tried to break them down yet. I was trying to figure a way to easily split the datasets and then print individual pdfs so I am still looking at that.

@ballardw, I did ensure all output was off except for PDF but that was a great idea. The first few times I tested I realized I had that on.

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

SAS Enterprise Guide vs. SAS Studio

What’s the difference between SAS Enterprise Guide and SAS Studio? How are they similar? Just ask SAS’ Danny Modlin.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 5 replies
  • 1622 views
  • 0 likes
  • 4 in conversation