BookmarkSubscribeRSS Feed
Loko
Barite | Level 11

Hello Experts,

 

We have an error when we run a datastep with hash tables. The table we try to load in hash is very big and probably the error is caused by lack of enough memory. What bothers is that the same datastep ran successfully on another environment but we can not determine what settings stop the process of running successfully everywhere.

 

The error is:

 

The SAS task name is [DATASTEP]
Segmentation Violation

Traceback of the Exception:

etc.

 

we use - SAS 9.3 on AIX

8 REPLIES 8
Kurt_Bremser
Super User

You need to work through everything pertinent. Physical memory available, OS settings, and the relevant SAS configuration sources (commandlines, metadata, config files) to find the difference between platforms. You must also consider all the processes running on a given environment, and their memory consumption.

Parallel to that I would investigate if the hash solution is necessary at all, or if you can solve the task with other means. Data/sort steps will run until disk storage is exhausted, which happens much later than a memory overflow.

Loko
Barite | Level 11

We have tried to configure the necessary parameters to the same value son both machines. We have even run the process alone on the environment where it caused problems and the result was the same (ERROR).

 

It seems that during the process running the memory system was not allocated. Any reason this might happen ?

 

Changing the datastep is the last solution because we need to send the results again to the people who validate them.

Loko
Barite | Level 11

we use - SAS 9.3 on AIX

Kurt_Bremser
Super User

Oh, fine. At least the system I'm quite familiar with.

Are both "environments" on the same host, or are these different (physical or logical) machines?

Are you using the same userID?

How is your SAS set up with regard to this particular problem? Are you running from an interactive frontend (Display Manager, Enterprise Guide, SAS Studio), or from batch? (This will point to the config files that you have to look at)

 

As a first step, run proc options in both environments and compare the memroy-relevant settings (MEMSIZE et al).

If you have different servers and/or different userIDs, run ulimit -a from the commandline with the userID used for SAS.

Loko
Barite | Level 11

We will do what you advised and come back .

 

Answering the questions:

There are different machines, we run from batch using the same user (shall have the same characteristics) .

 

WE have tried to set the same options on both machines (memsize - 0, BUFNO , IBUFNO etc.)

 

what do you mean by: How is your SAS set up with regard to this particular problem? 

 

Kurt_Bremser
Super User

Then you first must make sure that the different machines provide an approximately same environment. Have that checked by the system admins, or use smitty from the commandline and go to "System Environments - Change / Show Characteristics of Operating System".
Also run ulimit as suggested and compare the outputs.

 

If you run in batch, you either use the sasv9.cfg in the SAS installation directory, or one that you specifiy on the commandline. Those need to be compared.

With memsize=0, you might encounter your problem when the server runs out of physical memory and starts to page; if the paging spaces have different sizes, this might be the cause. If memory consumption gets really THAT large during your data step, I'd seriously consider a solution that does not rely on memory.

Loko
Barite | Level 11

Hi,

 

Thanks for the help . will follow your suggestions.

sas-innovate-2024.png

Available on demand!

Missed SAS Innovate Las Vegas? Watch all the action for free! View the keynotes, general sessions and 22 breakouts on demand.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 8 replies
  • 2271 views
  • 0 likes
  • 2 in conversation