What could be the reason for the delay in executing cron-Job in SAS 9.4 M6

[paparao28Ind]

Hi Admin Team,

               We have noticed that the SAS codes running on the SAS server - XYZ has been running very slow lately for last 3 weeks. There is one Sas code which used to run for around 4hours till 3 weeks back.  However, for last two weeks (till yesterday) it is taking around 8 hours to complete the same job.

user shared that :

1. network/bandwidth is good.

we could see:

2. Space is good to run.

can someone share your suggestions on it and do needful.

[carl_sommer · Accepted solution]

This is the sort of thing that requires knowing the baseline behavior, and then being able to measure from that.

Do you have a log of each, with the FULLSTIMER system option enabled?

With elapsed time changing, it's important to know if data volumes changed accordingly, or if an index has been dropped, or if data has moved to a slower network link, or... the list goes on.

If you don't have an earlier log to compare against, at least you could generate a log with the problem exhibited, and work your way through the steps with the longest elapsed times.   See how much CPU is being consumed vs the elapsed time; this will help you to make a determination if you are CPU or I/O constrained.   

Carl Sommer - SAS Technical Support

[ballardw]

When a program changes behavior, such as run time, where the files involved have not drastically increased in time my first place to check would be with the IT staff to see what may have been added or removed from the IT environment.

[JuanS_OCS]

Hi,

I agree with the last reply. In short either data increased which increases processing time or there is a non visibly bottleneck which adds wait times. Or there is a configuration which could be fine tuned (and changed until now, but that’s another type of discussion)

In my humble opinion and as you indicate resources are okay, I would start investigating the bottlenecks and wait times. It seems to me as SAS is not using your system fully.

Fullstimer will give you real time vs system time, which should give you wait times. In the procedures where you have wait times, you will find culprits.

For databases and such you have as well the sastrace option, which will tell you trace info about your data sources, a typical culprit for making SAS wait to receive information (eg bad join queries not optimized)

That is within the SAS area. You can also monitor things from the system or system plus SAS point of view.

From cron you could call SAS with the time command. This would add, at the process level, an output similar to fulltimer in SAS.

You could also use SAS Enterprise Session Monitor, which is an amazing tool to monitor and understand what happens in SAS, and you can compare with previous executions.

On one hand, you would need to know more about that/those jobs: are they connecting to a database, to a shared storage? Are the disks shared? How are the join queries be made? What type of operations are made? Stats or simple queries? Has data increased?

Of course you would need to know about your infrastructure and SAS configuration….

I would wonder as well if there was any recent update in SAS or your OS which could have changed any custom configuration by overwrite.

As you see, many areas of investigation. I’d start with finding out your wait times indeed. Once you find when and where wait happens, you will know what’s is the next step

[SASKiwi]

Do you use SAS Environment Manager? It would be a good starting point to check the dashboards for any bottlenecks regarding CPU, memory and IO.

[Kurt_Bremser]

Compare the logs from back then with the current logs, step by step.