BookmarkSubscribeRSS Feed
sht
Fluorite | Level 6 sht
Fluorite | Level 6

Hi Team,

 

I am facing issues in Dataflux job which are developed for cluster id generation and scheduled thru Windows scheduler.

If I execute those jobs manually then it executes fine(around hrs) but in windows scheduler jobs are taking long time(around 8-9 hrs )

 

Please suggest what can be the issue

OS: Windows8

DF Studio: 8.1

Data size: More than 40lacs record

 

Thank You

3 REPLIES 3
RonAgresta
SAS Employee

Hi - are you calling the dmpexec command from your scheduler? There are various options you can set when invoking the command that may impact performance, like choosing to write a log or selecting options that overwrite configuration file settings. There are also options for logging that might help you pinpoint the issue.

 

Another thought - I have seen in the past where dmpexec was executed by a user (different than the one used with Data Management Studio) that had different environment settings associated with it (like where the "temp" directory was located) and this impacted performance. 


Ron

sht
Fluorite | Level 6 sht
Fluorite | Level 6

Hi Ron,

 

Thanks for your reply..

 

I don't execute any dmpexec command.

These schedulers were usually executing smoothly but now it is taking longer time to execute than usual.

previously it get executed in 3hrs now it is taking 7-9 hrs.

 

No.of rows are increased hardly by 2-3 lacs

 

 

Thank you.

 

RonAgresta
SAS Employee

Review the product documentation. There are topics that specifically deal with how you should be calling your DMP jobs if you are not running them interactively.

 

Regardless, check a few things:

  • Make sure that each call by your scheduler to run the job didn't spawn a new process that is taking up system resources
  • If your job is accessing data, make sure that your queries are running as you would expect. You can turn on more verbose levels of logging to see database interaction or you can monitor that in your database log.
  • There are logging options you can set that will generate what are called node profile metrics. These logs will tell you how long each node is processing rather than the sum for the entire job. This is also in the documentation.

Ron

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to connect to databases in SAS Viya

Need to connect to databases in SAS Viya? SAS’ David Ghan shows you two methods – via SAS/ACCESS LIBNAME and SAS Data Connector SASLIBS – in this video.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 3 replies
  • 1250 views
  • 0 likes
  • 2 in conversation