11-05-2015 02:33 PM
I have reported few issues with EG while Executing the EG Projects. let me explain example of code.
Users are having EG Project which is combo of multiple SAS Programes.
Programe1 ---------------> Program2 ---->Program3 ----> Program4--->Program5------>Program6....so on.
(Lib Ref) daa step data step data steps data steps data steps
Actual happening: the Project is for Analytics to creates some target tables and takes 4-5 hours to complte it run, users running over night and going home to see result on next day morning.
Issue 1:- the Eg Project froze and EG unresponsice state on nest day morning. or
Issue 2:- when User run the above Project, Program1 assign library, and Program2 and Program3 creates few temp Tables into Work Library.
when SAS start execute Program4 or 5, it throughs error with the table names as not exists(which has created as part of Program2 or 3).
Issue 3:- sometimes it says that the libname statement not assigned though they are in part of Program1.
Issue 4:- If the User stay with PC and complete the EG Project Run and try to open the permenent table from Project, not opening. its says the table doesnot exits. but the same table is opens from file location Server> SASApp>Files> table.
Are this happening cause any issue with Server responce? is there any options to change in EG side to interactivley complete the EG Project run for 5 hours without Froze?
Can someone suggest on it to how to fix this issues?
11-05-2015 03:58 PM
I would check in the Projet properties, in code submission if allow parallel execution is checked.
After that I would check if every program are linked together properly.
This could resolve issue 2,3 and possibly 4. Issue 1 however, I don't know what could cause it.
11-06-2015 11:35 AM
Thank you for your reply, how can i check if parallal execution checked or not? what is the possible option should user check that option or not?
how to test if the every program linked proparly or not? is there any utility tool to test it?
11-06-2015 01:05 PM
If parallel processing is check all the program start simultaneous, so if one allocate the librairies, the other can't see those since they are running at the same time.
AS for the process flow with a bunch of programs, I always link the program in the order that they should run.
You have to open the project and go on then file menu-> properties -> code submission:
And to check if the program are link you look at the process flow:
11-06-2015 06:43 PM
Running regular, large, long jobs unattended in EG is inherently unreliable. Things like network problems, timeouts on EG connections, log and output size can easily cause the symptoms you are seeing.
I suggest you look at converting these to SAS batch jobs that are run and scheduled on your SAS server. Your jobs should run a lot more reliably this way.
11-06-2015 10:38 PM - edited 11-06-2015 10:57 PM
I'm very much with @SASKiwi on this one: In order to "productionize" EG projects you need to export the code and run it in batch on the server.
Depending on your environment and user priviledges this is still something which could be set-up and maintained by an end-user group (so it's not real IT like production grade and end-users could still have full control over the code with short change cycles).
What you describe is something I've seen happening already multiple times. It's kind-of the normal process.
First "business" users get EG and they use it for simple point and click stuff, then they get more experienced and the EG projects get bigger and more complex, and the business processes supported by these EG projects become more and more.
At some point the problem starts with missing "audit trails", reports can't get replicated as everybody uses a bit different data preparation steps for the same.
That's when such user groups start to create data marts as common source of truth for the whole group. But then more and more data gets added and some sort of organicly grown unplanned data warehouse environment gets established. The users start to spend more and more time maintaining a "messy" environment instead of spending this time to answer business questions.
I feel the situation you describe is for a group which is reaching the point where "things get out of hand". So besides of resolving the immediate issue I feel it's the time for some re-design and "process re-engineering" so that these users can spend their time with what they should be doing.