Add BATCH SUBMIT button to EG

[Quentin]

I just came from an EXCELLENT talk on Enterprise Guide for Programmers at Boston Area SAS Users Group, by Chris Bost.

The talk is available at: http://www.basug.org/downloads/2013q2/Bost.pdf

Not surprsingly in a room full of programmers, during the discussion the question came up, "can I do a batch submit?"

And I think currently, the correct answer is "no, you can't, because EG is an interactive environment, in the same (similar) way that DM SAS is an interactive environment."

Chris B. helpfully pointed out that you can schedule an EG project, which accomplishes something like a batch session.

But here's the thing.  When I programming in SAS, I was taught that I should ALWAYS submit my final code as a batch job.  There are two main benefits.  1. You ensure that your code works on it's own, and isn't working (or failing) because you have a lot of temporary datasets / formats / macro variables / whatever assembled junk sitting in your work library.  2.  You get a log file.

During discussion it was suggested by someone that might be nice if EG just had a "batch submit button."  I think it's a grand idea.  If I press "Batch Submit", I want EG to spawn a new SAS session on the server, and execute my program in that new session, then return the results as usual. Getting a log file would also be nice, but that is a separable issue (and PROC PRINTTO is available as a workaround).

Now when using EG, it seems like there are approximations of ways to ensure that you have a clean session to run your code in:

1.  Schedule the flow.

2.  Write a macro that will try to clean up your session (delete temp datasets/filerefs/librefs/macro vars/etc etc)

3.  Disconnect from the server and then reconnect.  (I feel like EG is unhappy with me when I do this, maybe it's because my code is linked instead of embedded in the project, I think EG loses track of the links when I disconnect)

4.  Close EG and restart it.

To me, those are all work arounds for not having a batch submit  button.  Sounded like a grand idea to me, so wanted to throw it out to the group.

--Q.

(edited 12/10/13 to add link to slides from presentation, and make clear that "Chris" is Chris Bost, not Chris H.)

[David]

If it helps qualify why so many people want a batch SAS sessions: Some industries and sometimes just within organisations have to demonstrate the entire process used to derive a result. A batch SAS session is encapsulated, hence there can be no data sets in the work library or macro variables that were created by other code, which are then inadvertently used by an important program. The log generated by a batch session audits the process from invocation to termination. That is a powerful audit trail.

I gather SAS Studio is gaining a further perspective, allowing programs to be submitted in batch (maybe later this year?).

[hixsone]

We have used a VB script kicked off daily by Windows scheduler to automate several EG projects.

Option Explicit
Dim app ' As SASEGuide.Application
Call dowork
'shut down the app
If not (app Is Nothing) Then
app.Quit
Set app = Nothing
End If

Sub dowork()
On Error Resume Next
'----
' Start up Enterprise Guide using the project name
'----
Dim prjName ' As String
Dim prjObject ' As SASEGuide.Project
Dim containerName ' As String
Dim containerObject ' As SASEGuide.Container
Dim containerColl ' As SASEGuide.ContainerCollection
prjName = "C:\Users\Public\SAS Jobs\New Prod\Service Line Forecast\SAS_ETS_MacroCode_All_Institutes_DynamicProgramming_v5 - AllModels.egp" ' Project Name
containerName = "Process Flow" ' Container Name

Set app = CreateObject("SASEGObjectModel.Application.6.1")
If Checkerror("CreateObject") = True Then
Exit Sub
End If

Set prjObject = app.Open(prjName,"")
If Checkerror("App.Open") = True Then
Exit Sub
End If

'-----
'Get The Container Collection and Object
'----- 
Set containerColl = prjObject.ContainerCollection
If Checkerror("Project.ContainerCollection") = True Then
Exit Sub
End If

Dim i ' As Long
Dim count ' As Long
count = containerColl.count
For i = 0 To count - 1
Set containerObject = containerColl.Item(i)
If Checkerror("ContainerCollection.Item") = True Then
Exit Sub
End If

If (containerObject.Name = containerName) Then
Exit For
Else
Set containerObject = Nothing
End If
Next 

If not (containerObject Is Nothing) Then
'----
' Run the Container
'----
containerObject.Run
If Checkerror("Container.Run") = True Then
Exit Sub
End If 
End If

'-----
' Save the new project
'-----
prjObject.Save
If Checkerror("Project.Save") = True Then
Exit Sub
End If

'-----
' Close the project
'-----
prjObject.Close
If Checkerror("Project.Close") = True Then
Exit Sub
End If

End Sub
Function Checkerror(fnName)
Checkerror = False

Dim strmsg ' As String
Dim errNum ' As Long

If Err.Number <> 0 Then
strmsg = "Error #" & Hex(Err.Number) & vbCrLf & "In Function " & fnName & vbCrLf & Err.Description
'MsgBox strmsg 'Uncomment this line if you want to be notified via MessageBox of Errors in the script.
Checkerror = True
End If

End Function

[Quentin]

Per this blog post, SAS Studio has a Background Job Submit (formerly "Batch Submit) feature: http://blogs.sas.com/content/sgf/2017/01/30/sas-studio-3-6-new-features/?utm_source=feedburner&utm_m...

Has anyone played with it?  Do you like enough that I should update this idea submission to reference the SAS Studio feature?

[samdupont]

@Quentin  It would be great to get any feedback on what is lacking in the existing background submit functionality in the current SAS Studio. It is important feedback not just for Studio but also for EG as our long-term vision is to bring the two products in sync, where it makes sense.

[Quentin]

Thanks @samdupont . I haven't used SAS studio much myself.  I will try to givve it a try.  Hopefully someone else can chime in with there thoughts / experiences. 

[Casey_SAS]

We think Batch Submit for Enterprise Guide is a good idea as well - I've added this as a feature request, stay tuned!

[ScottBass]

I'm late to the party, having only just come across this post via search.

Some thoughts:

* While I *can* run an EG project via VB script or Powershell (and have done so)...I don't want to.  I'm a SAS programmer, I want to do things the "SAS way" (so to speak).  Another way of saying this is, I'd like "batch mode EG" to be like SAS DMS where possible.

* I agree 1000% with @PhilC.  Since day dot, I’ve wished I could do something like seguide.exe projectname.egp to run EG in batch.

* My options would be something like:

seguide.exe

   [-project <path to project file.egp> {-project is optional if project file is first parameter}]

   [-profile <path to profile file> | -server -port -user -password]

   [-processflow <processflow1 [, ] processflow2 [, ] processflow3, …]

   [-program (or “entry”) program1 [, ] program2, [, ] program3, …]

   [-log {logparms to create consolidated or separate project/process flow/program object logs}]

I’ve probably forgotten a few parameters, but the gist is I run the EG executable, supply a project file, EG launches a workspace server, submits the project/process flows/program entries/etc. in batch mode, and creates logs as needed for auditing.  Optional parameters limit which items are submitted.

* In this case, EG is a container or “package” containing my code, the connection details to the remote server, and the internal code to “make it all work”.  I should be able to run my code stored internally in EG without forcing me to save it as external files.

* I have *often* worked in environments where I wanted to run SAS in batch, but did not have RDP access to the SAS server on which to launch batch SAS sessions.  EG nicely encapsulates the process needed to run my SAS code across the network.

* Here is an interesting Sourceforge project:  https://sourceforge.net/projects/cmdlets4sas/.  I’ve had a play with these in the past and they worked ok.  But 1) they’re not “official” – it would be much better if SAS provided “EG in batch” out of the box (not VB script or Powershell), and 2) you still have to write some Powershell code to call the cmdlets.  What about the brilliant SAS programmer who has no interest in Powershell (or VB script) and isn’t going to learn it?  Shouldn’t he/she have an easy way to run EG in batch?

[Amarnath7]

Interesting topic. Please consider this feature to have it in BASE SAS as well. 

Recently I have written very very simple macro but took my significant time to establish a link between current SAS session and batch run process. Does batch run of current program in SAS window (either one macro call/tool bar button), automatic log check and shows the log issues in special window in SAS session. So that you can fix your program immediately if there is any issue.  Also, it opens the log file as text file when there is an issue found. 

I am submitting this as abstract in SAS Global Forum. I will share the macro soon. 

I would request SAS to add this feature in BASE SAS as well. 

[Quentin]

Neat idea, @Amarnath7.  I have a log checker that works in interactive or batch session to scan the current log.  But in PC SAS (Display Manager SAS) I never thought about creating an easy way to batch submit the current program. Assuming you have the .sas program saved, I guess it could be just a systask call that starts a new sas session to batch submit the current program.  Or if you want to allow batch runs of code that hasn't been saved, you could use DM commands to write the code from the current editor window to a temporary .sas file, and batch submit that.  I think that's similar to what folks who create batch submit buttons in their favorite text editor do. I look forward to seeing your macro.

[Amarnath7]

Yes, it is similar to what you said with additional features.