EG Not Showing Datasets in Folders

[nbwest76]

I have an issue that's driving me crazy.  I created a new Project with a process flow.  When I first created the project and ran the process flow for the first time it runs correctly and each of the programs within the project all successfully create the datasets and I can see them on the project tree.  However when I save the project, close it, open it later and run any of the programs in the process flow I am not seeing the tables being created in the tree.  Also I am not getting an error.

For example, If I run the code below on the local connection:

data tblCITLoanList;
	infile datalines delimiter=",";
	informat	Client $3.
				Loan_Num $10.
				Specialty $8.;
				
	input Client $ Loan_Num $ Specialty $  ;

	datalines;
123,1234567898,CIT 8876
;

run;

the log shows:

NOTE: The data set WORK.TBLCITLOANLIST has 1 observations and 3 variables.
NOTE: DATA statement used (Total process time):
      real time           0.03 seconds
      cpu time            0.03 seconds

However I do not see it in my work library when I refresh the local server.  If I create a brand new SAS EG project and run the same exact code on the local server I can see the dataset in the work folder.  It's almost as if once I save a project then run it later it doesn't work.  Is there something in the settings I'm missing?  I also connect to  grid server and the same thing happens there too.

[Tom]

Each SAS session has its own WORK folder.  

If you start a new SAS session you will need to re-run the step that creates the dataset if you want to use it.

[nbwest76]

I did run the program several times but I do not see the dataset in the local WORK folder. If I open a new blank SAS EG session and paste in the same code and run it, I see the dataset in the work folder. If I re-run the program and do a simple proc sql to create a new table off of the original it says that the original is not found even though the log says it was successfully created.

[nbwest76]

Here is a photo example.  If I run the Create Cap Sheet Program, it is supposed to create the "capsheet_final" dataset.   I created this project yesterday and when it ran yesterday it created the dataset correctly and I could see it.  Today when I run it I don't get an error message but when I double click the dataset, it isn't there.

[nbwest76]

After further review it looks like it has something to do with Parallel Execution being turned on, but can someone explain why this is causing an issue?

[Kurt_Bremser]

With parallel execution, you get several SAS sessions, each with their own WORK. Use permanent libraries for datasets, or have a USER library defined in your workspace server.

[nbwest76]

I just went through each of the programs and added a permanent library in lieu of the temp work library and it still didn't work.

this libname statement below is running on the local server 

libname SAS_Temp "c:\Datasets";

NOTE: Libref SAS_TEMP was successfully assigned as follows: 
Engine: V9 

There are no errors but when I refresh the local server the folder isn't there.  If I turn off Parallel Execution and run the same libname statement, refresh the local server I then see the SAS_Temp folder.  Is there anything I can do to get this project to work, or should I just come to the realization that parallel execution isn't going to work for me here.

[lanceanz]

In addition to your declaring the permanent LIBNAME, have you added it to the output tablenames? e.g.

data SAS_Temp.tblCITLoanList;
	infile datalines delimiter=",";
	informat	Client $3.

 Also, can you please clarify what you mean by the "local server"? Do you have SAS installed on your PC, with the EG Connections dialog showing "do not use a profile" as Active? I didn't think you could do parallel execution without multiple workspace server sessions. If you're using grid servers, does LIBNAME SAS_Temp 'c:\datasets';  actually succeed?

If the log shows that you've successfully written the output from the DATA steps into a permanent folder on disk (C:\Datasets), then they should still be there after you've closed EG.

[nbwest76]

Yes I tried that after defining the permanent library it still did not assign the library even though the log says it is assigned.  I came across this SAS Blog

which I think helps explain my situation but I have still not been able to resolve it.  I tried creating an Autoexec process flow to assign the library immediately when the project opens but it still does not assign it when parallel is turned on.

My SAS is installed on my PC and by local server I mean the local server profile which is the non-grid environment.  My process flow does have one program that connects to a SASApp server but I can't seem to assign a perm library at all when parallel is turned on.

I tried an experiment by creating a new SAS project only running the code below and it successfully assigns the Temp_SAS library when parallel is turned off but when it is turned on it does not.

/* Input data library */

libname SAS_Temp "c:\temp";

data tblCITLoanList;
	infile datalines delimiter=",";
	informat	Client $3.
				Loan_Num $10.
				Specialty $8.;
				
	input Client $ Loan_Num $ Specialty $  ;

	datalines;
123,1234567898,CIT 8876
;

run;


NOTE: Libref SAS_TEMP was successfully assigned as follows: 
Engine: V9 

my computer does not have SAS Management Console that I can see in my Start/SAS folder so I'm not sure where to go from here yet.

[lanceanz]

Hi again. Thanks for that. I confess that I'm still confused. Hopefully someone more knowledgeable answer your questions. We've just got a non-distributed server environment running on Windows Server with no remaining desktop-SAS licenses, so I can't try running 1 flow on the PC and the other on a server. If I could, I'd be putting the outputs into a permanent folder on the network file-server (and not to my local C: drive, because the SAS server cannot see that).

I'll watch the thread with interest.

Cheers,

Lance

[nbwest76]

I think I figured out the issue partially.  I was able to see the SAS_Temp library once I added the libname statement within the Tools>Options>SAS Programs>Submit SAS code when server is connected option. I'm not sure why I didn't think about doing this sooner! Doh!

however now I'm running into another issue downstream that I'm not sure is related.  One of the programs within the process flow creates a dataset and stores it in the SAS_Temp library. The process flow then runs a task that uploads that dataset to the SASApp grid environment.  The task runs fine and when I refresh the SASApp server on the left window pane, I see the dataset uploaded successfully to the work folder on the SASApp server.  However one of subsequent programs that runs connects to the SASApp server but when it tries to reference the work dataset I uploaded previously it does not find it., it says that it doesn't exist.  I'm sure there is something I'm not thinking about, any suggestions?

[lanceanz]

Hah. Sounds like you're making progress. That's the way we assign frequently-used libraries in EG (sometimes by using a %INCLUDE) or otherwise by running the assignment code at the start of the process flow.

But you're writing to the WORK library on SASApp? If you write it to a permanent location, it won't go away.

[Kurt_Bremser]

"Parallel Execution" also influences the non-local SASApp server, so you have to apply the same strategies there (use a permanent library that is automatically assigned at server connection). 

[bbenbaruch]

I am having the same issue.  I try to be respectful of those who know more than I do about these issues, but I keep coming back to the "gremlins did it" explanation proposed back in 1978 in the seminal work "Towards a Unified Theory of Computer Glitches, SNAFUs, and FUBARs: The Gremlin Theory of Why Bad Things Happen to Good People and Their Computers and IT Infrastructures" (Journal of Zoological Cybernetics, 6:3 (April 1978), pp. 27-45).

The "gremlins did it" explanation still seems to have as much explanatory power as more recent attempts to explain paranormal cybernetic anomalies - especially if gremlins are posited to have some of the characteristics of Higgs-Boson particles as suggested in some more recent work.

Also see "The catastrophic Mysterious Nefarious Actions of Gremlins, Golems, Goblins and Tribbles in Cyberspace" in Roy Bradberry (ed.), Mysteries of Cyberspace, NY, Random Data House, 2020, pp.179-228.

[SASKiwi]

@bbenbaruch  - are you just making an observation or are you asking for help?