Emailing from SAS Studio - New files only accessible after signing off and back on

[GlennGail]

I have just begun an effort to migrate existing z/OS based programs from the mainframe to Linux.  I am using SAS Studio to run the programs.  So far everything has worked smoothly and I am very pleased with SAS Studio.  One mystery has me scratching my head...  When I create/update .CSV files in my SAS program that are written on the server my email step cannot find the file unless I log off SAS Studio and log back in.  Then it finds the file.  A minor annoyance, but I am sure there is a solution.  I am new to SAS Studio, but I am not new to the SAS Programming language.   

[JackHamilton]

I have not had that problem on Solaris Unix.  Are you closing the file before you attempt to email it?

Posting the code you use might be helpful.

[GlennGail]

Here is the code, I added the filename clears as an attempt to "close" the files if that is what you mean.  The code works if I log off and back on, but not as the last steps in the program.  

filename csv1 clear ;
filename csv2 clear ;
filename csv3 clear ;

RUN ;

*filename _all_ list ;

filename mymail email 'xxxx@xxx.com'
subject='REPORTS'
attach=('/home/user/test/output/csv1.csv'
             '/home/user/test/output/csv2.csv'
             '/home/user/test/output/csv3.csv' ) ;

data _null_;
file mymail;
put 'REPORTS';

run;

[Reeza]

What is the error message?

Is this through Viya?

[GlennGail]

Using SAS Studio...  Not on Viya

NOTE: The file MYMAIL is:

E-Mail Access Device

Error Messages:

ERROR: Error opening attachment file /HOME/USER/TEST/OUTPUT/CSV1.CSV.

ERROR: Physical file does not exist, /HOME/USER/TEST/OUTPUT/CSV1.CSV.

NOTE: The SAS System stopped processing this step because of errors.

NOTE: DATA statement used (Total process time):

real time 0.07 seconds

cpu time 0.00 seconds

[Kurt_Bremser]

The error message has the filename in capitals, this is not what I would expect.

Always keep an eye on the fact that UNIXen are case sensitive.

[JackHamilton]

I don't have a clue. Can a data step see the files in the same session, e.g.
data _null_;
infile '/home/user/test/output/csv1.csv';
input;
run;
I can understand this happening in z/OS, where the file won't be catalogued until it's been closed, but Unix doesn't have the same behavior.

[GlennGail]

You hit the nail on the head...  "Can a data step see the files in the same session?"  I think that may be the real question...  On z/OS the .CSV file is a permanent file that is overwritten in the SAS program using %DS2CSV (RUNMODE=B, DATA=DATAT1, CSVFREF=CSV1, LABELS=Y,...  Then the step ends and in the next job step I use SAS to email it.  

When I converted, I created some empty files manually, then in my SAS program I created filerefs for them and wrote to the file using %DS2CSV...  The last thing I did was attempt to email them (which fails with Physical file does not exist),,, 

So, again, I think you may have the right question...   "Can a data step see the files in the same session?"   My attempt to address this was the "filename csv1 clear ;" statement (which did not work), but maybe there is another command/statement that I have overlooked?  

[Reeza]

Out of curiosity, what happens if you save the files in all lower case to your work folder rather than the user folder?

[jimbarbour]

@GlennGail,

I really don't know why you would see that behavior, but here are some ideas:

• If you have SAS/CONNECT licensed, you could try putting your email code into an RSUBMIT block followed by a WAITFOR.  RSUBMIT will spawn a sub process which will have a different Process ID.  This is a separate process, so maybe it will function like logging out and logging back in without actually having to log out.  @LeonidBatkhan has a recent article on this topic:  https://blogs.sas.com/content/sgf/2021/01/13/running-sas-programs-in-parallel-using-sas-connect/
• If you don't have SAS/CONNECT, you can still use a SYSTASK to launch a subordinate process, you just have to do a bit more coding.  I have a paper on that subject if it's helpful:  https://www.lexjansen.com/wuss/2018/15_Final_Paper_PDF.pdf
• If the above fail or are too difficult to debug (parallel processing is a bit tricky sometimes), you might try putting your email code into a CALL EXECUTE and seeing if that resolves the problem.
• Lastly, you could use a DOSUBL for your email code, but the DOSUBL executes while your primary process is still executing, so I'm less hopeful about this option

Jim

[GlennGail]

Wow... Lots of great stuff for me to try, and I am sure I will learn a lot about this environment by doing so. This will likely take me a little time, but I am sure the solution to my issue is in this list somewhere... Time to start reading... THANKS SO MUCH... I will get back to you.

[Patrick]

@GlennGail 

Wouldn't know why but it feels like a timing issue.

Does below work for you? (after you change %let csv_file=c:\temp\csv1.csv; to a path valid in your environment). 

options mautolocdisplay;
%let csv_file=c:\temp\csv1.csv;

filename csv1 "&csv_file";
%DS2CSV (
  RUNMODE=B, 
  DATA=sashelp.class, 
  CSVFREF=CSV1, 
  LABELS=Y
  );
filename csv1 clear;

/*data _null_;*/
/*  call sleep(10,1);*/
/*run;*/

filename csv1 "&csv_file";
data _null_;
  infile csv1;
  input;
  put _infile_;
  stop;
run;
filename csv1 clear;

If it's not working uncomment the data _null_ step with the call sleep()

If it's working try the code with your logic sending an email - and if this is not working also try with using call sleep().

[GlennGail · Accepted solution]

Thanks for your suggestion...  It does not seem to be a timing issue.  I have no idea what the "root cause", but I have have found that if I begin the program with PROC OPTSAVE then issue the PROC OPTLOAD after creating the .CSV files and before the email code everything works great...  Now I will start digging in to understand why this works...

Ok, did a PROC OPTIONS at the beginning of the program and then again before the MAIL code and put the before and after in files then ran a compare... 

I believe the root cause is the OPTIONS parameter NOCAPS/CAPS is the culprit...

Start of program:  NOCAPS Does not convert certain types of input, and all data lines, into uppercase characters.

Later in program:  CAPS Converts certain types of input, and all data lines, into uppercase characters.

So...  The simple fix is to add OPTIONS NOCAPS before the email  

[Kurt_Bremser]

See my previous comment on the all-caps filename in the log. Up to now, I didn't even know that such an option existed, so you helped me learn something new 😉

Since filenames in z/OS are caps only, you will probably find that option in most, if not all, programs you migrate.

This option makes sense in z/OS, but not in a case sensitive UNIX. Remove it in all codes, and activate it only for steps where you find you need it.

[GlennGail]

Absolutely, I did read your comment, but being a "newbie" I was not sure if that was simply the way the error message came out since I had LOWER case in my code.  I still have a mystery though...  NO PLACE in my code do I specify an OPTIONS statement...  There is an "auto generated" message in the log when I run it though.  

The auto generated OPTIONS is:   OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

So, my mystery is:  What caused the CAPS option to be set? 

WELL, WELL, WELL...  Found it!!!  BURIED in some INCLUDE Code!  The OPTION CAPS is being explicitly set.  Mystery SOLVED.