topic Re: Emailing from SAS Studio - New files only accessible after signing off and back on in SAS Studio

Emailing from SAS Studio - New files only accessible after signing off and back on

GlennGail — Thu, 24 Jun 2021 19:45:51 GMT

I have just begun an effort to migrate existing z/OS based programs from the mainframe to Linux. I am using SAS Studio to run the programs. So far everything has worked smoothly and I am very pleased with SAS Studio. One mystery has me scratching my head... When I create/update .CSV files in my SAS program that are written on the server my email step cannot find the file unless I log off SAS Studio and log back in. Then it finds the file. A minor annoyance, but I am sure there is a solution. I am new to SAS Studio, but I am not new to the SAS Programming language.

Re: Emailing from SAS Studio - New files only accessible after signing off and back on

JackHamilton — Thu, 24 Jun 2021 20:50:06 GMT

I have not had that problem on Solaris Unix. Are you closing the file before you attempt to email it?

Posting the code you use might be helpful.

Re: Emailing from SAS Studio - New files only accessible after signing off and back on

GlennGail — Thu, 24 Jun 2021 20:56:32 GMT

Here is the code, I added the filename clears as an attempt to "close" the files if that is what you mean. The code works if I log off and back on, but not as the last steps in the program.

filename csv1 clear ;
filename csv2 clear ;
filename csv3 clear ;

RUN ;

*filename _all_ list ;

filename mymail email 'xxxx@xxx.com'
subject='REPORTS'
attach=('/home/user/test/output/csv1.csv'
'/home/user/test/output/csv2.csv'
'/home/user/test/output/csv3.csv' ) ;

data _null_;
file mymail;
put 'REPORTS';

run;

Re: Emailing from SAS Studio - New files only accessible after signing off and back on

Reeza — Thu, 24 Jun 2021 21:40:20 GMT

What is the error message?

Is this through Viya?

Re: Emailing from SAS Studio - New files only accessible after signing off and back on

JackHamilton — Thu, 24 Jun 2021 23:14:59 GMT

I don't have a clue. Can a data step see the files in the same session, e.g.
data _null_;
infile '/home/user/test/output/csv1.csv';
input;
run;
I can understand this happening in z/OS, where the file won't be catalogued until it's been closed, but Unix doesn't have the same behavior.

Re: Emailing from SAS Studio - New files only accessible after signing off and back on

GlennGail — Fri, 25 Jun 2021 13:42:33 GMT

Using SAS Studio... Not on Viya

NOTE: The file MYMAIL is:

E-Mail Access Device

Error Messages:

ERROR: Error opening attachment file /HOME/USER/TEST/OUTPUT/CSV1.CSV.

ERROR: Physical file does not exist, /HOME/USER/TEST/OUTPUT/CSV1.CSV.

NOTE: The SAS System stopped processing this step because of errors.

NOTE: DATA statement used (Total process time):

real time 0.07 seconds

cpu time 0.00 seconds

Re: Emailing from SAS Studio - New files only accessible after signing off and back on

GlennGail — Fri, 25 Jun 2021 13:52:23 GMT

You hit the nail on the head... "Can a data step see the files in the same session?" I think that may be the real question... On z/OS the .CSV file is a permanent file that is overwritten in the SAS program using %DS2CSV (RUNMODE=B, DATA=DATAT1, CSVFREF=CSV1, LABELS=Y,... Then the step ends and in the next job step I use SAS to email it.

When I converted, I created some empty files manually, then in my SAS program I created filerefs for them and wrote to the file using %DS2CSV... The last thing I did was attempt to email them (which fails with Physical file does not exist),,,

So, again, I think you may have the right question... "Can a data step see the files in the same session?" My attempt to address this was the "filename csv1 clear ;" statement (which did not work), but maybe there is another command/statement that I have overlooked?

Re: Emailing from SAS Studio - New files only accessible after signing off and back on

Kurt_Bremser — Fri, 25 Jun 2021 13:55:58 GMT

The error message has the filename in capitals, this is not what I would expect.

Always keep an eye on the fact that UNIXen are case sensitive.

Re: Emailing from SAS Studio - New files only accessible after signing off and back on

Reeza — Fri, 25 Jun 2021 15:17:58 GMT

Out of curiosity, what happens if you save the files in all lower case to your work folder rather than the user folder?

Re: Emailing from SAS Studio - New files only accessible after signing off and back on

jimbarbour — Fri, 25 Jun 2021 15:58:08 GMT

@GlennGail,

I really don't know why you would see that behavior, but here are some ideas:

If you have SAS/CONNECT licensed, you could try putting your email code into an RSUBMIT block followed by a WAITFOR. RSUBMIT will spawn a sub process which will have a different Process ID. This is a separate process, so maybe it will function like logging out and logging back in without actually having to log out. @LeonidBatkhan has a recent article on this topic: https://blogs.sas.com/content/sgf/2021/01/13/running-sas-programs-in-parallel-using-sas-connect/
If you don't have SAS/CONNECT, you can still use a SYSTASK to launch a subordinate process, you just have to do a bit more coding. I have a paper on that subject if it's helpful: https://www.lexjansen.com/wuss/2018/15_Final_Paper_PDF.pdf
If the above fail or are too difficult to debug (parallel processing is a bit tricky sometimes), you might try putting your email code into a CALL EXECUTE and seeing if that resolves the problem.
Lastly, you could use a DOSUBL for your email code, but the DOSUBL executes while your primary process is still executing, so I'm less hopeful about this option

Jim

Re: Emailing from SAS Studio - New files only accessible after signing off and back on

GlennGail — Fri, 25 Jun 2021 16:01:42 GMT

Wow... Lots of great stuff for me to try, and I am sure I will learn a lot about this environment by doing so. This will likely take me a little time, but I am sure the solution to my issue is in this list somewhere... Time to start reading... THANKS SO MUCH... I will get back to you.

Re: Emailing from SAS Studio - New files only accessible after signing off and back on

Patrick — Sat, 26 Jun 2021 05:35:04 GMT

@GlennGail

Wouldn't know why but it feels like a timing issue.

Does below work for you? (after you change %let csv_file=c:\temp\csv1.csv; to a path valid in your environment).

options mautolocdisplay;
%let csv_file=c:\temp\csv1.csv;

filename csv1 "&csv_file";
%DS2CSV (
  RUNMODE=B, 
  DATA=sashelp.class, 
  CSVFREF=CSV1, 
  LABELS=Y
  );
filename csv1 clear;

/*data _null_;*/
/*  call sleep(10,1);*/
/*run;*/

filename csv1 "&csv_file";
data _null_;
  infile csv1;
  input;
  put _infile_;
  stop;
run;
filename csv1 clear;

If it's not working uncomment the data _null_ step with the call sleep()

If it's working try the code with your logic sending an email - and if this is not working also try with using call sleep().

Re: Emailing from SAS Studio - New files only accessible after signing off and back on

GlennGail — Sat, 26 Jun 2021 07:05:55 GMT

Thanks for your suggestion... It does not seem to be a timing issue. I have no idea what the "root cause", but I have have found that if I begin the program with PROC OPTSAVE then issue the PROC OPTLOAD after creating the .CSV files and before the email code everything works great... Now I will start digging in to understand why this works...

Ok, did a PROC OPTIONS at the beginning of the program and then again before the MAIL code and put the before and after in files then ran a compare...

I believe the root cause is the OPTIONS parameter NOCAPS/CAPS is the culprit...

Start of program: NOCAPS Does not convert certain types of input, and all data lines, into uppercase characters.

Later in program: CAPS Converts certain types of input, and all data lines, into uppercase characters.

So... The simple fix is to add OPTIONS NOCAPS before the email

Re: Emailing from SAS Studio - New files only accessible after signing off and back on

GlennGail — Sat, 26 Jun 2021 07:07:15 GMT

Thanks for your suggestions... I have no idea what the "root cause", but I have have found that if I begin the program with PROC OPTSAVE then issue the PROC OPTLOAD after creating the .CSV files and before the email code everything works great... Now I will start digging in to understand why this works...

Ok, did a PROC OPTIONS at the beginning of the program and then again before the MAIL code and put the before and after in files then ran a compare...

I believe the root cause is the OPTIONS parameter NOCAPS/CAPS is the culprit...

Start of program: NOCAPS Does not convert certain types of input, and all data lines, into uppercase characters.

Later in program: CAPS Converts certain types of input, and all data lines, into uppercase characters.

So... The simple fix is to add OPTIONS NOCAPS before the email

Re: Emailing from SAS Studio - New files only accessible after signing off and back on

Kurt_Bremser — Sat, 26 Jun 2021 11:02:55 GMT

See my previous comment on the all-caps filename in the log. Up to now, I didn't even know that such an option existed, so you helped me learn something new 😉

Since filenames in z/OS are caps only, you will probably find that option in most, if not all, programs you migrate.

This option makes sense in z/OS, but not in a case sensitive UNIX. Remove it in all codes, and activate it only for steps where you find you need it.

Re: Emailing from SAS Studio - New files only accessible after signing off and back on

GlennGail — Sat, 26 Jun 2021 14:47:41 GMT

Absolutely, I did read your comment, but being a "newbie" I was not sure if that was simply the way the error message came out since I had LOWER case in my code. I still have a mystery though... NO PLACE in my code do I specify an OPTIONS statement... There is an "auto generated" message in the log when I run it though.

The auto generated OPTIONS is: OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

So, my mystery is: What caused the CAPS option to be set?

WELL, WELL, WELL... Found it!!! BURIED in some INCLUDE Code! The OPTION CAPS is being explicitly set. Mystery SOLVED.

Re: Emailing from SAS Studio - New files only accessible after signing off and back on

jimbarbour — Sat, 26 Jun 2021 15:29:58 GMT

Which makes sense: When you logged off and logged back on the CAPS option was no longer set because you only executed (I presume) the email portion of the code and not the %INCLUDE that set the CAPS option.

When testing, I've found it helpful to log off and start with a fresh session one last time before implementing new code. Every once in a while, I'll catch a mistake where the code was only working because of some previous iteration of execution.

Jim

Re: Emailing from SAS Studio - New files only accessible after signing off and back on

JackHamilton — Mon, 28 Jun 2021 20:38:25 GMT

When testing, I've found it helpful to log off and start with a fresh session one last time before implementing new code. Every once in a while, I'll catch a mistake where the code was only working because of some previous iteration of execution.

For me, it has unfortunately been more often than "Every once in a while", but in SAS Studio the problem has always been fixed by restarting the session using the SAS Studio menu option. That's easier and faster than logging out and back in. There may be a circumstance when that doesn't suffice, but I haven't found it yet (other than the case where there's a Windows memory leak somewhere - SAS Studio or the browser - and I have to not only log out but also close and restart the browser).