Error handling for missing file using FILENAME statement.

[G_I_Jeff]

Thanks for your suggestion.

Wasn't making a contest, just trying to be funny.

[data_null__]

I knew you were making a joke. I'm glad to be the winner 🙂

Are you a G.I. as in serving in the Armed Forces?

[G_I_Jeff]

I used to be in the Navy, 90-94. Got out, went to college, got my tech degree, thought it would be fun to be a programmer, went to work for Hallmark Cards, left to go to Yellow Freight trucking company and recently just aquired a job with the federal government working for the USDA. I actually was programming before I joined the Navy and came back to it. I love it. I'm a mainframer for life, one of the youngest in Kansas City.

Got a new dillema with your suggested code:
I tried it with a different file, non-comma seperated file, and now the same code seems to be looping.

OPTIONS NOERRABEND;

DATA LNX_USERS;

LENGTH FILENAME $256;
FILENAME = CATS('SYS90H.SYSE.USERLOG.D',PUT(TODAY()-1,YYMMDDN6.));

IF FILEEXIST(FILENAME) THEN DO;

INFILE DUMMY END=EOF FILEVAR=FILENAME MISSOVER;
DO WHILE(NOT EOF);

INPUT
@1 RECTYPE $1.
@2 LNXUSER $8.
@13 LNXTIME TIME.
@25 CPU 4.0
@31 TOTCPUT 5.0
@37 VRTCPUT 5.0
@45 CPURATIO 3.0
@51 VRTIO 4.0
@57 TOTIO 4.0
@82 PAGESEC 4.0
@138 WEIGHT 3.0
@144 VCPUS 1.0;

IF RECTYPE NE 'U' THEN DELETE;
LNXDATE=TODAY()-1;
SYSTEM='SYSE';

OUTPUT;
END;
END;
ELSE DO;
ABORT RETURN 4;
END;
STOP;
RUN;

I removed the delimiter, dsd and firstobs options from the INFILE statement and added the MISSOVER (after several attempt thinking it might be missing values) and it still doesn't end. This new file has 143k rows but if I feed it the old way I used to, it only took less than a minute to run.

It's almost like it's not reaching the END=EOF condition. Is there something that I can write to the SASLOG that will tell me if it's stuck processing the same record or looping through the entire file? Could it be the FILEVAR statement? I've read up on it and it sounds like a placeholder variable for reading multiple files (which is what I want to do with this job, but just trying 1 file for now) but doesn't really sound like a culprit.

Jeff

[Robert_Bardos]

You might insert something like this between your two END statements:
[pre]
outer_loop+1 ;
putlog outer_loop= ;
[/pre]
O.k., I'm just guessing but I think a STOP statement between the ENDs might help.

[data_null__]

The problem is with the DELETE statement. It sends the program back to the top of the data step.

[pre]
IF RECTYPE NE 'U' THEN DELETE;
LNXDATE=TODAY()-1;
SYSTEM='SYSE';
OUTPUT;
END;
[/pre]

You can change DELETE to CONTINUE and get the desired result.

Also, since LNXDATE and SYSTEM are constants move those statements to the top of the data step so they are only executed once.

Just for good measure put a stop at the end of the step that will always execute, you want this step to stop when it gets to the bottom in all cases. In this data step _N_ should never be greater than 1.

I used mainframes for many years. You had to if you used SAS, in the 70s. I always enjoyed using mainframes and never understood those who thought otherwise.

Thanks for your service!

[Peter_C]

I only found SAS in the decade after "data _null_;", so hesitate (again) to offer any improvement to proposals from such illustrious expertise, but
I'll suggest not starting the step to read the file until satisfied that it exists.
Placing this step before the step of the infile removes that problem[pre]data _null_;
length filename $54 ;
filename = cats('SYS90H.SYSE.USERLOG.D',put(today()-1,yymmddn6.));
if fileexist(filename) then call symputx( 'file_name', filename );
else abort return 4;
stop;
run;[/pre]then the data step to read the data becomes much simpler[pre]DATA LNX_USERS;
infile "&file_name" truncover;
input
@ 1 rectype $1. @ ;
if rectype ne 'U' then delete ;
input
@ 2 lnxuser $8.
@13 lnxtime time.
@25 cpu 4.
@31 totcput 5.
@37 vrtcput 5.
@45 cpuratio 3.
@51 vrtio 4.
@57 totio 4.
@82 pagesec 4.
@138 weight 3.
@144 vcpus 1.;
retain SYSTEM 'SYSE'
LNXDATE %sysevalf( %sysfunc(TODAY()) -1 ) ;
format LNXDATE date11. ;
RUN;[/pre] I seem to remember this suggestion earlier in the thread

as I was adapting the original code, I also made those minor changes
1
stop reading data when a suitable break occurs, like testing for rectype=U
2
do not use a trailing zero on informats, and only define number of decimal places when that number needs to be "implied" (using binary informats like PD and S370FPD for data which have no decimal point and any "number of decimal" places is suppplied by the informat)
3
remove as much syntax as possible (while keeping logic clear)

[data_null__]

> I only found SAS in the decade after "data _null_;",
> so hesitate (again) to offer any improvement to
> proposals from such illustrious expertise, but
> I'll suggest not starting the step to read the file
> until satisfied that it exists.

I did not mention 80s because of the VAX and PC SAS. If you wanted to get any real work done you still needed big iron.

While I see no advantage to the FILEVAR method over yours there are situations where FILEVAR method may be preferred. In this scenario your approach is more efficient as you have noted.

When I first read Jeff's question I "read/heard" next step as SAS program step not job step as I understand now. I thought he wanted to read a file and create a data set and continue with the SAS program. In that situation the FILEVAR approach creates a data set with 0 obs and no errors so the program can continue. This is the result I most often desire in my programs.

[G_I_Jeff]

data _null_;

Thanks again. You are right, it was the DELETE statement. Although I can't understand why they would architect that command to restart a DATA step?!? I also need to read up on the STOP command because I don't fully understand it's function while in a DO loop.

Also, the first dilemma I presented in this thread (File SYS90H.LNXMEM.D*) was a single job. The job I'm working on now has 6 files I need to read and APPEND into a daily PDB. So, for now, to automate my job (which is my immediate goal) this code will allow this task. In the very near future I probably will need to learn and write a macro to run through all 6 files more efficiently than creating 6 different DATA steps with 6 different APPEND steps.

All of the Sas steps will be added to a daily batch job that I ultimately didn't want failing if any/all of these files were missing. If they are, I set an appropriate return code and add a condition check at the end to e-mail me the missing file names. This keeps me from getting woke up at 2 am.

Peter.C

Thanks for the suggestions and I will use them. I agree with your code, much more efficient and will make it easier to read. Although I will have to revisit your second suggestion soon. I had quite a difficult time getting Sas to recognize the numbers correctly (and I still have some jobs I'm still scratching my head on). Most of the data that I'm working with right now comes from z/VM partitions, straight out of IBM's Performance Toolkit. The data looks very straight forward but when I import it into pdb's, Sas keeps translating the numbers into different formats.

There have been many nights I've had to stop by the liquor store on the way home after dealing with these jobs. My main problem is that I am the only active "coder" in the entire building and the ONLY SAS user. So I have no one to bounce my ideas or problems off of. Google only gets me so far.

You guys have helped me out IMMENSELY.

Thank you!!

[data_null__]

> data _null_;
>
> Thanks again. You are right, it was the DELETE
> statement. Although I can't understand why they would
> architect that command to restart a DATA step?!? I
> also need to read up on the STOP command because I
> don't fully understand it's function while in a DO
> loop.

The data step is an implied do while not eof loop. But in the program I showed you the loop was explicitly written. Six of one half dozen kind of thing.

DELETE and RETURN are similar statements

DELETE returns to the top of data step with no output.

RETURN outputs and returns to top of data step, top of data step as in top of implied loop.

RETURN functions differently when used with LINK, and in the presents of explicit OUTPUT statement.

DELETE has a somewhat misleading name as nothing is removed just not output.

STOP stops the data step as soon as it is executed. Inside a DO loop or outside everything is stopped.

There are lots of good papers you can read on how the data step works. Look
at this site. http://www.lexjansen.com/

I think once you learn a bit more about how it works you will like data step programming, it is very powerfull.

[Ksharp]

Hi.
If you know the directory which contains these files.
You can use ' filename + pipe 'ls ....' ' which will display the exist files ,then there is not need to use fileexist() function to judge whether the file is existed.
and use filevar= just as data _null_; mentioned to input multi-files.


Ksharp

[G_I_Jeff]

Ok guys, new question, same subject:

I have my code in place and working, ready to implement into a daily batch job. Again, thank you for all your help in the matter. My new questions is, is there an easy way to code the following DATA step into a macro so that I don't have to repeat it for every file I need to input into a pdb?

DATA VMDATA.DAILY_SYSE_GUESTS;

LENGTH FILENAME $256;
FILENAME = CATS('SYS90H.SYSE.USERLOG.D',PUT(TODAY()-1,YYMMDDN6.));
LNXDATE=TODAY()-1;
SYSTEM='SYSE';

IF FILEEXIST(FILENAME) THEN DO;

INFILE DUMMY END=EOF FILEVAR=FILENAME MISSOVER;
DO WHILE(NOT EOF);

INPUT
@1 RECTYPE $1.
@2 LNXUSER $8.
@13 LNXTIME TIME.
@25 CPU 4.0
@31 TOTCPUT 5.0
@37 VRTCPUT 5.0
@45 CPURATIO 3.0
@51 VRTIO 4.0
@57 TOTIO 4.0
@82 PAGESEC 4.0
@138 WEIGHT 3.0
@144 VCPUS 1.0;

IF RECTYPE NE 'U' THEN CONTINUE;

OUTPUT;
END;
END;
ELSE DO;
ABORT RETURN 4;
END;
STOP;
RUN;

The only things I change in the code are the FILENAME & SYSTEM variables. Those I can figure out how to code simple macro variables for. My real question is the DATA step name. I create a pdb for every file I need to read and then APPEND them all to one pdb at the end of the job. If there is an easier or better way of doing this, I'm all ears!!

Again, I will reiterate to you all, I am SAS stupid and have just taken the beginners course. I do not have enough time under my belt with SAS to do what I want with it.

Thanks for your time and participation.

Jeff

[data_null__]

How about using FILEVAR= to read all the files into one SAS data set. You can have a variable to indicate which file the data came from.

[G_I_Jeff]

data _null_;

In my case, I am testing for the file to exist before I set the INFILE filename. I think I understand what you are suggesting but I don't see how it will work in my case. I also think I understand how the FILEVAR is used. I didn't realize you could assign more than one filename to the variable, but again, in my case I don't think this will work. I would rather complete this task in one DATA step if I can honestly.

Or maybe I don't understand (highly possible)....

Jeff

[sbb]

Personally, I setup with FILENAME ahead of the DATA step, then interrogate the results of the FILENAME in a macro to decide whether or not to execute the DATA step (and any downstream code).

Scott Barry
SBBWorks, Inc.

Suggested Google advanced search argument, this topic / post:
filename data step infile filevar site:sas.com

>> Search reveals this gem:

http://support.sas.com/techsup/technote/ts581.pdf

[Cynthia_sas]

Hi:
Only chiming in to provide some recommended reading in response to this question:
is there an easy way to code the following DATA step into a macro so that I don't have to repeat it for every file I need to input into a pdb?

And the answer is Yes, there is a way to code a macro program to do what you want. Whether it is easy or not is up to you. The way to make your learning process easier, is to learn about the SAS Macro facility and the difference between macro variables (&macvar) and macro programs (%macpgm) and the entire realm of macro processing in between.

This is a good place to start:
http://www2.sas.com/proceedings/sugi28/056-28.pdf

And then look for more papers and tutorials on the subject of SAS macro processing. If you Google, you will find them!

cynthia