topic Re: Physical file does not exist (INLCUDE MACRO) in SAS Programming

Physical file does not exist (INLCUDE MACRO)

galtay — Sun, 26 Feb 2017 21:01:46 GMT

[Edit]

Sorry for the double post, but it appears that my edited questions stay in limbo for a while (I have one from last night that hasn't reappeared yet). I made the mistake of editing a similar question to this and now its gone, so reposting as new question

[/Edit]

I'm trying to get the CMS HCC risk model to run in SAS Studio University Eddition and encountering some blockers. One bit of code attempts to inlcude lines from a file called "V22H79L1.sas".

%INCLUDE IN0(&LABELMAC)  /SOURCE2; %* hcc labels;

The file that contains the line above is a macro that defines the LABELMAC macro variable as an input argument,

%MACRO V2216O2M(INP=, IND=, OUTDATA=, IDVAR=, KEEPVAR=, SEDITS=,
                 DATE_ASOF=, DATE_ASOF_EDIT=,
                 FMNAME9=V22Y15RC, FMNAME0=V22Y16RC,
                 AGEFMT9=I9AGEY15MCE, SEXFMT9=I9SEXY15MCE,
                 AGEFMT0=I0AGEY16MCE, SEXFMT0=I0SEXY16MCE, DF=1,
                 LABELMAC=V22H79L1);

I have already defined IN0 such that it points to the directory containing the file "V22H79L1.sas" and I get the following error,

 1          OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
 WARNING: Physical file does not exist, /folders/myfolders/sasuser.v94/cms_hcc_2017/V22H79L1.sas.
 ERROR: Cannot %INCLUDE member V22H79L1 in the aggregate IN0.
 61         
 62         %INCLUDE IN0(&LABELMAC)  /SOURCE2; %* hcc labels;
 63         
 64         OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
 77

I get the same error if I define the macro variable just before the %INCLUDE statement,

%LET LABELMAC=V22H79L1;
%INCLUDE IN0(&LABELMAC)  /SOURCE2; %* hcc labels;

BUT, if I define the macro variable with quotes it finds the file to include,

 
 1          OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
 61         
 62         %LET LABELMAC='V22H79L1.sas';
 63              %INCLUDE IN0(&LABELMAC)  /SOURCE2; %* hcc labels;
 NOTE: %INCLUDE (level 1) file IN0(V22H79L1.sas) is file /folders/myfolders/sasuser.v94/cms_hcc_2017/V22H79L1.sas.

So what's going on here? The SAS code from CMS has all the macro variables defined w/o quotes and it'd take a while to go through and replace them all. Am I doing something wrong in the initial case where LABELMAC is defined w/o quotes and w/o the "sas" extension?

Re: Physical file does not exist (INLCUDE MACRO)

art297 — Sun, 26 Feb 2017 21:08:36 GMT

Can't say for sure without seeing the macro and how it's used, but can't you just double quote the resulting macro variable(s)? e.g.

%LET LABELMAC=V22H79L1;
%INCLUDE IN0("&LABELMAC")  /SOURCE2; %* hcc labels;

Art, CEO, AnalystFinder.com

Re: Physical file does not exist (INLCUDE MACRO)

galtay — Sun, 26 Feb 2017 21:16:38 GMT

Thanks for the reply Art. Adding the double quotes seems to prevent SAS from appending the extension ".sas",

 1          OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
 61         
 62              %LET LABELMAC=V22H79L1;
 63              %INCLUDE IN0("&LABELMAC")  /SOURCE2; %* hcc labels;
 WARNING: Physical file does not exist, /folders/myfolders/sasuser.v94/cms_hcc_2017/V22H79L1.
 ERROR: Cannot %INCLUDE member V22H79L1 in the aggregate IN0.
 64         
 65         OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
 78

Re: Physical file does not exist (INLCUDE MACRO)

art297 — Sun, 26 Feb 2017 21:20:39 GMT

Again, just guessing, does this work?:

%LET LABELMAC=V22H79L1;
%INCLUDE IN0("&LABELMAC..sas")  /SOURCE2; %* hcc labels;

Art, CEO, AnalystFinder.com

Re: Physical file does not exist (INLCUDE MACRO)

galtay — Sun, 26 Feb 2017 21:31:34 GMT

that does work, but (according to the error message) we shouldn't have to add the quotes right? It seems like SAS is performing some kind of search in the error below and appending the ".sas" characters to the LABELMAC variable and not finding the file that does exist. I'm hoping for a solution that doesn't involve me editing all of the SAS code that uses this pattern 🙂

 1          OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
 61         
 62              %LET LABELMAC=V22H79L1;
 63              %INCLUDE IN0(&LABELMAC)  /SOURCE2; %* hcc labels;
 WARNING: Physical file does not exist, /folders/myfolders/sasuser.v94/cms_hcc_2017/V22H79L1.sas.
 ERROR: Cannot %INCLUDE member V22H79L1 in the aggregate IN0.
 64         
 65         OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
 78

Re: Physical file does not exist (INLCUDE MACRO)

galtay — Sun, 26 Feb 2017 21:40:34 GMT

The full code ...

Main program file

 LIBNAME LIBRARY "/folders/myfolders/sasuser.v94/cms_hcc_2017";
 FILENAME IN0 "/folders/myfolders/sasuser.v94/cms_hcc_2017";
 LIBNAME  IN1 "/folders/myfolders/sasuser.v94/cms_hcc_2017/input";
 LIBNAME  IN2 "/folders/myfolders/sasuser.v94/cms_hcc_2017/input";
 LIBNAME  INCOEF "/folders/myfolders/sasuser.v94/cms_hcc_2017";
 LIBNAME  OUT "/folders/myfolders/sasuser.v94/cms_hcc_2017/output";

 %* include main macro;
 %INCLUDE IN0('V2216O2M.sas')/SOURCE2;

 %V2216O2M(INP      =IN1.PERSON,
           IND      =IN2.DIAG,
           OUTDATA  =OUT.PERSON,
           IDVAR    =HICNO,
           KEEPVAR  =HICNO &INPUTVARS &SCOREVARS &DEMVARS 
                     &HCCV22_list79 &CCV22_list79, 
           SEDITS   =1,
           DATE_ASOF="1FEB2016"D);

Main macro file

 %MACRO V2216O2M(INP=, IND=, OUTDATA=, IDVAR=, KEEPVAR=, SEDITS=,
                 DATE_ASOF=, DATE_ASOF_EDIT=,
                 FMNAME9=V22Y15RC, FMNAME0=V22Y16RC,
                 AGEFMT9=I9AGEY15MCE, SEXFMT9=I9SEXY15MCE,
                 AGEFMT0=I0AGEY16MCE, SEXFMT0=I0SEXY16MCE, DF=1,
                 AGESEXMAC=AGESEXV2, EDITMAC9=V22I9ED1,
                 EDITMAC0=V22I0ED1, LABELMAC=V22H79L1,
                 HIERMAC=V22H79H1, SCOREMAC=SCOREVAR);


 %**********************************************************************
 * step1: include external macros
 **********************************************************************;
 %IF "&AGESEXMAC" ne "" %THEN %DO;
     %INCLUDE IN0(&AGESEXMAC) /SOURCE2; %* create demographic variables;
 %END;
 %IF "&EDITMAC9" ne "" %THEN %DO;
     %INCLUDE IN0(&EDITMAC9)   /SOURCE2; %* perform edits on ICD9;
 %END;
 %IF "&EDITMAC0" ne "" %THEN %DO;
     %INCLUDE IN0(&EDITMAC0)   /SOURCE2; %* perform edits on ICD10;
 %END;
 %IF "&LABELMAC" ne "" %THEN %DO;
     %INCLUDE IN0(&LABELMAC)  /SOURCE2; %* hcc labels;
 %END;
 %IF "&HIERMAC" ne "" %THEN %DO;
     %INCLUDE IN0(&HIERMAC)   /SOURCE2; %* hierarchies;
 %END;
 %IF "&SCOREMAC" ne "" %THEN %DO;
     %INCLUDE IN0(&SCOREMAC)  /SOURCE2; %* calculate score variable;
 %END;

Re: Physical file does not exist (INLCUDE MACRO)

Tom — Sun, 26 Feb 2017 21:46:25 GMT

Are you sure that you actually have that file in that directory?

Is it possible that you created a file named

V22H79L1.SAS

instead?

Unix filenames are case sensitive.

Also watch out for files without the extension. In Unix a period is just a character so you could have a file named 'V22H79L1' and a different file named 'V22H79L1.'.

Re: Physical file does not exist (INLCUDE MACRO)

galtay — Sun, 26 Feb 2017 21:52:22 GMT

Re: Physical file does not exist (INLCUDE MACRO)

galtay — Sun, 26 Feb 2017 21:55:21 GMT

the previous image looked pretty small on my screen, here's a zoom in.

Re: Physical file does not exist (INLCUDE MACRO)

Tom — Sun, 26 Feb 2017 22:00:02 GMT

Also watch out for blanks. Your macro variable might have embedded blanks (or other hidden characters).

%put |&LABELMAC|;

Or even the filename could have blanks or other strange characters. This data step should find the names of every file in the folder that IN0 points to.

data names ;
  length fname filename $200;
  infile in0('*') filename=fname;
  input ;
  if fname ne lag(fname);
  filename = fname ;
  len = length(filename);
run;

Re: Physical file does not exist (INLCUDE MACRO)

galtay — Sun, 26 Feb 2017 23:33:54 GMT

Re: Physical file does not exist (INLCUDE MACRO)

galtay — Sun, 26 Feb 2017 22:10:04 GMT

the trimmed version doesn't seem to help. thanks for the code snippet to list the files in the directory. it failed when it encountered directories in the directory it was searching, but I'm pretty sure the files are named correctly

  1          OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
 61         
 62         %put |&LABELMAC|;
 |V22H79L1|
 63              %INCLUDE IN0(&LABELMAC)  /SOURCE2; %* hcc labels;
 WARNING: Physical file does not exist, /folders/myfolders/sasuser.v94/cms_hcc_2017/V22H79L1.sas.
 ERROR: Cannot %INCLUDE member V22H79L1 in the aggregate IN0.
 64         
 65         OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
 78

Re: Physical file does not exist (INLCUDE MACRO)

galtay — Sun, 26 Feb 2017 22:25:27 GMT

Thanks for the input folks. I think I'll just start putting the variables in quotes with the file extension as that it the only thing thats worked so far. I'm attempting to port this to Python and just need a baseline to see if a small sample produces the same results. Still curious as to why the warning prints the correct file name but then cant include it, but hey, that's life. Appreciate the help!

Re: Physical file does not exist (INLCUDE MACRO)

Tom — Sun, 26 Feb 2017 22:52:11 GMT

You need to make the whole filename be in lowercase, not just the .sas extension.

Run this little test program.

data _null_;
  length filename filepath $200;
  do filename='TESTFILE.sas','testfile.sas','TESTFILE.SAS';
    filepath = catx('/',pathname('work'),filename);
    file dummy filevar=filepath ;
     put '%PUT I am from ' filename 'file.;' ;
  end;
run;

filename test1 "%sysfunc(pathname(work))";
%include test1(testfile);
%INCLUDE TEST1(TESTFILE);
%include test1('testfile.sas');
%INCLUDE TEST1('TESTFILE.sas');
%INCLUDE TEST1('TESTFILE.SAS');

Same thing would apply to and data files that are porting from the IBM mainframe world. Only use '.dat' extension instead of '.sas' extension. So a program like this

data want ;
   infile fileref(DEMOG);
   input id $ sex $ age ;
run;

would look for a file named 'demog.dat' in the directory that FILEREF point to.

Re: Physical file does not exist (INLCUDE MACRO)

galtay — Sun, 26 Feb 2017 22:56:28 GMT

ahhh ha! that seems to have done the trick. thank you sas ninja. the comments at the tops of these files are a kind of fascinating look into the history of computing.

Re: Physical file does not exist (INLCUDE MACRO)

Tom — Sun, 26 Feb 2017 23:18:07 GMT

I opened a ticket with SAS Support to let them know that the WARNING message was displaying the wrong physical filename.

Re: Physical file does not exist (INLCUDE MACRO)

jreancsu — Tue, 20 Feb 2018 17:19:39 GMT

Any chance you could post your PERSON and DIAG files and where they go? Also, do you just smash the run button the v2217o1p.sas file directly? It looks like the call to the main macro supplies default parameter values.

%V2217O1M(INP      =IN1.PERSON,
           IND      =IN2.DIAG,
           OUTDATA  =OUT.PERSON,
           IDVAR    =HICNO,
           KEEPVAR  =HICNO &INPUTVARS &SCOREVARS &DEMVARS 
                     &HCCV22_list79 &CCV22_list79, 
           SEDITS   =1,
           DATE_ASOF="1FEB2018"D);

This is my setup:

When I run it, I just get a bunch of "ERROR 180-322: Statement is not valid or it is used out of proper order." messages, and one "ERROR: File OUT.PERSON.DATA does not exist" message. Do you have to make the output file yourself?

Re: Physical file does not exist (INLCUDE MACRO)

jreancsu — Tue, 20 Feb 2018 20:29:46 GMT

I got the following working (omg finally):

I think the lowercase column names matter.
There was also a random character at the bottom of the scorevar.sas file that had some messed up encoding.

All-in-all, it was a pretty absurd amount of trial and error to get this thing working. It'd be nice if CMS would put out a ready-to-run SAS program along with example input/output data. Special shout-out to the National Bureau of Economic Research for putting this page together with examples: http://www.nber.org/data/cms-risk-adjustment.html