reading files in a directory - help with space in directory name...

[asishgautam]

this is the code i am using to read a couple of text files but i keep getting a message telling me that "the system cannot find the path specified". from what i have gathered, i think my problem has to do with the spaces in the directory location.  could anyone point me in direction to resolve this problem?

code:

filename indata pipe '"dir C:\Documents and Settings\agautam\Desktop\Selfgen\Model\Commercial\Commercial_Model_Input_Output_Data\CED_2011\Preliminary\Floorspace\*.txt" /b' ;

data Floor_sp ;

infile indata ;

input f2r :$200. ;

fil2read = 'C:\Documents and Settings\agautam\Desktop\Selfgen\Model\Commercial\Commercial_Model_Input_Output_Data\CED_2011\Preliminary\Floorspace\' || f2r ;

infile dummy filevar = fil2read end = done firstobs = 2 ;

do while (not done) ;

    input @5  CZ  2. 

          @10 BTYPE  $14.

          @27 YEAR  4.

          @35 HISTF  8.4

          @47 BCASTF  8.4

          @59 ADDITIONS  8.4    ;

      output ;

end ;

run ;

[data_null__]

If there is space in output from PIPE the input statement

input f2r :$200. ;

will only read to the first blank.  The colon is for LIST input.  Remove the colon and add TRUNCOVER to the INFILE INDATA.

[asishgautam]

okay this is what i tried but i keep getting the same message.

Stderr output:

The system cannot find the path specified.

NOTE: 0 records were read from the infile INDATA.

NOTE: The data set WORK.FLOOR_SP has 0 observations and 8 variables.

NOTE: DATA statement used (Total process time):

      real time           0.04 seconds

      cpu time            0.03 seconds

data Floor_sp ;

infile indata ;

input f2r $200. TRUNCOVER ;

fil2read = 'C:\Documents and Settings\agautam\Desktop\Selfgen\Model\Commercial\Commercial_Model_Input_Output_Data\CED_2011\Preliminary\Floorspace\' || f2r ;

infile dummy filevar = fil2read end = done firstobs = 2 ;

do while (not done) ;

    input @5  CZ  2. 

          @10 BTYPE  $14.

          @27 YEAR  4.

          @35 HISTF  8.4

          @47 BCASTF  8.4

          @59 ADDITIONS  8.4    ;

      output ;

end ;

run ;

[data_null__]

You will have to resolve the problem with the values of the variable FIL2READ.  Use PUTLOG to inspect the value.  Copy the value from the log and see if you can use it to open the file.

[Tom]

Take the DIR command out the double quotes you have around the filename in your DOS command.

filename indata pipe 'dir "C:\Documents and Settings\agautam\Desktop\Selfgen\Model\Commercial\Commercial_Model_Input_Output_Data\CED_2011\Preliminary\Floorspace\*.txt" /b' ;

Your string is very long maybe it is wrapping in the editor?  Try breaking it into pieces using macros variables.  Use dquotes around the command and double the quotes that you want to send to DOS.

%let root=C:\Documents and Settings\agautam\Desktop\Selfgen\Model\Commercial\Commercial_Model_Input_Output_Data;

%let project=CED_2011\Preliminary\Floorspace;

filename indata pipe "dir "&root\&project\*.txt"" /b" ;

TRUNCOVER is an option on the INFILE statement, not the INPUT statement. 

Add the TRUNCOVER option to both of your INFILE statements.

[FriedEgg]

The pipe is not necessary for what you are trying to accomplish.

filename indata 'C:\Documents and Settings\agautam\Desktop\Selfgen\Model\Commercial\Commercial_Model_Input_Output_Data\CED_2011\Preliminary\Floorspace\*.txt';

data Floor_sp;

infile indata filevar = fil2read end = done firstobs = 2 ;

    input @5  CZ  2.

          @10 BTYPE  $14.

          @27 YEAR  4.

          @35 HISTF  8.4

          @47 BCASTF  8.4

          @59 ADDITIONS  8.4    ;

run ;

[asishgautam]

i used the program you listed and saw quite a few errors..i am posting them below:

ERROR: Invalid physical name.

fil2read=  done=0 CZ=. BTYPE=  YEAR=. HISTF=. BCASTF=. ADDITIONS=. _ERROR_=1 _N_=1

NOTE: The SAS System stopped processing this step because of errors.

WARNING: The data set WORK.FLOOR_SP may be incomplete.  When this step was stopped there were 0

         observations and 6 variables.

WARNING: Data set WORK.FLOOR_SP was not replaced because this step was stopped.

NOTE: DATA statement used (Total process time):

      real time           0.03 seconds

      cpu time            0.00 seconds

[art297]

Back to your original code, I have a suggestion, a comment and a question.

The suggestion: try it with double quotes at the start and end, and double double quotes around the file name.  i.e.

filename indata pipe "dir ""C:\Documents and Settings\agautam\Desktop\Selfgen\Model\Commercial\Commercial_Model_Input_Output_Data\CED_2011\Preliminary\Floorspace\*.txt"" /b ;

The comment: you are likely to run into problems with your next use of the directory as double quotes will likely be needed.

The question: what are you trying to do?  At first I thought you were trying to get a file (via the piped dir), then getting a 2nd file name from that file, then trying to read the file.  Is that correct or do you just want to read the file?

[asishgautam]

i have 3 text files that i am trying to read.  they are identical files (in terms of layout) but differe by one category. the first input should get me a list of files and then next input section "should" read in each file.

[BobD · Accepted solution]

Yes, in your original program, move the first double-quote immediately before the file path; you have it in front of the dir command.  In other words, change it to this:

filename indata pipe 'dir "C:\Documents and Settings\agautam\Desktop\Selfgen\Model\Commercial\Commercial_Model_Input_Output_Data\CED_2011\Preliminary\Floorspace\*.txt" /b' ;

Also, I suggest you use the TRUNCOVER option on your INFILE statement, and define your FIL2READ variable to a specific length (like $200).

[art297]

I actually like fried eggs' suggestion, but double quote the directory and leave off the filevar option.  e.g.,

filename indata "C:\art\test folder\*.txt";

data Floor_sp;

  infile indata end = done firstobs = 2 ;

  input x;

run ;

[BobD]

Yes, that would work, except that the FIRSTOBS option would only skip the first line of the first file.  I'm assuming the OP wants to skip the header record on each file, hence using the FILEVAR makes sense.

[asishgautam]

thank you so much!!!

[art297]

Glad to hear that you solved your problem, but I still think that my (extremely slight) modification of FriedEgg's code would be your easiest and least troublesome way of solving that particular problem.

And, while you apparently didn't confront the additional problem, I did, and for those who read this thread and can't figure out why a piped dir isn't working for them, they should take a look at: http://support.sas.com/kb/41/863.html

There is a problem with reading pipes on 64-bit window's machines.  One that can be circumvented, but a problem nonetheless.

[FriedEgg]

/* create some dummy files */

data _null_;

do i=1 to 3;

  call execute('filename file' || strip(i) || ' "/home/friedegg/file' || strip(i) || '.txt";');

  call execute('data _null_;');

  call execute(' file file' || strip(i) || ';');

  call execute(' put "a b c d e";');

  call execute(' put "f g h i j";');

  call execute('run;');

end;

run;

filename indata '/home/mkastin/file*.txt';

data want;

length _file _filex $512;

infile indata end=done filename=_file;

input (blah1-blah5) (:$1.);

retain _filex;

if _filex ne _file then

  do;

   _filex=_file;

   delete;

  end;

drop _:;

run;

/* remove this junk */

data _null_;

do i=1 to 3;

  call execute('x "rm -f /home/friedegg/file' || strip(i) || '.txt";');

end;

run;