DATA Step, Macro, Functions and more

reading files in a directory - help with space in directory name...

Accepted Solution Solved
Reply
Contributor
Posts: 66
Accepted Solution

reading files in a directory - help with space in directory name...

this is the code i am using to read a couple of text files but i keep getting a message telling me that "the system cannot find the path specified". from what i have gathered, i think my problem has to do with the spaces in the directory location.  could anyone point me in direction to resolve this problem?

code:

filename indata pipe '"dir C:\Documents and Settings\agautam\Desktop\Selfgen\Model\Commercial\Commercial_Model_Input_Output_Data\CED_2011\Preliminary\Floorspace\*.txt" /b' ;

data Floor_sp ;

infile indata ;

input f2r :$200. ;

fil2read = 'C:\Documents and Settings\agautam\Desktop\Selfgen\Model\Commercial\Commercial_Model_Input_Output_Data\CED_2011\Preliminary\Floorspace\' || f2r ;

infile dummy filevar = fil2read end = done firstobs = 2 ;

do while (not done) ;

    input @5  CZ  2. 

          @10 BTYPE  $14.

          @27 YEAR  4.

          @35 HISTF  8.4

          @47 BCASTF  8.4

          @59 ADDITIONS  8.4    ;

      output ;

end ;

run ;


Accepted Solutions
Solution
‎09-12-2011 06:03 PM
Contributor
Posts: 43

reading files in a directory - help with space in directory name...

Yes, in your original program, move the first double-quote immediately before the file path; you have it in front of the dir command.  In other words, change it to this:

filename indata pipe 'dir "C:\Documents and Settings\agautam\Desktop\Selfgen\Model\Commercial\Commercial_Model_Input_Output_Data\CED_2011\Preliminary\Floorspace\*.txt" /b' ;

Also, I suggest you use the TRUNCOVER option on your INFILE statement, and define your FIL2READ variable to a specific length (like $200).

View solution in original post


All Replies
Respected Advisor
Posts: 3,799

Re: reading files in a directory - help with space in directory name...

Posted in reply to asishgautam

If there is space in output from PIPE the input statement

input f2r :$200. ;

will only read to the first blank.  The colon is for LIST input.  Remove the colon and add TRUNCOVER to the INFILE INDATA.

Contributor
Posts: 66

reading files in a directory - help with space in directory name...

Posted in reply to data_null__

okay this is what i tried but i keep getting the same message.

Stderr output:

The system cannot find the path specified.

NOTE: 0 records were read from the infile INDATA.

NOTE: The data set WORK.FLOOR_SP has 0 observations and 8 variables.

NOTE: DATA statement used (Total process time):

      real time           0.04 seconds

      cpu time            0.03 seconds

data Floor_sp ;

infile indata ;

input f2r $200. TRUNCOVER ;

fil2read = 'C:\Documents and Settings\agautam\Desktop\Selfgen\Model\Commercial\Commercial_Model_Input_Output_Data\CED_2011\Preliminary\Floorspace\' || f2r ;

infile dummy filevar = fil2read end = done firstobs = 2 ;

do while (not done) ;

    input @5  CZ  2. 

          @10 BTYPE  $14.

          @27 YEAR  4.

          @35 HISTF  8.4

          @47 BCASTF  8.4

          @59 ADDITIONS  8.4    ;

      output ;

end ;

run ;

Respected Advisor
Posts: 3,799

reading files in a directory - help with space in directory name...

Posted in reply to asishgautam

You will have to resolve the problem with the values of the variable FIL2READ.  Use PUTLOG to inspect the value.  Copy the value from the log and see if you can use it to open the file.

Super User
Super User
Posts: 7,039

reading files in a directory - help with space in directory name...

Posted in reply to asishgautam

Take the DIR command out the double quotes you have around the filename in your DOS command.

filename indata pipe 'dir "C:\Documents and Settings\agautam\Desktop\Selfgen\Model\Commercial\Commercial_Model_Input_Output_Data\CED_2011\Preliminary\Floorspace\*.txt" /b' ;

Your string is very long maybe it is wrapping in the editor?  Try breaking it into pieces using macros variables.  Use dquotes around the command and double the quotes that you want to send to DOS.

%let root=C:\Documents and Settings\agautam\Desktop\Selfgen\Model\Commercial\Commercial_Model_Input_Output_Data;

%let project=CED_2011\Preliminary\Floorspace;

filename indata pipe "dir "&root\&project\*.txt"" /b" ;

TRUNCOVER is an option on the INFILE statement, not the INPUT statement. 

Add the TRUNCOVER option to both of your INFILE statements.

Trusted Advisor
Posts: 1,301

reading files in a directory - help with space in directory name...

Posted in reply to asishgautam

The pipe is not necessary for what you are trying to accomplish.

filename indata 'C:\Documents and Settings\agautam\Desktop\Selfgen\Model\Commercial\Commercial_Model_Input_Output_Data\CED_2011\Preliminary\Floorspace\*.txt';

data Floor_sp;

infile indata filevar = fil2read end = done firstobs = 2 ;

    input @5  CZ  2.

          @10 BTYPE  $14.

          @27 YEAR  4.

          @35 HISTF  8.4

          @47 BCASTF  8.4

          @59 ADDITIONS  8.4    ;

run ;

Contributor
Posts: 66

reading files in a directory - help with space in directory name...

i used the program you listed and saw quite a few errors..i am posting them below:

ERROR: Invalid physical name.

fil2read=  done=0 CZ=. BTYPE=  YEAR=. HISTF=. BCASTF=. ADDITIONS=. _ERROR_=1 _N_=1

NOTE: The SAS System stopped processing this step because of errors.

WARNING: The data set WORK.FLOOR_SP may be incomplete.  When this step was stopped there were 0

         observations and 6 variables.

WARNING: Data set WORK.FLOOR_SP was not replaced because this step was stopped.

NOTE: DATA statement used (Total process time):

      real time           0.03 seconds

      cpu time            0.00 seconds

PROC Star
Posts: 7,468

reading files in a directory - help with space in directory name...

Posted in reply to asishgautam

Back to your original code, I have a suggestion, a comment and a question.

The suggestion: try it with double quotes at the start and end, and double double quotes around the file name.  i.e.

filename indata pipe "dir ""C:\Documents and Settings\agautam\Desktop\Selfgen\Model\Commercial\Commercial_Model_Input_Output_Data\CED_2011\Preliminary\Floorspace\*.txt"" /b ;

The comment: you are likely to run into problems with your next use of the directory as double quotes will likely be needed.

The question: what are you trying to do?  At first I thought you were trying to get a file (via the piped dir), then getting a 2nd file name from that file, then trying to read the file.  Is that correct or do you just want to read the file?

Contributor
Posts: 66

reading files in a directory - help with space in directory name...

i have 3 text files that i am trying to read.  they are identical files (in terms of layout) but differe by one category. the first input should get me a list of files and then next input section "should" read in each file.

Solution
‎09-12-2011 06:03 PM
Contributor
Posts: 43

reading files in a directory - help with space in directory name...

Yes, in your original program, move the first double-quote immediately before the file path; you have it in front of the dir command.  In other words, change it to this:

filename indata pipe 'dir "C:\Documents and Settings\agautam\Desktop\Selfgen\Model\Commercial\Commercial_Model_Input_Output_Data\CED_2011\Preliminary\Floorspace\*.txt" /b' ;

Also, I suggest you use the TRUNCOVER option on your INFILE statement, and define your FIL2READ variable to a specific length (like $200).

PROC Star
Posts: 7,468

reading files in a directory - help with space in directory name...

I actually like fried eggs' suggestion, but double quote the directory and leave off the filevar option.  e.g.,

filename indata "C:\art\test folder\*.txt";

data Floor_sp;

  infile indata end = done firstobs = 2 ;

  input x;

run ;

Contributor
Posts: 43

reading files in a directory - help with space in directory name...

Yes, that would work, except that the FIRSTOBS option would only skip the first line of the first file.  I'm assuming the OP wants to skip the header record on each file, hence using the FILEVAR makes sense.

Contributor
Posts: 66

reading files in a directory - help with space in directory name...

thank you so much!!!

PROC Star
Posts: 7,468

reading files in a directory - help with space in directory name...

Posted in reply to asishgautam

Glad to hear that you solved your problem, but I still think that my (extremely slight) modification of FriedEgg's code would be your easiest and least troublesome way of solving that particular problem.

And, while you apparently didn't confront the additional problem, I did, and for those who read this thread and can't figure out why a piped dir isn't working for them, they should take a look at: http://support.sas.com/kb/41/863.html

There is a problem with reading pipes on 64-bit window's machines.  One that can be circumvented, but a problem nonetheless.

Trusted Advisor
Posts: 1,301

reading files in a directory - help with space in directory name...

Posted in reply to asishgautam

/* create some dummy files */

data _null_;

do i=1 to 3;

  call execute('filename file' || strip(i) || ' "/home/friedegg/file' || strip(i) || '.txt";');

  call execute('data _null_;');

  call execute(' file file' || strip(i) || ';');

  call execute(' put "a b c d e";');

  call execute(' put "f g h i j";');

  call execute('run;');

end;

run;

filename indata '/home/mkastin/file*.txt';

data want;

length _file _filex $512;

infile indata end=done filename=_file;

input (blah1-blah5) (:$1.);

retain _filex;

if _filex ne _file then

  do;

   _filex=_file;

   delete;

  end;

drop _:;

run;

/* remove this junk */

data _null_;

do i=1 to 3;

  call execute('x "rm -f /home/friedegg/file' || strip(i) || '.txt";');

end;

run;

🔒 This topic is solved and locked.

Need further help from the community? Please ask a new question.

Discussion stats
  • 16 replies
  • 5338 views
  • 2 likes
  • 6 in conversation