topic Re: searching through a directory in SAS Procedures

searching through a directory

nickb — Thu, 05 Aug 2010 17:04:32 GMT

I recently posted a question on how to strip leading zeros so that I could build up a FTP put statement. It worked great but the first put statement referenced a file that doesn't exist. I was wondering if there is a way to grab a directory and put it into a data set? Once this is accomplished I can then join the valid files in the directory.

Nick

Re: searching through a directory

ArtC — Thu, 05 Aug 2010 17:29:18 GMT

Take a look at the FEXIST function. It can be used in the DATA step or with %SYSFUNC. There are other solutions as well, but check this one out first.

Re: searching through a directory

Robert_Bardos — Thu, 05 Aug 2010 17:31:42 GMT

Couldn't you just use the FILEEXIST function to verify the file's existence?

Robert

Re: searching through a directory

nickb — Thu, 05 Aug 2010 17:37:44 GMT

I'm not familiar with that? Will it give you the capability to continue on if the file doesn't exist?

Re: searching through a directory

sbb — Thu, 05 Aug 2010 17:39:59 GMT

The two functions have different purposes - yes, you can code a SAS program to provide condition path-execution depending on the results of the function call -- most often done with SAS macro language but conceivably possible with DATA step code that generates subsequent-executed SAS statements (other PROCs or DATA steps).

Scott Barry
SBBWorks, Inc.

Re: searching through a directory

Robert_Bardos — Thu, 05 Aug 2010 17:42:07 GMT

Sure, just include it in the flow of your datastep. Somewhat like

data _null_ ;
...
if fileexist(your_file)=1 then put 'put ....' ;
....
run;

Referring to your FTP problem, you will have to construct a variable containing the complete filename. In my example above that would be variable "your_file".

Robert

Re: searching through a directory

ArtC — Thu, 05 Aug 2010 17:43:59 GMT

Both the FEXIST and the FILEEXIST functions allow you to verify that a file (or even a directory) exists on your system. The function returns a value which you can test.

Re: searching through a directory

nickb — Thu, 05 Aug 2010 18:00:02 GMT

I have one hard file value and it isn't working properly. Also, there are 44,000 'put' statements that I need to verify on the directory.

data _null_ ;
set work.display_data end=lastobs ;
file max_pics;
if _n_ = 1 then put 'cd incoming/images' ;
idnum = input(person_id_nb,best.) ;
if fileexist ("\\myfile\61.jpg")=1
then put 'put \\myfilepath' idnum +(-1) '.jpg' ;
if lastobs then put 'quit' ;
run ;

Re: searching through a directory

Robert_Bardos — Thu, 05 Aug 2010 18:12:33 GMT

No, you just have two statements that verify all 44,000 filenames, namely:

[pre]
my_file = '\\my_file\'!!trim(put(id,best.-l))!!'.jpg' ;
if fileexist(my_file)=1 then put 'put ....' ;
[/pre]

Robert

Re: searching through a directory

nickb — Thu, 05 Aug 2010 18:29:19 GMT

still not working.

data _null_ ;
set work.display_data end=lastobs ;
file max_pics;
if _n_ = 1 then put 'cd incoming/images' ;
idnum = input(person_id_nb,best.) ;
my_file= '\\batgirl\webroot_root$\JRun4\servers\cfusion2\cfusion.ear\cfusion.war\data\images\'!!trim(put(idnum,best.-1))!!'.jpg' ;
if fileexist (my_file)=1 then put my_file;
if lastobs then put 'quit' ;
run ;

Re: searching through a directory

Robert_Bardos — Thu, 05 Aug 2010 18:49:11 GMT

I see one typo in your my_file construction statement, where it should say "best.-L" not "best.-1". The -L suffix means "justified left" (although I think it is not really needed there).

As an aside: things like this should always be tested with a minimal setup.

Another point (and completely untested): you might need to quote my_file in the fileexist invocation, somewhat like

if fileexist("'"!!my_file!!"'") ....

Time to go home now ... kind regards
Robert

Re: searching through a directory

sbb — Thu, 05 Aug 2010 18:53:47 GMT

Also, you should test your code first by echoing it back to the SASLOG for your own desk-checking.....I don't see where you actually generate the FTP "put " command verb, only the file-name. Hopefully you are reviewing any FTP log output, right?

Scott Barry
SBBWorks, Inc.

Re: searching through a directory

nickb — Thu, 05 Aug 2010 18:59:28 GMT

I'm using a batch file for my SFTP. I plan on having this output go to a file that the batch will reference when it runs.

Re: searching through a directory

sbb — Thu, 05 Aug 2010 19:01:40 GMT

Okay, so you have a "cd" command, a "quit" command, but no "put" command (verb)? Only the file-name to be transfered is generated in the output file? Is that what you are trying to say?

Scott Barry
SBBWorks, Inc.

Re: searching through a directory

nickb — Thu, 05 Aug 2010 19:13:02 GMT

For the most part. My code is missing the 'put' statement and once I get this working, I will add it in. The put statement will be based on the records that are in a temp dataset (roughly 40,000 ids) and of those ids, I only want the ones that exist in my lookup directory location. Once the code is correct, I should have a put statement of ids that are in my temp data set and also exist in my lookup directory. Basically, I'm trying to build up a 'PUT' statement for student pictures and I only want the ids in my temp dataset and those that are located in the pics directory. I had the file built earlier but once the put statement encountered a file that didn't exist, it bombed.

Re: searching through a directory

Patrick — Thu, 05 Aug 2010 21:57:04 GMT

Hi

The code below should give you an idea of how you could proceed.

filename dirlist pipe 'dir c:\temp\*.jpg /b';

data dirlist;
infile dirlist delimiter='.';
input id:$32.;
run;

data have;
length id $32;
do id='1','10','42','43','44';
output;
end;
run;

data want;
/* declare hash table to look up ID from dirlist */
if _n_=1 then
do;
length id $32.;
declare hash h (dataset:'dirlist');
rc = h.defineKey('id');
rc = h.defineDone( );
call missing(id);
end;

set have;
if h.check()=0 then /* true if ID exists in lookup table */
do;
put 'ID ' id 'exists in the have table and as file under c:\temp';
end;
run;

For testing I've created two files under c:\temp : 2.jpg and 43.jpg

Running the code above "putted" the following line to the log:
"ID 43 exists in the have table and as file under c:\temp"

HTH
Patrick