topic Re: how to read filenames of a directory into a file in SAS Programming

how to read filenames of a directory into a file

deleted_user — Wed, 28 Jan 2009 03:12:16 GMT

for example, there are many files in one directory, how can i read files' name into another file?
thanks

Re: how to read filenames of a directory into a file

andreas_lds — Wed, 28 Jan 2009 07:39:57 GMT

Is this a sas problem?

Have a look at the dopen function: http://support.sas.com/onlinedoc/913/getDoc/de/lrdict.hlp/a000209538.htm

Re: how to read filenames of a directory into a file

LinusH — Wed, 28 Jan 2009 12:25:21 GMT

I don't think that DOPEN or any other directory SAS function will do the job. Here's a sample that uses FILENAME PIPE with an OS directory listing command (such as dir or ls).

http://support.sas.com/kb/24/820.html

/Linus

Re: how to read filenames of a directory into a file

data_null__ — Wed, 28 Jan 2009 14:55:25 GMT

> I don't think that DOPEN or any other directory SAS
> function will do the job.

You would need to combine DOPEN with DNUM DREAD and perhaps DINFO but it can be done with relative ease.

I prefer the INFILE PIPE method you eluded to but find that on unix the FIND command is much better than LS as FIND provides many more options with regard to file info and format of returned output.

Also also on UNIX always point directly to the command as /usr/bin/ls or where ever because an alias can cause unpleasant results.

Re: how to read filenames of a directory into a file

deleted_user — Wed, 28 Jan 2009 18:55:21 GMT

i am using windows xp and only have sas and matlab. i tried

filename DIRLIST pipe 'dir "C:\Users\rong\Documents\high-frequency\2006\20061
1\200611sh_csv\sh\20061107\" /s';

data dirlist ;
length buffer $256 ;

infile dirlist length=reclen ;

input buffer $varying256. reclen ;
run ;

but only get a file which contains "volume in C has no drive" and "volume series num is***"

can anyone write a easy program for me?

Re: how to read filenames of a directory into a file

data_null__ — Wed, 28 Jan 2009 19:20:38 GMT

Do you also see

[pre]
Stderr output:
File Not Found
[/pre]

in the log. Your program looks OK I suspect an error in the path.

Re: how to read filenames of a directory into a file

deleted_user — Wed, 28 Jan 2009 19:30:45 GMT

i used above program again, then this time, it works!!!
interesting
:)))
thanks for all

Re: how to read filenames of a directory into a file

darrylovia — Fri, 30 Jan 2009 14:32:46 GMT

I have a macro that does exactly what you need. Just pass the folder name ending with '/' and a data set name and the entire contents of the folder will be outputted to your data set

Darryl

/*******************************************************************************************************************************************************/
/* Program Name: get_folder_info (macro of the same name) */
/* */
/* Description: SAS Utility: */
/* Dump the contents of folder into a SAS data set (user defined) */
/* Output data set contains the input folder name, all the files inside the folder, and */
/* whether those files are a directory or file. */
/* */
/* If a file is inputted the program stops with a message in the log stating that the input */
/* directory is not a directory. */
/* */
/* */
/* Macro Input Parameters: dir = Unquoted Full Directory Path ending in \. */
/* foldercontents_dataset = SAS data set containing the directory information. */
/* */
/*******************************************************************************************************************************************************/

%macro get_folder_info(dir=,output_ds=);

%put &dir;
/* Check for Existence of directory path */
%if %sysfunc(fileexist(&dir)) %then %do;

/* Assigns the fileref of mydir to the directory and opens the directory */
/* Assigns the fileref of myfile to the directory and opens the file */
/* Assign using file i/o functions and directory i/o functions, will then compare the output of the functions */
%let dirrf=md;
%let filrf=mf;
%let dc=%sysfunc(filename(dirrf,&dir));
%let fc=%sysfunc(filename(filrf,&dir));
%let did=%sysfunc(dopen(&dirrf)); /* opens as a directory, 0 means could not be opened
note: unable to open a file with dopen */
%let fid=%sysfunc(fopen(&filrf)); /* opens as a file, 0 means could not be opened
note: unable to open a directory with fopen */
/* Returns the number of members in the directory, */
%if &did > 0 %then %do;
%let memcnt=%sysfunc(dnum(&did));
%put memcnt=&memcnt;
%end;

%* QA: put macro variable values into log;
%put QA: dc=&dc fc=&fc did=&did fid=&fid;

%if &fid>0 and &did=0 %then %put &dir is file not a directory;

%if &did > 0 %then %do;

%do i=1 %to &memcnt;
%let dir&i=&dir.%qsysfunc(dread(&did,&i)); %* create full path of directory/file;
%let mem&i=%qsysfunc(dread(&did,&i));

%* Assign filerefs;
%let dirrf&i=md&i;
%let filrf&i=mf&i;
%let dc&i=%sysfunc(filename(dirrf&i,&&dir&i));
%let fc&i=%sysfunc(filename(filrf&i,&&dir&i));

%* open files;
%let did&i=%sysfunc(dopen(&&dirrf&i));
%let fid&i=%sysfunc(fopen(&&filrf&i));

%* QA: put macro variable values into log;
%put QA: dirrf=&&dirrf&i filrf=&&filrf&i did=&&did&i fid=&&fid&i name=&&dir&i;

%* close files;
%let dc&i=%sysfunc(dclose(&&did&i));
%let fc&i=%sysfunc(fclose(&&fid&i));

%* clear filerefs;
filename md&i clear;
filename mf&i clear;

%end;

%* create output table;
data &output_ds;
attrib DirName length=$512 label='Name of Input Directory'
FileName length=$1024 label='Full File Name with Path of Directory Member'
MemberName length=$512 label='File Name of Directory Member'
FileType length=$9 label='Directory Member: File or Directory';

%do i=1 %to &memcnt;
DirName="&dir";
FileName="&&dir&i";
MemberName="&&mem&i";
if (&&did&i=0 and &&fid&i>0) then FileType='File';
else FileType='Directory';
output;
%end;
run;

title "Contents of &dir";
proc print data=FolderContents;
run;
title;

%end;
/* Closes the input directory/file. All opened files must be closed. */
%let dc=%sysfunc(dclose(&did));
%let fc=%sysfunc(fclose(&fid));

%* clear filerefs;
filename md clear;
filename mf clear;

%end;
%else %do;
%put ERROR: Folder &dir Not Found;
%end;

%mend get_folder_info;
* END OF SAS PGM;

Re: how to read filenames of a directory into a file

TimArm — Wed, 21 Nov 2012 14:41:37 GMT

Hello Darryl - I just found your macro get_folder_info and I really liked it. However, I wanted file size too, so I tweaked it...

...I hope you don't mind. I really like the fact that exactly the same code runs on my AIX box and Windows box.

Tim

/*******************************************************************************************************************************************************/

/* Program Name: get_folder_info (macro of the same name) */

/* */

/* Description: SAS Utility: */

/* Dump the contents of folder into a SAS data set (user defined) */

/* Output data set contains the input folder name, all the files inside the folder, and */

/* whether those files are a directory or file. */

/* */

/* If a file is input, the program stops with a message in the log stating that the input */

/* directory is not a directory. */

/* */

/* Macro Input Parameters: dir = Unquoted Full Directory Path ending in \ or /. */

/* output_ds = SAS data set containing the directory information. */

/* */

/*******************************************************************************************************************************************************/

%macro get_folder_info(dir=,output_ds=);

%*put &dir;

** Check for Existence of directory path ;

%if %sysfunc(fileexist(&dir)) %then %do;

* create output table;

data &output_ds (drop=t_:);

attrib DirName length=$512. label='Name of Input Directory'

FileName length=$1024. label='Full File Name with Path of Directory Member'

MemberName length=$512. label='File Name of Directory Member'

FileType length=$9. label='Directory Member: File or Directory'

FileSize length=8. label='File Size (Bytes)' format=COMMA14.;

* Try to open the input directory;

t_rc = filename('dref',"&dir.");

t_topdir = dopen('dref');

if t_topdir > 0 then do;

* Top directory was opened, so get number of members;

t_memct = dnum(t_topdir);

do t_i=1 to t_memct;

DirName="&dir.";

MemberName=dread(t_topdir,t_i);

FileName=cats("&dir.",MemberName); * create full path of directory/file;

* Get fileref;

t_ref=filename('ref',FileName);

* open file/folder;

t_did=dopen('ref');

t_fid=fopen('ref');

* Get the size;

if t_fid>0 then FileSize=finfo(t_fid,'File Size (bytes)');

else FileSize=.;

if (t_did=0 and t_fid>0) then FileType='File';

else FileType='Directory';

output;

* close files;

t_dc=dclose(t_did);

t_fc=fclose(t_fid);

end;

else put "Folder &dir. could not be opened";

run;

%end;

%else %do;

%put "Folder &dir. does not exist";

%end;

%mend get_folder_info;

Re: how to read filenames of a directory into a file

htalbott — Mon, 14 Mar 2022 20:42:43 GMT

TimArm, Thanks for this. I made some modifications to get file modified and the file created times, which was working well in SAS 9.4 TS Level 1M3 X64_DSRV12 platform. Now I'm in SAS 9.4 TS Level 1M7 X64_10PRO platform and this macro no longer pulls the file size, modified, and created dates. Any ideas on what is causing the finfo-derived information to no longer work?

/* Program Name: get_folder_info (macro of the same name)                                                                           
/* Description: SAS Utility:                                                                                
/*              Dump the contents of folder into a SAS data set (user defined)                                                                     
/*              Output data set  contains the input folder name, all the files inside the folder, and                                                
/*              whether those files are a directory or file.                                                                              
/*                                                                               
/*              If a file is input, the program stops with a message in the log stating that the input                                         
/*              directory is not a directory.                                                                              
/* Macro Input Parameters: dir = Unquoted Full Directory Path ending in \ or /.                                                                        
/*                         output_ds = Desired name for SAS data set containing the directory information.                                           
/*****************************************************************************************************/

%macro get_folder_info(dir=,output_ds=);                                                                              

%*put &dir;
  ** Check for Existence of directory path ;
  %if %sysfunc(fileexist(&dir)) %then %do;
    * create output table;
    data &output_ds (drop = t_:);
         attrib DirName    length=$512.  label='Name of Input Directory'
		 		FileName   length=$1024. label='Full File Name with Path of Directory Member'
                MemberName length=$512.  label='File Name of Directory Member'
				FileCr     length=8.     label='Create Time' format = DATETIME16.
 				FileMod    length=8.     label='Last Modified' format = DATETIME16.
                FileType   length=$9.    label='Directory Member: File or Directory' 
                FileEx     length=$20.   label='File Extension' 
				FileSize   length=8.     label='File Size (bytes)' format=COMMA14.;
      * Try to open the input directory;
      t_rc = filename('dref',"&dir.");
      t_topdir = dopen('dref');
      if t_topdir > 0 then do;
        * Top directory was opened, so get number of members;
        t_memct = dnum(t_topdir);
        do t_i=1 to t_memct;
          DirName="&dir.";
          MemberName=dread(t_topdir,t_i);
          FileName=cats("&dir.",MemberName);  * create full path of directory/file;  
		  FileEx = scan(MemberName, -1, .); * find extension of file 
          * Get fileref;
          t_ref=filename('ref',FileName);       
          * open file/folder;
          t_did=dopen('ref');
          t_fid=fopen('ref');
		  * Get the modified date;
          if t_fid>0 then FileMod = input(finfo(t_fid,'Last Modified'), ANYDTDTM.);
          else FileMod=.;
		  * Get the created date;
          if t_fid>0 then FileCr = input(finfo(t_fid,'Create Time'), ANYDTDTM.);
          else FileCr=.;
          * Get the size;
          if t_fid>0 then FileSize=finfo(t_fid,'File Size (bytes)');
          else FileSize=.;
		  * Directory or file;
          if (t_did=0 and t_fid>0) then FileType='File';
          else FileType='Directory';
          output;
		  * close files;
          t_dc=dclose(t_did);                                                                              
          t_fc=fclose(t_fid);
        end;
      end;
      else put "Folder &dir. could not be opened";
    run;
  %end;
  %else %do;
      %put "Folder &dir. does not exist";
  %end;
%mend get_folder_info;

Re: how to read filenames of a directory into a file

Patrick — Mon, 14 Mar 2022 21:08:53 GMT

In your code change from...

to...

Not sure how this code would have worked in your previous environment without this backslash.

Btw: Suggest in the future to not add to old discussions but to create a new question instead where you then reference the old discussion. This will help to keep this forum more useful and easier to search (ideally: a single question and then an answer marked as solution).

Re: how to read filenames of a directory into a file

htalbott — Mon, 14 Mar 2022 21:51:18 GMT

@Patrick Thanks for your help! I'll be sure to make a new post next time, I'm still a beginner SAS user.

Re: how to read filenames of a directory into a file

Tom — Mon, 14 Mar 2022 21:52:11 GMT

I usually add \ in front of a command to disable aliases.

That way I don't need to know where in the search path the command actually lives.

infile "\ls -A /mydir" pipe truncover ;