topic Re: Help comparing file size of files in a directory. in SAS Programming

Help comparing file size of files in a directory.

Jsendzik — Thu, 07 Mar 2013 17:44:50 GMT

Hello world,

I'm fairly new to SAS and need some help. I'm trying to write a macro in EG that can open up a directory and create a data set that gives the size and file name of all files in that directory. I've seen some great examples in SAS papers but they require the pipe function and I don't have permission to use PIPE.

So far I've had success in doing two things.

1) Get all the filename's written to the log. This is done with a macro I found on SAS support.

%macro drive(dir,ext);

%let filrf=mydir;

/* Assigns the fileref of mydir to the directory and opens the directory */

%let rc=%sysfunc(filename(filrf,&dir));

%let did=%sysfunc(dopen(&filrf));

/* Returns the number of members in the directory */

%let memcnt=%sysfunc(dnum(&did));

/* Loops through entire directory */

%do i = 1 %to &memcnt;

/* Returns the extension from each file */

%let name=%qscan(%qsysfunc(dread(&did,&i)),-1,.);

/* Checks to see if file contains an extension */

%if %qupcase(%qsysfunc(dread(&did,&i))) ne %qupcase(&name) %then

%do;

/* Checks to see if the extension matches the parameter value */

/* If condition is true prints the full name to the log */

%if (%superq(ext) ne and %qupcase(&name) = %qupcase(&ext)) or

(%superq(ext) = and %superq(name) ne) %then

%do;

%put %qsysfunc(dread(&did,&i));

%end;

/* Closes the directory */

%let rc=%sysfunc(dclose(&did));

%mend drive;

2) Get the size of one file at a time. I used the previous macro's output to manually make macro variables. Then feed them into this code.

%let Invoice=\\mydir;

%let file1=<the file name returned by the last macro>

data info;

length infoname infoval $60;

drop rc fid infonum i close;

rc=filename("abc","&invoice.&file1.");

fid=dopen("abc");

infonum=fnum(fid);

do i=1 to infonum;

infoname=foptname(fid,i);

infoval=finfo(fid,infoname);

output;

end;

close=fclose(fid);

run;

A major problem with this is it only handles one file at a time, and I need all of this information to go into one location so I can look for files with the same size. I know I could make this into a loop that will output a bunch of datasets but this doesn't seem very efficient.

When I envision this is see the first macro storing all file names as individual macro variables. I know this can be done with a loop to update a %let statement <%let file&I. = maybe dread(&did,&I.)>. But the right side of this equality is a major road block for me.

After getting the file names stored, I need to grab the file size. I'm having a hard time doing this because i know i need to use finfo() and I have a dopen() situation. Is there a way to trickle down so to speak? Stating at dopen() and eventually using fopen() so that I can use finfo().

Major Questions:

How can I store system output functions as macro variable?

How can I go from a dopen() situation to a finfo() situation?

As a new programmer I'd really like guidance more than anything. I truly want to learn this not just have the code written for me. Any help would be much appreciated.

Re: Help comparing file size of files in a directory.

DBailey — Thu, 07 Mar 2013 18:24:28 GMT

what operating system do you use? You may be able to do this more efficiently using operating system commands.

Re: Help comparing file size of files in a directory.

DBailey — Thu, 07 Mar 2013 18:25:23 GMT

sorry..just saw the no pipes requirement. Can you execute batch files?

Re: Help comparing file size of files in a directory.

Jsendzik — Thu, 07 Mar 2013 18:32:03 GMT

I'm not sure If I can. I can tell you I'm running SAS EG on a windows XP system.

Re: Help comparing file size of files in a directory.

Jsendzik — Thu, 07 Mar 2013 18:37:09 GMT

I'm writing a batch file now to see if I can use them

Re: Help comparing file size of files in a directory.

Jsendzik — Thu, 07 Mar 2013 18:40:05 GMT

Okay just wrote one that would ping google. It worked fine. Yes I can execute batch files.

Re: Help comparing file size of files in a directory.

DBailey — Thu, 07 Mar 2013 19:09:22 GMT

So for all files on your c: drive, you could execute a batch program which gets a directory listing:

dir /s c:\ *.* > listing.txt

and then read listing.txt to parse out directory names, file names, and file sizes.

Re: Help comparing file size of files in a directory.

Jsendzik — Thu, 07 Mar 2013 19:18:44 GMT

Thanks ill try this now.

Re: Help comparing file size of files in a directory.

TomKari — Fri, 08 Mar 2013 16:23:27 GMT

Here's some code that I've used for a while. You may need to change the date and time informats, as the "dir" command uses your system settings.

Tom

data dirlist;

length test_rec $ 4096;

retain vol_let vol_name vol_ser dir_name first_dir;
length dir_name first_dir $ 1024;
length file_name $ 256;
length file_prefix $ 256;
length file_ext $ 256;
format file_date yymmdd10.;
format file_time time5.;

drop test_rec num_dots num_slash;

infile "C:\listing.txt";
input;

test_rec = _infile_;

if substr(test_rec, 1, 1) ^= " " then
do;
   if substr(test_rec, 25, 5) = "<DIR>" | substr(test_rec, 25, 10) = "<JUNCTION>" then; else
   do;
      file_date = input(substr(test_rec, 1, 10), ddmmyy10.);
      file_time = input(substr(test_rec, 13, 8), time8.);
      file_size = input(substr(test_rec, 22, 17), comma17.);
      file_name = substr(trim(test_rec), 40);
      num_dots = countc(file_name, ".", "o");
      if num_dots = 1 then
      do;
         file_prefix = scan(file_name, 1, ".");
         file_ext = scan(file_name, 2, ".");
      end; else
      if num_dots = 0 then
      do;
         file_prefix = file_name;
         file_ext = "";
      end; else
      do;
         file_prefix = reverse(scan(reverse(file_name), 2, "."));
         file_ext = reverse(scan(reverse(file_name), 1, "."));
      end;
      output;
   end;
end; else

if substr(test_rec, 1, 14) = " Directory of " then
do;
dir_name = substr(trim(test_rec), 15);
num_slash = countc(dir_name, "\", "o");
if num_slash = 1
then first_dir = dir_name;
else first_dir = scan(dir_name, 1, "\")||"\"||scan(dir_name, 2, "\");
end; else

if substr(test_rec, 1, 17) = " Volume in drive " then
do;
vol_let = substr(test_rec, 18, 1);
vol_name = substr(trim(test_rec), 23);
end; else

if substr(test_rec, 1, 25) = " Volume Serial Number is " then
do;
vol_ser = substr(trim(test_rec), 26);
end;

run;

Re: Help comparing file size of files in a directory.

Jsendzik — Fri, 08 Mar 2013 16:26:09 GMT

Thanks a bunch. I'll try this out now.

Re: Help comparing file size of files in a directory.

Jsendzik — Fri, 08 Mar 2013 16:38:37 GMT

I noticed that you have an in-file statement in the top portion. Is this a text document that lists all files within the directory? Or is that the file input i.e. what we eventually get output for.

Re: Help comparing file size of files in a directory.

TomKari — Fri, 08 Mar 2013 19:01:54 GMT

In 's post, the "dir /s c:\ *.* > listing.txt" will create a file named listing.txt that contains the results of the "dir" statement, in other words a listing of all of the files on the disk.

In my code, the "infile "C:\listing.txt";" statement will establish this as the file that I read the directory information from, in the "input;" statement. Using an input statement with just a semicolon will read a record, and store it in a variable named _infile_, which I then copy to test_rec and parse.

Re: Help comparing file size of files in a directory.

Jsendzik — Fri, 08 Mar 2013 19:13:59 GMT

I did not make that connection at all. That was a complete AHHAA moment. Thanks for your help.

Re: Help comparing file size of files in a directory.

Jsendzik — Fri, 08 Mar 2013 19:45:58 GMT

WOW WOW Wow. Thank you all so much it works and I got to start learning about .bat files. I'm now one step closer to becoming an even lazier programmer.