Help comparing file size of files in a directory.

[Jsendzik]

Hello world,

I'm fairly new to SAS and need some help. I'm trying to write a macro in EG that can open up a directory and create a data set that gives the size and file name of all files in that directory. I've seen some great examples in SAS papers but they require the pipe function and I don't have permission to use PIPE.

So far I've had success in doing two things.

1) Get all the filename's written to the log. This is done with a macro I found on SAS support.

%macro drive(dir,ext);

  %let filrf=mydir;

  /* Assigns the fileref of mydir to the directory and opens the directory */

  %let rc=%sysfunc(filename(filrf,&dir));

  %let did=%sysfunc(dopen(&filrf));

  /* Returns the number of members in the directory */

  %let memcnt=%sysfunc(dnum(&did));

  /* Loops through entire directory */

  %do i = 1 %to &memcnt;

  /* Returns the extension from each file */

  %let name=%qscan(%qsysfunc(dread(&did,&i)),-1,.);

  /* Checks to see if file contains an extension */

  %if %qupcase(%qsysfunc(dread(&did,&i))) ne %qupcase(&name) %then

  %do;

  /* Checks to see if the extension matches the parameter value */

  /* If condition is true prints the full name to the log */

  %if (%superq(ext) ne and %qupcase(&name) = %qupcase(&ext)) or

  (%superq(ext) = and %superq(name) ne) %then

  %do;

  %put %qsysfunc(dread(&did,&i));

  %end;

  %end;

  %end;

  /* Closes the directory */

  %let rc=%sysfunc(dclose(&did));

%mend drive;

2) Get the size of one file at a time. I used the previous macro's output to manually make macro variables. Then feed them into this code.

%let Invoice=\\mydir;

%let file1=<the file name returned by the last macro>

data info;

   length infoname infoval $60;

   drop rc fid infonum i close;

   rc=filename("abc","&invoice.&file1.");

   fid=dopen("abc");

   infonum=fnum(fid);

   do i=1 to infonum;

      infoname=foptname(fid,i);

      infoval=finfo(fid,infoname);

      output;

   end;

   close=fclose(fid);

run;

A major problem with this is it only handles one file at a time, and I need all of this information to go into one location so I can look for files with the same size. I know I could make this into a loop that will output a bunch of datasets but this doesn't seem very efficient.

When I envision this is see the first macro storing all file names as individual macro variables. I know this can be done with a loop to update a %let statement <%let file&I. = maybe dread(&did,&I.)>. But the right side of this equality is a major road block for me.

After getting the file names stored, I need to grab the file size. I'm having a hard time doing this because i know i need to use finfo() and I have a dopen() situation. Is there a way to trickle down so to speak? Stating at dopen() and eventually using fopen() so that I can use finfo().

Major Questions:

How can I store system output functions as macro variable?

How can I go from a dopen() situation to a finfo() situation?

As a new programmer I'd really like guidance more than anything. I truly want to learn this not just have the code written for me. Any help would be much appreciated. 

[DBailey]

what operating system do you use?  You may be able to do this more efficiently using operating system commands.

[DBailey]

sorry..just saw the no pipes requirement.  Can you execute batch files?

[Jsendzik]

I'm not sure If I can. I can tell you I'm running SAS EG on a windows XP system.

[Jsendzik]

I'm writing a batch file now to see if I can use them

[Jsendzik]

Okay just wrote one that would ping google. It worked fine. Yes I can execute batch files.

[DBailey]

So for all files on your c: drive, you could execute a batch program which gets a directory listing:

dir /s c:\ *.* > listing.txt

and then read listing.txt to parse out directory names, file names, and file sizes.

[Jsendzik]

Thanks ill try this now.

[TomKari · Accepted solution]

Here's some code that I've used for a while. You may need to change the date and time informats, as the "dir" command uses your system settings.

Tom

data dirlist;

length test_rec $ 4096;

retain vol_let vol_name vol_ser dir_name first_dir;
length dir_name first_dir $ 1024;
length file_name $ 256;
length file_prefix $ 256;
length file_ext $ 256;
format file_date yymmdd10.;
format file_time time5.;

drop test_rec num_dots num_slash;

infile "C:\listing.txt";
input;

test_rec = _infile_;

if substr(test_rec, 1, 1) ^= " " then
do;
   if substr(test_rec, 25, 5) = "<DIR>" | substr(test_rec, 25, 10) = "<JUNCTION>" then; else
   do;
      file_date = input(substr(test_rec, 1, 10), ddmmyy10.);
      file_time = input(substr(test_rec, 13, 8), time8.);
      file_size = input(substr(test_rec, 22, 17), comma17.);
      file_name = substr(trim(test_rec), 40);
      num_dots = countc(file_name, ".", "o");
      if num_dots = 1 then
      do;
         file_prefix = scan(file_name, 1, ".");
         file_ext = scan(file_name, 2, ".");
      end; else
      if num_dots = 0 then
      do;
         file_prefix = file_name;
         file_ext = "";
      end; else
      do;
         file_prefix = reverse(scan(reverse(file_name), 2, "."));
         file_ext = reverse(scan(reverse(file_name), 1, "."));
      end;
      output;
   end;
end; else

if substr(test_rec, 1, 14) = " Directory of " then
do;
  dir_name = substr(trim(test_rec), 15);
  num_slash = countc(dir_name, "\", "o");
  if num_slash = 1
  then first_dir = dir_name;
  else first_dir = scan(dir_name, 1, "\")||"\"||scan(dir_name, 2, "\");
end; else

if substr(test_rec, 1, 17) = " Volume in drive " then
do;
  vol_let = substr(test_rec, 18, 1);
  vol_name = substr(trim(test_rec), 23);
end; else

if substr(test_rec, 1, 25) = " Volume Serial Number is " then
do;
  vol_ser = substr(trim(test_rec), 26);
end;

run;

[Jsendzik]

Thanks a bunch. I'll try this out now.

[Jsendzik]

I noticed that you have an in-file statement in the top portion. Is this a text document that lists all files within the directory? Or is that the file input i.e. what we eventually get output for.

[TomKari]

In 's post, the "dir /s c:\ *.* > listing.txt" will create a file named listing.txt that contains the results of the "dir" statement, in other words a listing of all of the files on the disk.

In my code, the "infile "C:\listing.txt";" statement will establish this as the file that I read the directory information from, in the "input;" statement. Using an input statement with just a semicolon will read a record, and store it in a variable named _infile_, which I then copy to test_rec and parse.

[Jsendzik]

I did not make that connection at all. That was a complete AHHAA moment. Thanks for your help.

[Jsendzik]

WOW WOW Wow. Thank you all so much it works and I got to start learning about .bat files. I'm now one step closer to becoming an even lazier programmer.