topic Re: Compare two Folders to see Changes. in SAS Programming

Compare two Folders to see Changes.

SASuserlot — Fri, 28 Oct 2022 18:20:55 GMT

I am looking to compare two folder which contains same number of files, lets say 'folder1 ' and 'folder 2'. I wan to run a program to show if there is any changes in files between folders. We get a data every month which we will copy into folder 1 and folder 2 .Folder 1 used by some other purpose . we use Folder 2 for analysis. So we expect both folder1 and folder 2 have latest and same data. I am thinking the process as mentioned like in the flow chart Image ( Just initial thoughts of my brain may be there are certain things I did not considered). How can we do this? I do have the Idea and have macro to compare datasets in two folder ( which is last part of the flow chart). Thank you for your thoughts and inputs.

Re: Compare two Folders to see Changes.

SASJedi — Fri, 28 Oct 2022 19:38:31 GMT

Here is a quick check that uses some utility macros available from my GitHub stash. The code will get the macros for you:

/* Get the macros */
filename code url "https://raw.githubusercontent.com/SASJedi/sas-macros/master/exist.sas";
%include code;

filename code url "https://raw.githubusercontent.com/SASJedi/sas-macros/master/fileattribs.sas";
%include code;

filename code url "https://raw.githubusercontent.com/SASJedi/sas-macros/master/findfiles.sas";
%include code;

filename code url "https://raw.githubusercontent.com/SASJedi/sas-macros/master/translate.sas";
%include code;

filename code clear;

/* Get list of files and attributes into 2 data sets */
%findfiles(s:/folder1,,work.folder1)
%findfiles(s:/folder2,,work.folder2)

/* Report the differences*/
proc sql;
select catx('/',coalesce(f1.path,f1.path),coalesce(f1.filename,f1.filename)) as File
		,case 
			when missing(f1.filename) then catx(' ','Missing from',f1.path)
			when missing(f2.filename) then catx(' ','Missing from',f2.path)
			else 'OK'
		end as Files
		,case 
			when f1.size <> f2.size then 'Differs'
			else 'OK'
		end as Size
		,case 
			when f1.CRDate <> f2.CRDate then 'Differs'
			else 'OK'
		end as CreationDate
		,case 
			when f1.ModDate <> f2.ModDate then 'Differs'
			else 'OK'
		end as ModDate
	from folder1 f1
		full join 
		  folder2 f2
	on f1.filename=f2.filename
	where f1.filename is null 
		or f2.filename is null 
		or f1.size    <> f2.size   
		or f1.CRDate  <> f2.CRDate 
		or f1.ModDate <> f2.ModDate
;
quit;

May the SAS be with you!
Mark

Re: Compare two Folders to see Changes.

SASuserlot — Fri, 28 Oct 2022 21:56:15 GMT

Thank you @SASJedi . It worked like magic. However I have a couple of questions.

1. Find files macro what are these macro variables refers to?

%macro findfiles(dir,ext,dsn,sub) / minoperator;

I am not sure about the dsn and sub and minoroperator do ?

2. Present code going to the sub folder lever. How I can control if I don't want subfolder lever?

In my case

Folder 1 Folder 2
Count.sas7bdat Count.sas7bdat
Expenses .sas7bdat Expenses .sas7bdat
loss.sas7bdat loss.sas7bdat
profit.sas7bdat profit.sas7bdat
SUB FOLDER

With present code it going to subfolder and finding the sas7bdat files and comparing them. However is it possible can I restrict not going to the subfolder? So with example above I am expecting both folders are same when no changes in the information . Show differ if any changes in the sas7bdat of main folder. Thank you.

Re: Compare two Folders to see Changes.

SASJedi — Sat, 29 Oct 2022 14:14:52 GMT

For any of the macros I use here, just submit %macroname(?) to get syntax help in the log. For example, submitting this:

%findfiles(?)

Gets this result in the log:

NOTE:  *FINDFILES Documentation *******************************

       Produces a list of files with a specified extension in the log
       or optionally writes them to a dataset.

       SYNTAX: %FindFiles(dir,<ext,dsn,sub>)
          dir=fully qualified directory path
          ext=Space delimited list of file extensions (Optional, default is ALL)
          dsn=name of data set to store filenames (Optional, otherwise writes to log.)
          sub=look in subfolders? (Y|N default is Y)

       Example:
       %FindFiles(c:\temp, csv)
       %FindFiles(\\server\folder\, xls xlsx xlsm, work.myfiles)
       %FindFiles(s:/workshop,sas,work.pgm_files,N)

       *************************************************************

As you can see above, the meaning and usage of each macro parameter is explained in the docs. To stop the macor from looking into subfolders, set the fourth parameter to N. So, for the example macro calls in the original response, you would use:

%findfiles(s:/folder1,,work.folder1,N)
%findfiles(s:/folder2,,work.folder2,N)

Hope this helps!

Mark

Re: Compare two Folders to see Changes.

AllanBowe — Sat, 29 Oct 2022 14:30:29 GMT

This macro will recursively scan a directory and produce an output table with a file listing and their corresponding md5 hashes.

https://core.sasjs.io/mp__hashdirectory_8sas.html

So if there is a change of any data in any of the files, you can can identify it by the change in the hash value.

This feature is used by the SASjs "fs sync" function as documented here: https://cli.sasjs.io/fs

This lets you synchronise a remote (SAS) directory with a local (desktop) one.