topic Re: How do I amend lines of data within multiple text files in SAS Programming

How do I amend lines of data within multiple text files

ciaranhillery — Mon, 25 Feb 2019 14:52:02 GMT

Hi,

I'm looking to amend the following lines of text on multiple .txt files, using a macro if at all possible. These .txt files are configuration files that are used for running daily jobs which need to be manually amended every (working) day on Unix e.g. today's directory will be /client_Feb2019/25022019/, with the filename being amended to client_25022019.

directory=/root_location/client_MonYYYY/DDMMYYYY/
filename=client_DDMMYYYY

I am able to amend the lines using the sed command, example below, but am looking for an easier way of amending the lines than manually changing the dates etc. on each sed command.

sed -e "s/directory=(old)/directory=(needed)/g" -e "s/filename=(old)/filename=(needed)/g" (location of root file)/(root filename).txt > (run location)/(new filename).txt

I'm looking to do this for 4 separate clients, and amending/creating the amended .txt files for every working day of the month. I've already found code to extract the working days per month, and assign to a macro variable.

Re: How do I amend lines of data within multiple text files

RW9 — Mon, 25 Feb 2019 15:04:12 GMT

You could do something like:

/* Set up dir list - change for your operating system */
filename tmp pipe 'dir "c:/root_location" /b';

/* Read list of files in and for each one execute a system command to rename */
data _null_;
  length buff want str $200;
  infile tmp;
  input buff $;
  want=catx("_","client",scan(buff,2,"/"));
  str=cat('x ',"'",'ren ','"c:/root_location/',strip(buff),'" "c:/root_location/',strip(want),'"');
  call execute(str);
run;

filename tmp clear;

Do note, that is based on Windows, and I have nothing to test it on. But something like that should work. Its just a matter of pulling in a list of file names, then using call execute to shoot out a rename from to.

Re: How do I amend lines of data within multiple text files

ciaranhillery — Mon, 25 Feb 2019 16:15:03 GMT

Thanks for responding. My text files don't have data fields as such, see the beginning of the file below:

############################################################
#
# Batch ini file
#
# lines beginning with a "#" are ignored as comments
#
############################################################
#
# Common Variables here
#
############################################################

params=COMMON

debug=no
directory=/ROOT_LOCATION/CLIENT_MONYYYY/DDMMYYYY/
filename=CLIENT_DDMMYYYY

What I'm trying to do is amend the directory and filename settings on the files... the root file will be named as above, so want to replace the relevant text with what I need.

Re: How do I amend lines of data within multiple text files

andreas_lds — Mon, 25 Feb 2019 16:48:40 GMT

Why do want to do this with sas? I would use a scripting language, maybe Powershell.

Re: How do I amend lines of data within multiple text files

ciaranhillery — Wed, 27 Feb 2019 13:08:49 GMT

I am looking to doing it via SAS (on Unix) as the folders and files I need are used on another Unix box, so I was hoping to be able to make all the amendments on SAS, then transfer the newly created folders and amended files across using scp function.

Re: How do I amend lines of data within multiple text files

RW9 — Wed, 27 Feb 2019 13:19:25 GMT

Sorry, you have totally lost me now. What is it you want to do, show examples - clear examples. The code I provided creates a series of rename operating system calls to rename a file from one filename to another. Are you saying now that you want to alter text within a file? If so then that is a totally different setup, you would read the file in and then write it out again:

data _null_;
  infile "....txt";
  file "....txt";
  length buff $2000;
  input buff $;
  if index(buff,"filename") then do;
    /* do you text change here */
  end;
  put buff;
run;

With this you read the file one line at a time then write that back out, the if takes effect on the line read in.

Re: How do I amend lines of data within multiple text files

ciaranhillery — Thu, 28 Feb 2019 16:02:13 GMT

Ok, so I'm looking to create folders for every working day of a month, and amend a couple of lines within a text file and copy to each of the newly created folders. I've figured out how create the folders, I'm looking to alter text within a file.

I will have a default file that I want to edit the text within, and output the amended file to a different folder location. The default file looks as below, with the text highlighted in green is what I'll need to change.

params=COMMON

debug=no
directory=/ROOT_LOCATION/CLIENT_MONYYYY/DDMMYYYY/
filename=CLIENT_DDMMYYYY

e.g. the file for today will have been manually changed from the above to the following format:

directory=/data_location/Test_Feb2018/28022019/
filename=Test_28022019

I will have a look at the background steps of the jobs that the above file is needed for to see if all of the lines above the params=COMMON line are required, as if not, it would cut down how many lines need reading in... on your code, I assume that any subsequent lines on the files would need to be read in too? There are quite a number of additional settings beneath the lines I want to amend which do not need editing... it's solely the above mentioned lines I want to change.

Re: How do I amend lines of data within multiple text files

Astounding — Thu, 28 Feb 2019 19:03:51 GMT

Would it help you if you could run a one-line command?

%macro ini (client=Test, ddmmyyyy=20022019, folder=/ROOT_LOCATION)

The idea is that this command would generate the required text file, with the proper changes.

Some short-cuts could be built in. So far, it looks like we can compute MONYYYY based on DDMMYYYY. If we knew that the default should always be to use the current month, that could possibly be shortened further.

You might need to submit many such lines, or it might be possible to automate the calls to the macro. But first let's address whether that would be a viable solution.

Re: How do I amend lines of data within multiple text files

ciaranhillery — Fri, 01 Mar 2019 10:19:47 GMT

I was hoping for a solution along these lines tbh! I have already got the relevant working day dates written to a macro created in a previous step on my code (the ddmmyyyy date that I'm looking to change on both lines within my file)... the previous step of my code extracts the working days per month and creates the date folder used in the directory line:

directory=/data_location/Test_Feb2018/28022019/

And yes, the default MONYYYY date in the directory line should always be to use the current month.

Ultimately my goal is to be able to run a program at the beginning of each month, which will create the daily working day folders, and amend the lines within the needed control file for each working day so that it doesn't have to be manually done every day of the month.

Re: How do I amend lines of data within multiple text files

RW9 — Fri, 01 Mar 2019 10:34:20 GMT

Perhaps thinking about this wrong, what I would do is this:

%macro Create_Folder (d=);

  /* From d derive monyyyy var */
  data _null_;
    temp=input("&d.",ddmmyy8.);
    temp2=substr(put(temp,date9.),3);
    call symputx('mmmyyyy',temp2);
  run;

  /* Create folder */
  x 'mkdir "c:/data_location/Test_&mmmyyyy./&d."';

  /* Write out text file to directory */
  data _null_;
    file "c:/data_location/Test_&mmmyyyy./&d./batch_ini_file.txt";
    put "#####################################";
    ...
    put "directory=/data_location/Test_&mmmyyyy./&d./";
    put "filename="Test_28022019";
  run;

%mend Create_Folder;

/* Call macro for each month of year (note you can't do by day as you don't have day in filename */
data _null_;
  do i=1 to 12;
    call execute('%Create_Folder (d=2018',put(i,z2.),'01);');
  end;
run;

This will create a folder with the text file inside for each month of the year.

I would highly advise against using MMMYYYY format for anything (its not granular enough, and its a bad representation of date as language changes can mean different spellings, it doesn't sort etc.) Use Iso dates, they are much better.

Also, I would question why you would need to do this, sound like a big process for something which doesn't sound like it adds any value to me? I would hope most people nowadays use version control software of some kind, so this dated files/folders thing should be long gone, let the software manage that for you.