Macro and data step combination to read multiple text files as a string variable in a SAS dataset

[nirupmenon]

Folks,

I have multiple files in a directory containing descriptive text. I want to read each file completely into a SAS dataset as variables with the filename in one column and the entire text from each file as the second variable. There should be as many rows in the output dataset as files in the directory.

I followed Chris H.'s blog https://blogs.sas.com/content/sasdummy/2016/07/12/how-to-read-the-contents-of-a-file-into-a-sas-macr... and set the code as follows:

%macro fileread; /* run this macro for each file in the directory*/
data null;
    length filecontent $1000 textcontent $32627 ;
    retain filecontent '';
  infile "C:\ContestTextfile\&filein..txt" dlmstr='//' flowover end=last;
    %if _n_ = 1 %then input filecontent;
    %else %do;
       input;
       %let filecontent=cats(filecontent,_infile_);
    %end;
  if last then call symput('textcontent', filecontent);
 run;
%mend fileread;

/* the data step below uses the file list in the directory to assign each file name to a macro variable */
data htmdata.accel (keep=fname filecontent); set htmdata.file_list;
   length filecontent $32627 ;
   call symput('filein', trim(fname)); /* put the file name from the file list in a macro variable */
   %fileread; /* run the macro for each file in the file list */
   %let filecontent = textcontent;
run;

What I want is two columns in the dataset: File_Name File_Content.

However, I get multiple errors with the above code: 1) both filecontent and textcontent are uninitialized; 2)  the output file htmdata.accel contains only fname which was read from the file_list dataset.

Any help would appreciated. Thanks!

Nirup

[PaigeMiller]

If you get errors, show us the LOG. Run the program again with

options mprint;

as the first command.

We need to see the complete log of one of these steps that has an error, that's 100% of the log for one of those steps, with nothing chopped out. Format the log properly by copying it as text and pasting it into the window that appears when you click on the </> icon here. Do not provide the log any other way.

[nirupmenon]

Hey Paige,

I appreciate your reply. Recent Google search yielded https://stackoverflow.com/questions/31848367/filevar-infile-statement-to-read-in-multiple-txt-file.

So I changed the program to read two infiles (first the list of files and second the content of each file) instead of the macro in my earlier program:

options mprint;
libname htmdata "C:\Users\Owner\OneDrive - George Mason University - O365 Production\TopCoder";
filename DataIn pipe "dir C:\ContestTextfile\*.txt /S /B"; 
data htmdata.accel ;
  infile DataIn truncover;
    length fil2read $256; 
    input fil2read;
  infile xyz filevar = fil2read dlmstr='//' flowover end=last; 
    length filecontent $32627 ; 
    retain filecontent '';
    do while (not last);
     if _n_ = 1 then input filecontent;
     else do; 
        input; 
        filecontent=cats(filecontent,_infile_);
     end;
    end;
   if last then output;
keep fil2read filecontent;
run;
run;

Good news is that the program captures the filecontent variable from all files. However, the fil2read is not getting captured in the htmdata.accel as a column. I have attached the log file as a Word document.

Thanks,

Nirup

[PaigeMiller]

Many people including me will not open Microsoft Office documents as they can be security threats. Many people including me will not download attachments from this or other forums.

Please include follow the instructions I gave to provide the log file. No other format is acceptable.

[Tom · Accepted solution]

FIL2READ is not part of the output dataset because you used as the variable named in the FILEVAR= option on the INFILE statement. If you want to save the value assign it to another variable that will be kept.

[Kurt_Bremser]

The variable named as FILEVAR is not included in the output; you need to use another variable to which you assign the value from the FILEVAR= variable.

[nirupmenon]

Thanks, Kurt. Apologies, Paige that I could not figure out the SAS program / log combination to upload.