topic Re: Line break inserted in macro string every ~268 characters in SAS Programming

Line break inserted in macro string every ~268 characters

cau83 — Fri, 27 Oct 2017 12:30:54 GMT

I'm summarizing diagnostics and putting them in an email by selecting distinct values from diagnostic data sets and putting into macro variables that are separated by a carriage return ( byte(10) ). This all works fine except for when the macro variable contents exceeds ~268 characters-- I get extra line breaks. See example below, where the 3rd line and 5th line are broken when they should not be.

Why, and how to prevent it/work around it?

1) WARNING: Data set WORK.OUTLIER_CAL_HEATMAP2 was not replaced because this step was stopped.

2) WARNING: Multiple lengths were specified for the variable ATTRIBUTE_VALUE by input data set(s). This can cause truncation of data.

3) WARNING: Multiple lengths were sp [[breaks here]]

ecified for the variable breaktype by input data set(s). This can cause truncation of data.

4) WARNING: The data set WORK.OUTLIER_CAL_HEATMAP2 may be incomplete. When this step was stopped there were 0 observations and 5

5) WARNING: The variable iteration in th [[breaks here]]

e DROP, KEEP, or RENAME list has never been referenced.

%macro export_weekly_diags(ds,vars);
	proc export data=&ds outfile="XXX" dbms=xlsx replace;
		sheet="&ds";
	run;
	%global diags_&ds;
	proc sql noprint;
		select distinct &vars into :diags_&ds separated by "%sysfunc(byte(10))" from &ds; *insert ascii character for carriage return between each record;
	quit;
%mend;
%export_weekly_diags(ds=allbatch_ucm_log,vars=msg)
%export_weekly_diags(ds=post_log,vars=log)
[[etc.]]

filename mail email  to=("XXX") subject="XXX";
 ;
DATA _NULL_; 
  FILE mail; 
  PUT "UCM Log:"; put "&diags_allbatch_ucm_log";
  PUT; 
  PUT "Post Log:"; put "&diags_post_log";
  PUT ; 
  [[etc.]]
RUN;

Re: Line break inserted in macro string every ~268 characters

Kurt_Bremser — Thu, 26 Oct 2017 21:43:21 GMT

That's not a dataset, these are messages from the SAS log.

Re: Line break inserted in macro string every ~268 characters

Kurt_Bremser — Thu, 26 Oct 2017 21:44:22 GMT

PS show your code.

Re: Line break inserted in macro string every ~268 characters

Astounding — Thu, 26 Oct 2017 22:26:47 GMT

Since you have SAS data sets as your starting point, you should be able to remove the break character(s) there (before copying any data values into macro variables). You'll need to identify the character(s) that are causing the extra breaks, and then apply the COMPRESS function to the SAS variables that you intend to copy into macro variables.

Re: Line break inserted in macro string every ~268 characters

Tom — Fri, 27 Oct 2017 04:53:45 GMT

I'm summarizing diagnostics and putting them in an email by selecting distinct values from diagnostic data sets and putting into macro variables ...

I would recommend not taking data out of datasets and putting it into macro variables. Macro variables and macros are designed for generating code, not processing data.

You should be able to generate the email directly from the data without using macro variables.

Re: Line break inserted in macro string every ~268 characters

cau83 — Fri, 27 Oct 2017 12:28:01 GMT

It is a data set-- I've written the log out and read it back in with a data step.

Re: Line break inserted in macro string every ~268 characters

cau83 — Fri, 27 Oct 2017 12:33:24 GMT

I added the code.

Tom-- can you point me to information on how to generate the email directly from the data? I want to be able to insert information from multiple data sets into an email body (I could concatenate and combine into one data set if necessary).

Are you able to use the SET statement to place all of the contents of a data set into the email (rather than the PUT statements)?

Re: Line break inserted in macro string every ~268 characters

Kurt_Bremser — Fri, 27 Oct 2017 12:39:37 GMT

Use a set with the option

end=done

At _n_ = 1, put your email header.

Put your strings from the dataset

At done, put your email footer

Re: Line break inserted in macro string every ~268 characters

Tom — Fri, 27 Oct 2017 13:49:31 GMT

There is a lot of help with writing email messages from data in the documentation.

http://documentation.sas.com/?docsetId=hostunx&docsetTarget=p1hl3t66coao7bn18vrmhx2gte1q.htm&docsetVersion=9.4&locale=en

If you are reading messages from the SAS log then it is also likely that the messages in the SAS log were already split into multiple lines because the SAS log has a maximum record length, which you can control using the LINESIZE option.

Are you asking how to process the log and figure out when a message has spanned multiple lines?

Re: Line break inserted in macro string every ~268 characters

Kurt_Bremser — Fri, 27 Oct 2017 13:55:57 GMT

My preferred way to do this would be to filter the relevant lines from the log with grep into a text file, and attach that text file to the email.

Or do the grep in a filename pipe and use that as an infile in the data _null_ that creates the email.

Re: Line break inserted in macro string every ~268 characters

cau83 — Fri, 27 Oct 2017 14:00:22 GMT

"it is also likely that the messages in the SAS log were already split into multiple lines"

This is not the case. You can see that longer lines (or just as long) are not split in my example. Furthermore, the fact that it happens about every 268 characters is a sign that the behavior is within the macro variables. It's not coincidence, I saw it happen across 6 different macro strings.

Re: Line break inserted in macro string every ~268 characters

cau83 — Fri, 27 Oct 2017 14:02:58 GMT

I am using IF's to filter the lines and create a data set that exports to excel. If I was going to attach a file I'd just attach the excel.
Regardless-- I'm not so much looking for workarounds unless it accomplishes this exact thing. I can easily read this with the line breaks, it's just an annoyance but one I'll put up with versus creating and attaching text files.
I'd like to understand better the macro behavior.

Re: Line break inserted in macro string every ~268 characters

Tom — Fri, 27 Oct 2017 14:23:08 GMT

I really cannot tell what your program is doing. But if you have data like this:

data have ;
  infile cards truncover ;
  input line $char200. ;
cards4;
1) WARNING: Data set WORK.OUTLIER_CAL_HEATMAP2 was not replaced because this step was stopped.
2) WARNING: Multiple lengths were specified for the variable ATTRIBUTE_VALUE by input data set(s). This can cause truncation of data.
3) WARNING: Multiple lengths were specified for the variable breaktype by input data set(s). This can cause truncation of data.
4) WARNING: The data set WORK.OUTLIER_CAL_HEATMAP2 may be incomplete.  When this step was stopped there were 0 observations and 5
5) WARNING: The variable iteration in the DROP, KEEP, or RENAME list has never been referenced.
;;;;

And you want to send it in an email then write it using a data step. Do NOT first put it into a single macro variable.

data _null_;
  set have ;
  file email ;
  len = length(line);
  put line $varying200. len ;
run;

If you did for some strange reason what to put those 5 lines of text into a single macro variable then mark the line breaks with something and use the data step to split it into lines for the PUT statement to output.

data _null_;
  file email ;
  longline="&my_macro_variable";
  dlm='0A'x ;
  do i=1 to countw(longline,dlm);
    length line $200;
    line = scan(longline,i,dlm);
    len = length(line);
    put line $varying200. len ;
  end;
run;

Note that this will only work for macro variables whose length is short enough to fit into a data set variables. Macro variables can be up to 64K bytes and regular variables only 32K.

Re: Line break inserted in macro string every ~268 characters

cau83 — Fri, 27 Oct 2017 15:05:36 GMT

data _null_;
  set have ;
  file email ;
  len = length(line);
  put line $varying200. len ;
run;

This puts the datasets in the body of the email w/o line breaks. However I can't add single headers and blank rows as easily. Using this code I get these results:

DATA _NULL_; 
  FILE mail; 
  set diags_post_log(in=a) diags_zero_fc2(in=b);
  if a then do;
  	if _n_=1 then put "Post Log:";
	put log;
	end;
  else if b then do;
	if _n_=1 then put "Zero FC:";
	put new_skill_name attribute_value;
	end;
RUN;

Results:

Post Log:

WARNING: Data set WORK.OUTLIER_CAL_HEATMAP2 was not replaced because this step was stopped.

WARNING: Multiple lengths were specified for the variable ATTRIBUTE_VALUE by input data set(s). This can cause truncation of data.

WARNING: Multiple lengths were specified for the variable breaktype by input data set(s). This can cause truncation of data.

WARNING: The data set WORK.OUTLIER_CAL_HEATMAP2 may be incomplete. When this step was stopped there were 0 observations and 5

WARNING: The variable iteration in the DROP, KEEP, or RENAME list has never been referenced.

bm-collections-checks NONE

kwys-document-image-error NONE

nb-annuity-nondesktop-dup-policy SUPERSTAR nbd-transfer-rate-lock-flwp STAR pos-72t-calculation NONE pos-annuity-ntos-over-100k NONE

I would like a blank row and then a header row before "bm-collections-checks". _n_=1 is not relative to the input data set. I could possibly count the records and substitute if _N_=&recordcount1 (or something like that, i might need to add to it for the blank). That's getting more complicated however as I go down to groups 3 - 6.

Re: Line break inserted in macro string every ~268 characters

Tom — Fri, 27 Oct 2017 15:39:21 GMT

Not sure what is causing all of your other warnings but if you want to concatenate two dataset and count the number of observations contributed by each then make your own counter variable, do not try to use the data step loop counter _N_.

data concat;
  row+1;
  set sashelp.class(obs=3 in=in1) sashelp.class(obs=2 in=in2);
  if in2 and not lag(in2) then row=1;
run;

proc print ;
  var row name;
run;

Obs    row     Name

 1      1     Alfred
 2      2     Alice
 3      3     Barbara
 4      1     Alfred
 5      2     Alice

Re: Line break inserted in macro string every ~268 characters

cau83 — Fri, 27 Oct 2017 16:43:51 GMT

Thanks, I'll do the appending and counting a bit differently from that based on complexity but I'll probably switch my approach from macro to the SET statement approach you posted previously.