Apostrophe in a comment in a macro - why after 40+ years does SAS still get "confused"

[ChrisNZ]

> My approach for commenting a section of code is:

That's a brand new approach, that just just became possible in the most recent version of SAS.

And for 40 years SAS has been, and still is, tripped by silly gotchas for no good reason.

And still no way to use end-of-line comments, or easily debug macro code.

.

[ScottBass]

---

@ghosh wrote:

My approach for commenting a section of code is:

%if 0 %then %do;
* It's a sad world where an apostrophe in a comment in an uncalled macro causes a problem ;
data bar;set sashelp.class;run;
data foo;    /* the output dataset is named "foo" */
   set bar;  /* the input dataset is named "bar" */
             /* and if I enclose this code in an uncalled macro I get "fubar" */
run;
%end;

Then, if you need to execute the commented coded, just change the first line to :

%if 1 %then %do;

---

Doesn't work, plus that's conditional execution, not a commenting out of code:

%macro test;
%if 0 %then %do;
* It's a sad world where an apostrophe in a comment in an uncalled macro causes a problem ;
data bar;set sashelp.class;run;
data foo;    /* the output dataset is named "foo" */
   set bar;  /* the input dataset is named "bar" */
             /* and if I enclose this code in an uncalled macro I get "fubar" */
run;
%end;
%mend;
%test

Edit:

I just read @ChrisNZ 's reply.  I'm currently on SAS 9.3, so can't test this.  So perhaps this new SAS 9.4 feature of a tiny subset of macro functionality in open code works around this issue by removing the macro tokenizer. 

[jimbarbour]

@ScottBass,

I hear you, and I like you I have basically just learned "don't do that."  I avoid using any kind of quote mark in a comment.  

However, there is a way out for when you get stuck.  See below.

---

@ScottBass wrote:

...I tried submitting the magic tokens from EG ;*';*";*/;quit;run; but this didn't fix the problem.  The only thing I could think to fix this was restart the server.

 So the "fix" to get unstuck is in two parts:

1.  Highlight the quote (only!) in the comment that got you in trouble in the first place.  Run your program with only the quote highlighted.

2.  Now highlight %Mend.  Run your program with only %Mend highlighted.

You should now be out of the netherworld of a botched macro compile.  Don't run again until you've either commented out or removed the quote that got you in trouble in the first place.

If you do the above two steps and then comment out the offending quote, you can now get a normal run without having to shut everything down (and lose all your work datasets!! 😱 )

I'll put the logs from each step below.

The problem:

29         %macro comment;
30         
31         * It's a sad world where an apostrophe in a comment in an uncalled macro causes a problem ;
32         data bar;set sashelp.class;run;
33         data foo;    /* the output dataset is named "foo" */
34            set bar;  /* the input dataset is named "bar" */
35                      /* and if I enclose this code in an uncalled macro I get "fubar" */
NOTE: The quoted string currently being processed has become more than 262 characters long.  You might have unbalanced 
      quotation marks.
36         run;
37         
38         %mend;
39         
40         data blah;
41            set sashelp.class;
42         run;

The log from just highlighting the just quote in the comment and running.  The log just has some irrelevant gobbledy-**bleep** from EG:

7          %LET _CLIENTTASKLABEL='Program';
8          %LET _CLIENTPROCESSFLOWNAME='Process Flow';
9          %LET _CLIENTPROJECTPATH='';
10         %LET _CLIENTPROJECTPATHHOST='';
11         %LET _CLIENTPROJECTNAME='';
12         %LET _SASPROGRAMFILE='';
13         %LET _SASPROGRAMFILEHOST='';
14         
15         ODS _ALL_ CLOSE;
16         OPTIONS DEV=SVG;
17         FILENAME EGHTML TEMP;
18         ODS HTML5(ID=EGHTML) FILE=EGHTML
19             OPTIONS(BITMAP_MODE='INLINE')
20             ENCODING='utf-8'
21             STYLE=HTMLBlue
22             NOGTITLE
23             NOGFOOTNOTE
24             GPATH=&sasworklocation
25         ;

But then highlight just the %mend; and run:

15         ODS _ALL_ CLOSE;
16         OPTIONS DEV=SVG;
17         FILENAME EGHTML TEMP;
18         ODS HTML5(ID=EGHTML) FILE=EGHTML
19             OPTIONS(BITMAP_MODE='INLINE')
20             ENCODING='utf-8'
21             STYLE=HTMLBlue
22             NOGTITLE
23             NOGFOOTNOTE
24             GPATH=&sasworklocation
25         ;
NOTE: The macro COMMENT completed compilation without errors.
      76 instructions 3284 bytes.

The first step of the fix closes the quote; the second completes the macro.  Now you have a "clear" EG session.  Comment out the offending quote, and run the entire program:

29         %macro comment;
30         
31         /** It's a sad world where an apostrophe in a comment in an uncalled macro causes a problem ;*/
32         data bar;set sashelp.class;run;
33         data foo;    /* the output dataset is named "foo" */
34            set bar;  /* the input dataset is named "bar" */
35                      /* and if I enclose this code in an uncalled macro I get "fubar" */
36         run;
37         
38         %mend;
NOTE: The macro COMMENT completed compilation without errors.
      7 instructions 164 bytes.
39         
40         data blah;
41            set sashelp.class;
42         run;

NOTE: There were 19 observations read from the data set SASHELP.CLASS.
NOTE: The data set WORK.BLAH has 19 observations and 5 variables.
NOTE: Compressing data set WORK.BLAH increased size by 100.00 percent. 
      Compressed is 2 pages; un-compressed would require 1 pages.
NOTE: DATA statement used (Total process time):
      real time           0:00:00.00
      cpu time            0:00:00.00
      Timestamp           August 31, 2020 00:45:03

Voila!  Working code!  (and you didn't have to give up the Work datasets that may have taken you hours  of run time to create)

I wish I could fix the SAS compiler for you, but at least you now have a way to get out of the netherworld created by a quote inside a comment in a macro.

Jim

P.S.  While we're on the subject of comment rants, I wish that /* */ style comments could be nested, i.e. that if one applied /* */ comments to a block of code that already contained some /* */ style comments SAS would accept the entire block as a comment.  As it is, SAS will stop the comment at the first */ it encounters even if two /*'s preceded the */.  My work around is to always use * ; style comments wherever I can so that I can comment out whole blocks of code easily.   Yes, the uncalled MACRO-MEND style of comment works for blocks of code for the most part, but as with a quote mark in a comment, even uncalled macros can generate errors.

[ScottBass]

---

@jimbarbour wrote:

@ScottBass,

I hear you, and I like you have basically just learned "don't do that."  I avoid using any kind of quote mark in a comment.  

While we're on the subject of comment rants, I wish that /* */ style comments could be nested, i.e. that if one applied /* */ comments to a block of code that already contained some /* */ style comments SAS would accept the entire block as a comment.  As it is, SAS will stop the comment at the first */ it encounters even if two /*'s preceded the */.  My work around is to always use * ; style comments wherever I can so that I can comment out whole blocks of code easily.   Yes, the uncalled MACRO-MEND style of comment works for blocks of code for the most part, but as with a quote mark in a comment, even uncalled macros can generate errors.

Jim

---

@jimbarbour 

I'd give more likes if I could (or if my reputation score was higher you'd get 2 😄 ).

I agree with this 100%.

[jimbarbour]

@ScottBass,

I just revised my above comment to include a workaround/fix for how to get unstuck (without shutting down EG and restarting) if you get bit by the quote in a comment in a macro problem.  Check it out.  🙂

Jim

[andreas_lds]

+1 for

---

@jimbarbour wrote:

P.S.  While we're on the subject of comment rants, I wish that /* */ style comments could be nested, i.e. that if one applied /* */ comments to a block of code that already contained some /* */ style comments SAS would accept the entire block as a comment.  As it is, SAS will stop the comment at the first */ it encounters even if two /*'s preceded the */.  My work around is to always use * ; style comments wherever I can so that I can comment out whole blocks of code easily.   Yes, the uncalled MACRO-MEND style of comment works for blocks of code for the most part, but as with a quote mark in a comment, even uncalled macros can generate errors.

---

[Tom]

The problems are in the parser(s).  When the regular SAS parser sees the * it ignores the unbalanced quotes. When the macro parser sees the * it treats it the same as any other non-macro word.  So when it sees the unbalanced quotes it treats everything as text to be included in the quoted string.

You are not going to get any changes made to this behavior as I doubt that SAS could find a solution that would not also break many working programs.  Plus changes at such a deeply internal part of the process is harder and riskier.

Personally I just don't put unbalanced quotes in comments (except perhaps in /* */ and then only when trying to illustrate why to avoid them).

[ScottBass]

When the regular SAS parser sees the * it ignores the unbalanced quotes.

When the macro parser sees the * it treats it the same as any other non-macro word.

@Tom 
This is rather an academic question...but I'm having problems seeing why this would be a difficult issue to fix?

You've already got C code in the SAS parser that recognizes a "*" as the beginning of a comment, and treats the entire statement as a comment, unbalanced quote and all.  Perhaps it has to add context to this, such as "*" could be a multiplication sign, is it itself contained in a quoted string, etc.

You've got the macro parser which is "pre-processing" the SAS code, analogous to the C pre-processor, and substituting text when appropriate (plus other stuff, such as looping, data step interface, etc).

Why can't they take the SAS parser code and use that in the macro processor code to indicate that the "*" is a comment, to be treated in the same way that the SAS parser does?

IOW, why can't they make the comment statement within a macro act the same as a comment statement outside a macro?

You are not going to get any changes made to this behavior as I doubt that SAS could find a solution that would not also break many working programs.

Ok, so let's pretend we are just starting to develop the macro code.  It has not yet been released in production.  Would this problem be hard to fix then?  Is it possible that this "feature" wasn't picked up in initial QA testing and, if it had, it would have been fixed?  And what if it slipped into production, but caught say 6 months later, what then?

Is the problem not that it is hard to fix, but rather too dangerous to fix now that there are 30+ years of macros in production?

Personally I just don't put unbalanced quotes in comments (except perhaps in /* */ and then only when trying to illustrate why to avoid them).

Yeah I guess I should develop the habit to not use contractions, possessive, etc. in comments.  I guess that's the "solution".

[jimbarbour]

@ScottBass, 

I wonder if EG can be set up such that common contractions (don't, won't, can't, etc.) are automatically converted to their non-contraction form (do not, will not, cannot, etc.).  This might be something along the lines of what @ChrisHemedinger wrote about in the third item of this blog post:  https://blogs.sas.com/content/sasdummy/2017/07/03/sas-program-editor-tricks/.  I mean it stands to reason that the power of the SAS EG editor ought to be able to help us out here, right?

Jim