Macro language tokenization bug: "%%foo" will not trigger macro processor

[Quentin]

In answering another question, @Ksharp  pointed out that code like:

"%%foo"

will not trigger the macro processor to execute the macro FOO.  https://communities.sas.com/t5/SAS-Programming/warning-in-macro/m-p/895236/highlight/true#M353679. 

This was a surprise to me, and I think it is likely a word scanner / tokenization bug.  I'm not aware of any documented feature of the macro language that would explain this behavior.

In testing more, found some interesting results.  

The issue only seems to happen when there are an even number of consecutive macro triggers, i.e. %% or &% that appear inside of double quotes or parentheses.

I would expect all examples below to trigger the macro processor to execute the macro FOO.

Basic example:

%put %%Foo ;    *Does trigger macro processor ;
%put "%%Foo" ;   *Does NOT trigger macro processor ;

It's not a %PUT statement issue.  And it's not that %% is somehow resolving to %:

1    data want;
2      x="%%Foo" ;  *Does NOT trigger macro processor ;
3      put x= ;
4    run ;

x=%%Foo
NOTE: The data set WORK.WANT has 1 observations and 1 variables.

An odd number of % signs triggers macro processor, even does not:

%put "%Foo" ;    *Does trigger macro processor ;
%put "%%Foo" ;   *Does NOT trigger macro processor ;
%put "%%%Foo" ;  *Does trigger macro processor ;
%put "%%%%Foo" ; *Does NOT trigger macro processor ;

Parentheses can also cause the issue:

%put %length(%foo) ;   *Does trigger macro processor ;
%put %length(%%foo) ;  *Does NOT trigger macro processor ;

An ampersand followed by one or more percent signs seems to have the same effect:

%put &%Foo ;     *Does trigger macro processor ;
%put "&%Foo" ;   *Does NOT trigger macro processor ;

%put "%Foo" ;    *Does trigger macro processor ;
%put "&%Foo" ;   *Does NOT trigger macro processor ;
%put "&%%Foo" ;  *Does trigger macro processor ;
%put "&%%%Foo" ; *Does NOT trigger macro processor ;

data want;
  x="&%Foo" ;  *Does NOT trigger macro processor ;
  put x= ;
run ;

%put %length(&%foo) ;  *Does NOT trigger macro processor ;

I submitted the issue to tech support (CS0017798).  They confirmed the behavior, and have escalated to R&D for an explanation.

[Quentin · Accepted solution]

I heard back from Tech Support.  They didn't provide any interesting insights or explanations, but they did confirm that this is a bug, and said they will be documenting it with a new problem note.  I asked them to share the URL after the note is published so that I can link to it from here.  But for now, I'll mark this as the accepted answer to close this out.