Stepping into the Matrix in SAS

[#1]

After seeing an wonderful presentation that SAS' very own "The Graph Guy" (Robert Allison) did at a company SAS event, my interest in creating visualisations within SAS was sparked. So I thought, what's best than creating something fun...

If you are interested in seeing some of Rob's wonderful work on generating visualisations from SAS, then visit: www.robslink.com.

I've always wanted to create something with animations in SAS, so thought that in the simplest way of implementing animations with text. So, I thought, can I create the iconic text/code from the film series "Matrix".

Keanu Reeves in the Matrix

PROC GANNO

After some thought and looking through examples in Rob's website, the PROC GANNO statement was quickly identified as the one that will allow me to do that I wanted to do. This procedure would allow me to create a graphics output with annotations (text) defined by an input dataset. This dataset would contain the x and y coordinates of the annotation/text.

To create the animation, I needed to then multiple version of the graphics output as a .GIF.

Creating some parameters

At the start of the code, I needed to consider some input parameters which would help configure the output. For these, I've added comments in the code. These will be more evident as the rest is explained.

 data _null_;

      * Set the frames per second;
      fps=7;

      * Define the animation duration using FPS;
      animduration=1/fps;

      * Spacing between characters;
      charspace=4; * The number of pixels between characters;
      colspace=2; * The number of pixels between columns of characters;

      * Define the text size in points;
      textsize=12;

      * Define the start y (vertical) position of text;
      y_start=99;

      * Put them to the macro variable list;
      call symputx('fps',fps);
      call symputx('animduration',put(animduration,6.4));
      call symputx('charspace',charspace);
      call symputx('colspace',colspace);
      call symputx('textsize',textsize);
      call symputx('y_start',y_start);

    run;

Defining the Character Set

Whilst there were some 'matrix fonts' to download, I didn't want to risk downloading them to my employer's machine and installing them. So I took the decision to use the standard character set.

This next step create the format called ALPHABET and using PROC FORMAT CTLIN. This will help map a numeric (1-36) to the alphanumeric string of [A-Z0-9]. This avoids repeating the same code throughout the solution, another method might be to develop a user-defined function using PROC FCMP but that seemed a bit overkill. 

    * Create a dataset that will feed into proc format;
    data work.alphabet_format (drop=alphabet);

      * Use the full alphanumeric list;
      alphabet = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890';

      fmtname = 'alphabet';
      do label=1 to length(alphabet);
        start=substr(alphabet, label, 1);
        /* Type = 'i' for the Text-to-Number informat, and */
        /* Type = 'n' for the Number-to-Text format. */
        type = 'n'; output;
      end;
    run;

    /* Now feed the dataset into PROC FORMAT to build the format/informat */
    proc format cntlin=Alphabet_Format (where = (type = 'n') rename = (start = label label = start)); 
    run;

Defining the colours (colors)

The next thing was to ensure that the greens of the text looked similar to those used in the film. After a bit of research (aka googling!), I managed to find the RBG values for the different shades of greens (I'm sure I've read a book with a slightly different name) and the shade of black.

SAS provides the out-of-the-box colour macro utility called %RGB to define the RGB value from the input values of red, green and blue. However, before you can use this macro you need to call the macro %COLOURMAC which initialises the colour macro utilities.

    * Need to call the colormac macro statement;
    * This allows us to use the RGB macro;
    * https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/graphref/n0f54aj6pc5nkbn1jvl6bcoc240y.htm;
    %colormac; 

    * Colors taken from Matrix Colors after Googling!!;
    %let black=%RGB(13,2,8);
    %let darkgreen=%RGB(0,59,0);
    %let middlegreen=%RGB(0,143,17);
    %let lightgreen=%RGB(0,255,65);

    * Print to log for debug;
    %put &=black. &=darkgreen &=middlegreen. &=lightgreen.;

Once I've defined the RBG value for the required colours, I map the colours to the sequence of frames that character has been visible. This is done within the PROC FORMAT to create the format MTRXCL.

    * Use a format to define the frame number and the colour;
    * In this example, the first 5 frames will be white then...;
    proc format;
      value mtrxcl 
         1- 5='white'
         6-10="&lightgreen."
        11-15="&middlegreen."
        16-20="&darkgreen."
        other="&black."
      ;
    run;

Starting to build the visualisation

Now this is the exciting bit, where we build the visualisation...

Before we get into the code, we need to understand that the annotation to SAS graphics has a number of different systems. These can be based on percentages, values etc. More information on this can be found here: SAS Help Center: XSYS Variable

The Inputs to PROC GANNO

Next, we need to understand the inputs to the PROC GANNO procedure. These are:

Column Name	Length	Description
xsys	$1.	This defines the system used for the x-axis variable.
ysys	$1.	This defines the system used for the y-axis variable.
function	$32.	This defines the "action" to do/draw in the annotation
style	$32.	This defines the type of item from the function.
color	$32.	What colour is to be used
text	$15.	Contains the text to annotate, if specified in the function.
x	8.	The x-axis position
y	8.	The y-axis position

The background

For each frame in the output, we will need it to have a consistent background. This will be defined where frame=0.

Setting up the structure of the table:

    * Create a dataset containing the background box;
    data work.annotate_background;

      * Define variables;
      attrib
              FrameNum      length=8.
              xsys          length=$1.
              ysys          length=$1.
              function      length=$32.
              style         length=$32.
              color         length=$32.
              text          length=$15.
              ;
        
      * Define the coordinate system;
      * https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/graphref/annotate_xsys.htm;
      xsys='1'; ysys='1';

      * Create my data to help with what Frame we are on;
      FrameNum=0; * Frame zero will always be printed as this is the background;

Then define the colour of the background which will be drawn as a solid-bar from the coordinates (0,0) to (100,100). Remember that these coordinates are percentages of the image.

    * Create the instructions to draw the background box;

        * Define the colour;
        color="&black.";

        * Make sure that we are in the correct starting position;
        * Move to the position (0,0) in the coordinate system;
        function="move"; x=0; y=0; output;

        * Draw a bar (aka box) from the current position to (0,0), (100,100) with a solid colour;
        function="bar";  x=100; y=100; style="solid"; output;

    run;

Taking a pause and putting this dataset on it's own through the PROC GANNO statement we have:

The drawn background in black

 I know this doesn't look much, but it's a start.

The frames of text

This is where the fun starts... We need to create a dataset containing the same columns as above, with each individual frame. Lets go...

The initial setup is below. We add another concept which is the height system using the column hsys. Here we opt for the "point size (text only)" system to help define the point size of the text. More information can be found here: SAS Help Center: HSYS Variable

    * Create a dataset containing the individual frames;
    data work.annotate_rain;

      * Define variables;
      attrib
              FrameNum      length=8.
              xsys          length=$1.
              ysys          length=$1.
              hsys          length=$1.
              function      length=$32.
              style         length=$32.
              color         length=$32.
              text          length=$15.
              ;
        
      * Define the coordinate system;
      * https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/graphref/annotate_xsys.htm;
      xsys='1'; ysys='1';

      * Define the height system as point size;
      * This system only works for text;
      *https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/graphref/annotate_hsys.htm;
      hsys='D';

Given that here we are dealing with all text annotations, we can define the function, style and size for text annotations.

      * All of the annotate commands will have the same function, style & size;
      function='text';
      style='';
      size=&textsize.;

Before we get to some looping-fun, we need to initialise some variables. I opted to consider the area of the output to be an n x n grid. This concept helped explain the positioning of each character or element by either a column or a row. Using that concept, we define some starting positions of frame starting columns and row.

      * Frame Start Column - This the frame number where the column (or rain drop) will appear;
      FrameStartCol=0;

      * Frame Start Row - similar to FrameStartCol and this allows us to iterate through the frames;
      * for that particular drop;                    
      FrameStartRow=0;

Using the analogy of rain-drops falling on a window, we will create 500 rain drops of matrix code:

      * Create a do-loop to iterate between each rain drop;
      do i=1 to 500;

FrameStartCol defines when this individual rain drop will appear in the animation. For early frames, we want to populate an initial bulk of rain drops.

          * Randomise the frame interval between the start of each rain drop;
          if FrameStartCol<=15 then FrameStartCol+rand('integer',1,6);
          else if FrameStartCol>15 then FrameStartCol+rand('integer',-3,5);

Hopefully, we will never get to this point, but just in case, we need to exit if FrameStartCol is less than zero.

          * Exit if FrameStartCol is lower than 0;
          if FrameStartCol<0 then stop;

For each given rain drop, it has a consistent x-axis coordinate value. Lets randomise this.

          * Define the x-coordinate (left to right) using a random number generator;
          * Between 5% and 95% of the width of the output;
          x=rand('integer',5,95);

Now, we start working on the y-axis. What we want is that a character appears at the top (y=99) in a bright white colour. Each character will appear for a set number of frames, getting darker and darker. Subsequent characters will appear below that character (lower value of y) and follow the same pattern for the same number of frames. This should continue until y<0 and disappears from view.

          * Now we have a column, we need to then iterate through;
          * Do this whilst our Y coordinate is >0 (that is above the horizon);
          * This will then create a text character with the given spacing downwards;
          ThisRow=1;
          y=(&y_start.-(&colspace.*ThisRow)+&colspace.);

          * Iterate whilst the y-coordinate is greater than 0;
          * Once y<=0 then we have gone off the printer output;
          do while (y>0);

We opt to process the first time the rain drop has appeared:

            * For the first time this raindrop has been created the set the 
            * FrameStartRow as the same as FrameStartCol;
            * This will allow us to iterate through the frames for that raindrop;
            if ThisRow=1 then FrameStartRow=FrameStartCol;

We generate a random character for that point:

            * Generate a random char for that point;
            char=rand('integer',1,36);
            Text=put(char,alphabet.);

Increment the frame start row value

            * Increment FrameStartRow;
            FrameStartRow=FrameStartRow+1;

Define the y-axis coordinate:

            * Define the y-coordinates;
            y=(&y_start.-(&colspace.*ThisRow)+&colspace.);

At this point, we know the:

• X-axis coordinate held in the variable x.
• Y-axis coordinate held in the variable y.
• Character to print stored in the variable text.

Now we know this, we need to create the next set of frames for this character at the (x,y) position by slowly changing the colour of the text. Using the example where we create 20 frames per character:

            * Create the next Frames;
            * Assume that we only need to do 20 frames for each character of text;
            * That is the text is only visible for 20 frames;
            do F=1 to 20;
      
              * Define the FrameNum;
              FrameNum=FrameStartRow+F;

              * Define the color of the text depending on the value of F;
              color=put(F,mtrxcl.);

              * Define the text;
              Text=put(char,alphabet.);
              output; 

Adding a bit of pizzazz to the output, for those avid watchers of the film, they will notice that the leading edge of the raindrop isn't defined as a cleaned character. It is as if the matrix hasn't yet decided what character to show. Whilst we are limited to what we can do, we can replicate the look of this by added extra characters on the earlier frames at the leading edge. This is achieved by:

              * On the matrix rain, the leading edge of the raindrop has multiple characters;
              * So overlay multiple random letters for the first few frames of the leading edge;
              if F<=3 then do;
                do r=1,2,3;
                  Text=put(rand('integer',1,36),alphabet.);
                  output; 
                end;
              end;

Once we have iterated through the 20 future frames, we want to increment the row and move down the y-axis for the next drop of rain.

            * Increment Row and move down the y-axis;
            ThisRow+1;
            y=(&y_start.-(&colspace.*ThisRow)+&colspace.); 

          end; /* End of (do while y>0) */

      end; /* End of (do i=1 to...) */

    run;

Appending the result of that table to the background, we have all of the data points for the annotation.

    * Append the boarder and rain;
    data work.make_it_rain;
    set work.annotate_background
        work.annotate_rain
        ;
    run;

Creating the output

Whilst I'm not an expert when it comes to SAS graphics, I've attempted to comment the output within the code.

The main points to look for are the options for animation=start and animation=stop.

Before we do that, we need to find out the maximum number of frames to loop over in the GIF. This will be passed into a macro variable called maxframes and then used in a subsequent macro to loop over.

    * Put into macro variable table the number of frames;
    * Chop 100 frames off the end so that it does not end with a blank screen;
    proc sql noprint;
    select max(FrameNum)-1 into: MaxFrames
    from work.annotate_rain 
    ;
    quit;

Now the fun stuff...

  * Define options;
  options dev=sasprtc 
          printerpath=gif                 /* show that we are outputing to GIF format */
          animation=start                 /* start building the animation */
          papersize=('8 in', '8 in') 
          animduration=&animduration.     /* the value of the time between frames                */
          animloop=yes                    /* yes = signals we want to keep the animation looping */
          noanimoverlay
          ;

  * Define the file we are ouputting to;
  ods printer file="rain.gif";

  * Define the output graphics size;
  ods graphics / width=800px height=800px imagefmt=gif;

  options nodate nonumber nobyline;
  ods listing select none;

  * Define the font used for the text;
  * Courier New looks more like terminal font;
  goptions ftext='Courier New';

  %* Create a macro to create a new gchart using proc ganno;
  %* Each chart will be a combination of FrameNum 0 (the background) and the given ThisFrameNum;
  %macro print_frames;

  %do ThisFrameNum=1 %to &MaxFrames.;
    proc ganno annotate=work.make_it_rain (where=(FrameNum in (0,&ThisFrameNum.)));
    run;
    quit;
  %end;

  %mend;
  %print_frames;

  * Stop printing and the animation;
  options printerpath=gif animation=stop;               
  ods printer close;                                    

  quit;
  ods html close;
  ods listing;

The Finished Animation

I think Neo would be proud:

Full Code

*********************************************************************************;
**      Script: make_it_rain.sas                                               **;
** Description: Create a GIF to replicate Matrix's digital rain                **;
**  Created by: Clark Lawson                                                   **;
**  Created on: 29/07/2022                                                     **;
**     Contact: https://www.linkedin.com/in/clarklawson/                       **;
*********************************************************************************;

  * -----------------------------------------------------------------------------;
  * Part 1: Define some parameters                                               ;
  * -----------------------------------------------------------------------------;
  *         This is in a data step rather than only macro statements as we       ;
  *         might be working with decimal values                                 ;
  * -----------------------------------------------------------------------------;

    data _null_;

      * Set the frames per second;
      fps=7;

      * Define the animation duration using FPS;
      animduration=1/fps;

      * Spacing between characters;
      charspace=4;
      colspace=2;

      * Define the text size in points;
      textsize=12;

      * Define the start y (vertical) position of text;
      y_start=99;

      * Put them to the macro variable list;
      call symputx('fps',fps);
      call symputx('animduration',put(animduration,6.4));
      call symputx('charspace',charspace);
      call symputx('colspace',colspace);
      call symputx('textsize',textsize);
      call symputx('y_start',y_start);

    run;

    * Show to the log;
    %put &=fps. &=animduration. &=charspace. &=colspace. &=textsize. &=y_start.;

  * -----------------------------------------------------------------------------;
  * Part 2: Define an informat for mapping characeters from numbers              ;
  * -----------------------------------------------------------------------------;
  *         Here I attempted to use a more visual font in a matrix style but     ;
  *         this did not work. So ended up sticking with standard characters     ;
  * -----------------------------------------------------------------------------;

    * Create a dataset that will feed into proc format;
    data work.alphabet_format (drop=alphabet);

      * Use the full alphanumeric list;
      alphabet = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890';

      fmtname = 'alphabet';
      do label=1 to length(alphabet);
        start=substr(alphabet, label, 1);
        /* Type = 'i' for the Text-to-Number informat, and */
        /* Type = 'n' for the Number-to-Text format. */
        type = 'n'; output;
      end;
    run;

    /* Now feed the dataset into PROC FORMAT to build the format/informat */
    proc format cntlin=Alphabet_Format (where = (type = 'n') rename = (start = label label = start)); 
    run;

  * -----------------------------------------------------------------------------;
  * Part 3: Define an format for defining the text colour                        ;
  * -----------------------------------------------------------------------------;
  *         The colour of the text depends on how many frames it has been visible;
  *         for. Early frames are white, turn light green and then darker        ;      
  * -----------------------------------------------------------------------------;

    * Need to call the colormac macro statement;
    * This allows us to use the RGB macro;
    * https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/graphref/n0f54aj6pc5nkbn1jvl6bcoc240y.htm;
    %colormac; 

    * Colors taken from Matrix Colors after Googling!!;
    %let black=%RGB(13,2,8);
    %let darkgreen=%RGB(0,59,0);
    %let middlegreen=%RGB(0,143,17);
    %let lightgreen=%RGB(0,255,65);

    * Print to log for debug;
    %put &=black. &=darkgreen &=middlegreen. &=lightgreen.;

    * Use a format to define the frame number and the colour;
    * In this example, the first 5 frames will be white then...;
    proc format;
      value mtrxcl 
         1- 5='white'
         6-10="&lightgreen."
        11-15="&middlegreen."
        16-20="&darkgreen."
        other="&black."
      ;
    run;

  * -----------------------------------------------------------------------------;
  * Part 4: Creation of the data for PROC GANNO                                  ;
  * -----------------------------------------------------------------------------;
  *         A PROC GANNO statement takes an input dataset which is a list of     ;
  *         commands to execute to draw things like boxes and position text      ;
  * -----------------------------------------------------------------------------;
  *   Note: We will use FrameNum here so that we can select a different frame    ;
  *         when we are running multiple PROC GANNO statements to create the GIF.;
  *         FrameNum=0 will always be output as this is the background.          ;                                    
  * -----------------------------------------------------------------------------;

    * Create a dataset containing the background box;
    data work.annotate_background;

      * Define variables;
      attrib
              FrameNum      length=8.
              xsys          length=$1.
              ysys          length=$1.
              function      length=$32.
              style         length=$32.
              color         length=$32.
              text          length=$15.
              ;
        
      * Define the coordinate system;
      * https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/graphref/annotate_xsys.htm;
      xsys='1'; ysys='1';

      * Create my data to help with what Frame we are on;
      FrameNum=0; * Frame zero will always be printed as this is the background;

      * Create the instructions to draw the background box;

        * Define the colour;
        color="&black.";

        * Move to the position (0,0) in the coordinate system;
        function="move"; x=0; y=0; output;

        * Draw a bar (aka box) from the current position to (0,0), (100,100) with a solid colour;
        function="bar";  x=100; y=100; style="solid"; output;

    run;

    * Create a dataset containing the individual frames;
    data work.annotate_rain;

      * Define variables;
      attrib
              FrameNum      length=8.
              xsys          length=$1.
              ysys          length=$1.
              hsys          length=$1.
              function      length=$32.
              style         length=$32.
              color         length=$32.
              text          length=$15.
              ;
        
      * Define the coordinate system;
      * https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/graphref/annotate_xsys.htm;
      xsys='1'; ysys='1';

      * Define the height system as point size;
      * This system only works for text;
      *https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/graphref/annotate_hsys.htm;
      hsys='D';

      * All of the annotate commands will have the same function, style & size;
      function='text';
      style='';
      size=&textsize.;


      * Frame Start Column - This the frame number where the column (or rain drop) will appear;
      FrameStartCol=0;

      * Frame Start Row - similar to FrameStartCol and this allows us to iterate through the frames;
      * for that particular drop;                    
      FrameStartRow=0;

      * Create a do-loop to iterate between each rain drop;
      do i=1 to 500;

          * Randomise the frame interval between the start of each rain drop;
          if FrameStartCol<=15 then FrameStartCol+rand('integer',1,6);
          else if FrameStartCol>10 then FrameStartCol+rand('integer',-3,5);

          * Exit if FrameStartCol is lower than 0;
          if FrameStartCol<0 then stop;

          * Define the x-coordinate (left to right) using a random number generator;
          * Between 5% and 95% of the width of the output;
          x=rand('integer',5,95);

          * Now we have a column, we need to then iterate through;
          * Do this whilst our Y coordinate is >0 (that is above the horizon);
          * This will then create a text character with the given spacing downwards;
          ThisRow=1;
          y=(&y_start.-(&colspace.*ThisRow)+&colspace.);

          * Iterate whilst the y-coordinate is greater than 0;
          * Once y<=0 then we have gone off the printer output;
          do while (y>0);

            * For the first time this raindrop has been created the set the 
            * FrameStartRow as the same as FrameStartCol;
            * This will allow us to iterate through the frames for that raindrop;
            if ThisRow=1 then FrameStartRow=FrameStartCol;

            * Generate a random char for that cell;
            char=rand('integer',1,36);
            Text=put(char,alphabet.);

            * Increment FrameStartRow;
            FrameStartRow=FrameStartRow+1;

            * Define the y-coordinates;
            y=(&y_start.-(&colspace.*ThisRow)+&colspace.);

            * Create the next Frames;
            * Assume that we only need to do 20 frames for each character of text;
            * That is the text is only visible for 20 frames;
            do F=1 to 20;
      
              * Define the FrameNum;
              FrameNum=FrameStartRow+F;

              * Define the color of the text depending on the value of F;
              color=put(F,mtrxcl.);

              * Define the text;
              Text=put(char,alphabet.);
              output; 

              * On the matrix rain, the leading edge of the raindrop has multiple characters;
              * So overlay multiple random letters for the first few frames of the leading edge;
              if F<=3 then do;
                do r=1,2,3;
                  Text=put(rand('integer',1,36),alphabet.);
                  output; 
                end;
              end;

            end;

            * Increment Row and move down the y-axis;
            ThisRow+1;
            y=(&y_start.-(&colspace.*ThisRow)+&colspace.); 

          end;

      end;

    run;

    * Append the boarder and rain;
    data work.make_it_rain;
    set work.annotate_background
        work.annotate_rain
        ;
    run;

  * -----------------------------------------------------------------------------;
  * Part 5: Creation the animation                                               ;
  * -----------------------------------------------------------------------------;
  *         Define the setings to make a GIF and lopp through the PROC GANNO     ;
  *         multiple times                                                       ;
  * -----------------------------------------------------------------------------;
  
    * Put into macro variable table the number of frames;
    * Chop 100 frames off the end so that it does not end with a blank screen;
    proc sql noprint;
    select max(FrameNum)-1 into: MaxFrames
    from work.annotate_rain 
    ;
    quit;

  * Define options;
  options dev=sasprtc 
          printerpath=gif                 /* show that we are outputing to GIF format */
          animation=start                 /* start building the animation */
          papersize=('8 in', '8 in') 
          animduration=&animduration.     /* the value of the time between frames                */
          animloop=yes                    /* yes = signals we want to keep the animation looping */
          noanimoverlay
          ;

  * Define the file we are ouputting to;
  ods printer file="rain.gif";

  * Define the output graphics size;
  ods graphics / width=800px height=800px imagefmt=gif;

  options nodate nonumber nobyline;
  ods listing select none;

  * Define the font used for the text;
  * Courier New looks more like terminal font;
  goptions ftext='Courier New';

  %* Create a macro to create a new gchart using proc ganno;
  %* Each chart will be a combination of FrameNum 0 (the background) and the given ThisFrameNum;
  %macro print_frames;

  %do ThisFrameNum=1 %to &MaxFrames.;
    proc ganno annotate=work.make_it_rain (where=(FrameNum in (0,&ThisFrameNum.)));
    run;
    quit;
  %end;

  %mend;
  %print_frames;

  * Stop printing and the animation;
  options printerpath=gif animation=stop;               
  ods printer close;                                    

  quit;
  ods html close;
  ods listing;

  * -----------------------------------------------------------------------------;
  * End of Code                                                                  ;
  * -----------------------------------------------------------------------------;

[ClarkLawson]

@Ksharp has posted another method using PROC IML here:

Fun With SAS ODS Graphics: Matrix Plot - SAS Support Communities