Animation Running Slow

[lindseyn]

I am running this in SAS 3.8. I am attempting to create an animated map that has the points appear over time as the date increases. My data set includes a date column, latitude, and longitude. Below is the code I am using and it has given me no errors however, it is running extremely slow. I do not have the log because I have to terminate the session each time I try to run it. I originally thought it was because my dat set was too large but then I tried it on a test set of only 20 rows and the program still is running slow. Any ideas on why this ,ay be? Thanks!

ODS RTF ClOSE;
ods html;
ods listing;

PROC IMPORT OUT=new_data DATAFILE="C:/Users/lindsey/Documents/new_data.csv" DBMS=CSV REPLACE;
GETNAMES=YES;
DATAROW=2; 
RUN;

proc sort data=new_data; by Date_Processed; run;


ods graphics / imagefmt=GIF outputfmt=gif width=6.4in height=4.8in; 
options papersize=('6.4 in', '4.8 in')
		nodate nonumber
		animduration=0.5 animloop=yes noanimoverlay
		printerpath=gif animation=start;
ods printer file='C:\Users\lindsey\Documents\Gifs\map.gif';

ods listing select none;
proc sgmap 
	plotdata=flu_DATA;
	openstreetmap;
	scatter x=LONG y=LATI/ group=Date_Processed name="scatterPlot"
	markerattrs=(size=5 symbol=circlefilled);
run;
ods html select all;

options printerpath=gif animation=stop;
ods printer close;

[GraphGuy]

Try using a smaller animduration value ...

https://go.documentation.sas.com/?docsetId=lesysoptsref&docsetTarget=n0x1g1krtpumv2n1mp91s497n3ca.ht...

-----

correction:

Oops - I thought your animation was running slow, whereas you're actually saying your SAS job is running slow.

[Reeza]

Try turning off the HTML and listing output as well unless you need them.

[lindseyn]

When I turn off listing and HTML I get new erros:

 
 1          OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
 NOTE: ODS statements in the SAS Studio environment may disable 
 some output features.
 73         
 74         ods _all_ close;
 NOTE: ODS PDF(WEB) printed no output. 
       (This sometimes results from failing to place a RUN 
       statement before the ODS PDF(WEB) CLOSE statement.)
 75         
 76         
 77         PROC IMPORT OUT=new
 77       ! DATAFILE="C:/Users/lindsey/Documents/new.csv"
 77       ! DBMS=CSV REPLACE;
 78         GETNAMES=YES;
 79         DATAROW=2;
 80         RUN;
 
 NOTE: Unable to open parameter catalog: 
 SASUSER.PARMS.PARMS.SLIST in update mode. Temporary parameter 
 values will be saved to WORK.PARMS.PARMS.SLIST.
 NOTE: A byte-order mark in the file 
       "C:\Users\lindsey\Documents\new.csv" (for fileref 
       "#LN00175") indicates that the data is encoded in 
       "utf-8".  This encoding will be used to process the file.
 NOTE: A byte-order mark in the file 
       "C:\Users\lindsey\Documents\new.csv" (for fileref 
       "#LN00176") indicates that the data is encoded in 
       "utf-8".  This encoding will be used to process the file.
 81          /***************************************************
 81       ! *******************
 82          *   PRODUCT:   SAS
 83          *   VERSION:   9.4
 84          *   CREATOR:   External File Interface
 85          *   DATE:      18JUL19
 86          *   DESC:      Generated SAS Datastep Code
 87          *   TEMPLATE SOURCE:  (None Specified.)
 88          ****************************************************
 88       ! *******************/
 89             data WORK.NEW    ;
 90             %let _EFIERR_ = 0; /* set the ERROR detection
 90       ! macro variable */
 91             infile
 91       ! 'C:/Users/lindsey/Documents/new.csv' delimiter
 91       !  = ',' MISSOVER DSD lrecl=13106 firstobs=2 ;
 92                informat Date_processed anydtdtm40. ;
 93                informat Lat best32. ;
 94                informat long best32. ;
 95                format Date_processed datetime. ;
 96                format Lat best12. ;
 97                format long best12. ;
 98             input
 99                         Date_processed
 100                        Lat
 101                        long
 102            ;
 103            if _ERROR_ then call symputx('_EFIERR_',1);  /*
 103      ! set ERROR detection macro variable */
 104            run;
 
 NOTE: A byte-order mark in the file 
       "C:\Users\lindsey\Documents\new.csv" (for fileref 
       "#LN00177") indicates that the data is encoded in 
       "utf-8".  This encoding will be used to process the file.
 NOTE: The infile 'C:/Users/lindsey/Documents/new.csv' is:
       Filename=C:\Users\lindsey\Documents\new.csv,
       RECFM=V,LRECL=52424,File Size (bytes)=711,
       Last Modified=18Jul2019:08:45:45,
       Create Time=18Jul2019:08:15:35
 
 NOTE: 19 records were read from the infile 
       'C:/Users/lindsey/Documents/new.csv'.
       The minimum record length was 31.
       The maximum record length was 36.
 NOTE: The data set WORK.NEW has 19 observations and 3 variables.
 NOTE: DATA statement used (Total process time):
       real time           0.04 seconds
       cpu time            0.00 seconds
       
 
 19 rows created in WORK.NEW from 
 C:/Users/lindsey/Documents/new.csv.
   
   
   
 NOTE: WORK.NEW data set was successfully created.
 NOTE: The data set WORK.NEW has 19 observations and 3 variables.
 NOTE: PROCEDURE IMPORT used (Total process time):
       real time           0.15 seconds
       cpu time            0.09 seconds
       
 
 105        
 106        proc sort data=new; by Date_Processed; run;
 
 NOTE: There were 19 observations read from the data set 
       WORK.NEW.
 NOTE: The data set WORK.NEW has 19 observations and 3 variables.
 NOTE: PROCEDURE SORT used (Total process time):
       real time           0.02 seconds
       cpu time            0.00 seconds
       
 
 107        
 108        
 109        ods graphics / imagefmt=GIF outputfmt=gif width=6.4in
 109      !  height=4.8in;
 110        options papersize=('6.4 in', '4.8 in')
 111        nodate nonumber
 112        animduration=0.5 animloop=yes noanimoverlay
 113        printerpath=gif animation=start;
 114        ods printer
 114      ! file='C:\Users\lindsey\Documents\Gifs\testmap.
 114      ! gif';
 NOTE: Writing ODS PRINTER output to DISK destination 
       "C:\Users\lindsey.nelson\Documents\Gifs\testmap.gif", 
       printer "gif".
 115        
 116        ods listing select none;
 ERROR: The LISTING destination is not active; 
        no select/exclude lists are available.
 117        proc sgmap
 118        plotdata=new;
 119        openstreetmap;
 120        scatter x=LONG y=LAT/ name="scatterPlot"
 121        by Date_processed;
             __
             22
             202
 122        markerattrs=(size=5 symbol=circlefilled);
             ___________
             180
 ERROR 22-322: Syntax error, expecting one of the following: ;, 
               DATALABEL, DATALABELATTRS, DATALABELPOS, GROUP, 
               LEGENDLABEL, MARKERATTRS, NAME, NOMISSINGGROUP, 
               TRANSPARENCY.  
 ERROR 202-322: The option or parameter is not recognized and 
                will be ignored.
 ERROR 180-322: Statement is not valid or it is used out of 
                proper order.
 123        run;
 
 NOTE: The SAS System stopped processing this step because of 
       errors.
 NOTE: PROCEDURE SGMAP used (Total process time):
       real time           0.00 seconds
       cpu time            0.00 seconds
       
 124        ods html select all;
 ERROR: The HTML destination is not active; 
        no select/exclude lists are available.
 125        
 126        options printerpath=gif animation=stop;
 127        ods printer close;
 NOTE: ODS PRINTER printed no output. 
       (This sometimes results from failing to place a RUN 
       statement before the ODS PRINTER CLOSE statement.)
 128        
 129        
 130        
 131        
 132        
 133        
 134        OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
 146        

[Reeza]

Run your proc code - you're missing a semicolon which is why you're getting an error.

You did close the LISTING output but then refered to it later, you need to remove listing from that line as well. If you post the actual code + log, we can fix it and post it back, but I would have to remove the line numbers and format and then fix....which is more steps and I'm lazy. If it doesn't work after those changes post back.

FYI-make one change, test it, if it works continue and repeat, if it doesn't fix it first.

[Reeza · Accepted solution]

I decided to test this and made some fake data. The results are useless (attached) but you can see that it does work. Then you can add your own data into it and test it. I took the skeleton code I originally had, change the time intervals for days (month to day) because i was doing daily increments.

%macro mapAnnually(dsn=, start=, end=);
   %let start=%sysfunc(inputn(&start,date9.));
   %let end=%sysfunc(inputn(&end,date9.));
   %let dif=%sysfunc(intck(day,&start,&end));
     %do i=0 %to &dif;
      %let date=%sysfunc(intnx(day,&start,&i,b));


		proc sgmap 

			plotdata=&dsn;
			 where start_date <= &date.;
			openstreetmap;

			scatter x=LONG y=LAT/ markerattrs=(size=5 symbol=circlefilled);
		run;


    %end;
%mend mapAnnually;

ods html close;
/*--Create animation--*/
options papersize=('11 in', '7 in') 
printerpath=gif 
animation=start 
animduration=0.1 
animloop=yes 
noanimoverlay
nodate;
ods printer file='/home/fkhurshed/my_courses/demoMap.gif';

ods graphics / width=10in height=6in imagefmt=GIF;

%mapAnnually(dsn=Cities, start=01Jan2019, end = 06Jan2019);

options printerpath=gif animation=stop;
ods printer close;
ods html;

And the data for this code:

*make fake data;
proc sort data=mapsgfk.uscity_all out=cities;
by pop_type;
run;

data cities;
set cities;
by pop_type;
retain start_date '31Dec2018'd group 0;
if first.pop_type then group+1;
start_date = group + start_date;
keep lat long pop_type start_date group city city2 county state;
format start_date date9.;
run;



proc sort data=cities; 
by start_date; 
run;

[Reeza]

And AFAIK, PROC SGMAP is one of the few procs that currently does not support a BY statement 😞 At least it errored out in SAS 9.4 TS1M5. Maybe it's implemented in M6.

[lindseyn]

Thank you! You are right and that the by statement is not supported by PROC SGMAP. Could you maybe explain what this chunk of code is doing here?

%macro mapAnnually(dsn=, start=, end=);
   %let start=%sysfunc(inputn(&start,date9.));
   %let end=%sysfunc(inputn(&end,date9.));
   %let dif=%sysfunc(intck(day,&start,&end));
     %do i=0 %to &dif;
      %let date=%sysfunc(intnx(day,&start,&i,b));

[Reeza]

That's the control for the dates loops. In the original code, I was incrementing by months which is a more onerous process. In the example I provided it was incrementing about days, which could be simplified. But this is also a generic program that loops through date intervals.

Dates are provided to the macro as 01Jan2019
And the start/end convert them to SAS date values against to use.
The dif value (not the SAS function) calculates the number of days/months/weeks (intervals) between the two dates so it determines how many loops are needed.

[GraphGuy]

Have you tried running this proc sgmap *without* creating gif animation output?

Did that work, and how long did it take?

Are you wanting this to animate over the Date_processed?

If so, you'll probably want to use a "by Date_processed;" (rather than using group=Date_processed).

[lindseyn]

The map itself is created when I just run that chunk of code. However, when I add in the by statement it gives me an error. I am trying to animate over date processed.

Log:

 
 1          OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
 NOTE: ODS statements in the SAS Studio environment may disable 
 some output features.
 73         
 74         ods _all_ close;
 NOTE: ODS PDF(WEB) printed no output. 
       (This sometimes results from failing to place a RUN 
       statement before the ODS PDF(WEB) CLOSE statement.)
 75         ods listing;
 76         ods html;
 NOTE: Writing HTML Body file: sashtml3.htm
 77         
 78         PROC IMPORT OUT=new
 78       ! DATAFILE="C:/Users/lindsey/Documents/new.csv"
 78       ! DBMS=CSV REPLACE;
 79         GETNAMES=YES;
 80         DATAROW=2;
 81         RUN;
 
 NOTE: Unable to open parameter catalog: 
 SASUSER.PARMS.PARMS.SLIST in update mode. Temporary parameter 
 values will be saved to WORK.PARMS.PARMS.SLIST.
 NOTE: A byte-order mark in the file 
       "C:\Users\lindsey\Documents\new.csv" (for fileref 
       "#LN00140") indicates that the data is encoded in 
       "utf-8".  This encoding will be used to process the file.
 NOTE: A byte-order mark in the file 
       "C:\Users\lindsey\Documents\new.csv" (for fileref 
       "#LN00141") indicates that the data is encoded in 
       "utf-8".  This encoding will be used to process the file.
 82          /***************************************************
 82       ! *******************
 83          *   PRODUCT:   SAS
 84          *   VERSION:   9.4
 85          *   CREATOR:   External File Interface
 86          *   DATE:      18JUL19
 87          *   DESC:      Generated SAS Datastep Code
 88          *   TEMPLATE SOURCE:  (None Specified.)
 89          ****************************************************
 89       ! *******************/
 90             data WORK.NEW    ;
 91             %let _EFIERR_ = 0; /* set the ERROR detection
 91       ! macro variable */
 92             infile
 92       ! 'C:/Users/lindsey/Documents/new.csv' delimiter
 92       !  = ',' MISSOVER DSD lrecl=13106 firstobs=2 ;
 93                informat Date_processed anydtdtm40. ;
 94                informat Lat best32. ;
 95                informat long best32. ;
 96                format Date_processed datetime. ;
 97                format Lat best12. ;
 98                format long best12. ;
 99             input
 100                        Date_processed
 101                        Lat
 102                        long
 103            ;
 104            if _ERROR_ then call symputx('_EFIERR_',1);  /*
 104      ! set ERROR detection macro variable */
 105            run;
 
 NOTE: A byte-order mark in the file 
       "C:\Users\lindsey\Documents\new.csv" (for fileref 
       "#LN00142") indicates that the data is encoded in 
       "utf-8".  This encoding will be used to process the file.
 NOTE: The infile 'C:/Users/lindsey/Documents/new.csv' is:
       Filename=C:\Users\lindsey\Documents\new.csv,
       RECFM=V,LRECL=52424,File Size (bytes)=711,
       Last Modified=18Jul2019:08:45:45,
       Create Time=18Jul2019:08:15:35
 
 NOTE: 19 records were read from the infile 
       'C:/Users/lindsey/Documents/new.csv'.
       The minimum record length was 31.
       The maximum record length was 36.
 NOTE: The data set WORK.NEW has 19 observations and 3 variables.
 NOTE: DATA statement used (Total process time):
       real time           0.03 seconds
       cpu time            0.01 seconds
       
 
 19 rows created in WORK.NEW from 
 C:/Users/lindsey/Documents/new.csv.
   
   
   
 NOTE: WORK.NEW data set was successfully created.
 NOTE: The data set WORK.NEW has 19 observations and 3 variables.
 NOTE: PROCEDURE IMPORT used (Total process time):
       real time           0.14 seconds
       cpu time            0.07 seconds
       
 
 106        
 107        proc sort data=new; by Date_Processed; run;
 
 NOTE: There were 19 observations read from the data set 
       WORK.NEW.
 NOTE: The data set WORK.NEW has 19 observations and 3 variables.
 NOTE: PROCEDURE SORT used (Total process time):
       real time           0.02 seconds
       cpu time            0.03 seconds
       
 
 108        
 109        
 110        ods graphics / imagefmt=GIF outputfmt=gif width=6.4in
 110      !  height=4.8in;
 111        options papersize=('6.4 in', '4.8 in')
 112        nodate nonumber
 113        animduration=0.5 animloop=yes noanimoverlay
 114        printerpath=gif animation=start;
 115        ods printer
 115      ! file='C:\Users\lindsey\Documents\Gifs\testmap.
 115      ! gif';
 NOTE: Writing ODS PRINTER output to DISK destination 
       "C:\Users\lindsey\Documents\Gifs\testmap.gif", 
       printer "gif".
 116        
 117        ods listing select none;
 118        proc sgmap
 119        plotdata=new;
 120        openstreetmap;
 121        scatter x=LONG y=LAT/ name="scatterPlot"
 122        by Date_processed;
             __
             22
             202
 123        markerattrs=(size=5 symbol=circlefilled);
             ___________
             180
 ERROR 22-322: Syntax error, expecting one of the following: ;, 
               DATALABEL, DATALABELATTRS, DATALABELPOS, GROUP, 
               LEGENDLABEL, MARKERATTRS, NAME, NOMISSINGGROUP, 
               TRANSPARENCY.  
 ERROR 202-322: The option or parameter is not recognized and 
                will be ignored.
 ERROR 180-322: Statement is not valid or it is used out of 
                proper order.
 124        run;
 
 NOTE: The SAS System stopped processing this step because of 
       errors.
 NOTE: PROCEDURE SGMAP used (Total process time):
       real time           0.00 seconds
       cpu time            0.00 seconds
       
 125        ods html select all;
 126        
 127        options printerpath=gif animation=stop;
 128        ods printer close;
 NOTE: ODS PRINTER printed no output. 
       (This sometimes results from failing to place a RUN 
       statement before the ODS PRINTER CLOSE statement.)
 129        
 130        
 131        
 132        
 133        
 134        
 135        OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
 147        

Code:

proc sgmap 
	plotdata=new;
	openstreetmap;
	scatter x=LONG y=LAT/ name="scatterPlot"
	by Date_processed;
	markerattrs=(size=5 symbol=circlefilled);
run;

[ballardw]

proc sgmap 
	plotdata=new;
	openstreetmap;
	scatter x=LONG y=LAT/ name="scatterPlot" ;
	by Date_processed;
	markerattrs=(size=5 symbol=circlefilled);
run;

Need a semicolon before the BY statement. Without it the BY is treated as an option on the Proc Sgmap statement.

[lindseyn]

Robert,

I found this example of yours that you did using sas 9.4. http://robslink.com/SAS/democd66/walmart_openings_info.htm

I have been trying to play around with this, but cannot seem to get this code to work with my data. How would I alter this code if I already have a data set with the lat, long, and date? I can''t seem to figure out what code does what here. I am essentially trying to create this example but instead of by year by individual dates of recorded flu outbreaks.

[GraphGuy]

Are you able to run my code as-is, without any changes? That would be the first step 🙂

(some things might work, or not work, depending on which release of SAS you're using, what products you have licensed, and whether you're running it in batch/dms/EG/other mode)

[lindseyn]

It does work as is with no errors! I am trying to find an alternative for creating my maps and animating them since sgmap does not seem to work with the animation features. 

However, I am not sure how to work GMAP and the animation that you achieved here with my data.

edit: Really I am just struggling finding good information on GMAP and how to use it like you did in your example. I am unsure about the role of "anno-" or how to create it using data that already provides the lat and long.