topic Re: how catch floating point error in an ods? in Statistical Procedures

how catch floating point error in an ods?

pmbrown — Thu, 03 Aug 2023 06:43:13 GMT

unfortunately Rick Wicklin's neat solution here: https://communities.sas.com/t5/Statistical-Procedures/NLMIXED-difference-between-convergence-status-and-non-positive/td-p/710934

does not work for this scenario: https://communities.sas.com/t5/Statistical-Procedures/proc-phreg-ties-handling-floating-point-zero-divide/td-p/496970

i'm wondering how to catch the error in a dataset when running a large macro containing several phreg's. Can Rick's code be modified in some way? That would be ideal because i am already making use of it

Re: how catch floating point error in an ods?

JosvanderVelden — Thu, 03 Aug 2023 09:41:35 GMT

Both threads you mention are marked as solved. Can you elaborate on why Rick's solution doesn't work for the scenario from the second post. Are you using the latest SAS 9.4 release (SAS 9.4M8 shipped in January 2023) or an older release?

Re: how catch floating point error in an ods?

pmbrown — Thu, 03 Aug 2023 10:04:47 GMT

it doens't work for the scenario metioned, i don't know why.

re version: i'm not looking to solve the problem, i am only looking to catch it

re they are marked as solved: that's why i had to create a new thread, they are closed (unfortunately)

Re: how catch floating point error in an ods?

sbxkoenk — Thu, 03 Aug 2023 10:16:06 GMT

Hello,

What procedure are you using? PROC PHREG?

I do not think there's any mentioning of "floating point zero divide error" in the output. So, ODS (Output Delivery System) solutions are impossible.

You will have to parse the log.

If you have BY-groups , analyzing the LOG by means of a self-written LOG-Analyzer ... should tell you which BY group to look at.

Koen

Re: how catch floating point error in an ods?

pmbrown — Thu, 03 Aug 2023 11:09:14 GMT

re ods solutions are impossible: that's right, it won't show up in the covergencestatus ods

re parse the log: but i need to catch it in a dataset, that's what's great about Rick Wicklin's solution, and i'm wondering how to adjust his code to catch this error, because it does not catch it

Re: how catch floating point error in an ods?

JosvanderVelden — Thu, 03 Aug 2023 11:13:20 GMT

Can you describe the scenario where it does not catch it, with a small data and code example?

Re: how catch floating point error in an ods?

pmbrown — Thu, 03 Aug 2023 12:14:31 GMT

sure

proc phreg data=...;
  class groupvar (ref=first) arm  / param=ref;
  model aval*cnsr(1,2)= groupvar arm  / risklimits=pl ties=exact; 
  ods output ParameterEstimates=.... ;
  estimate 'G1' groupvar 1 /cl exp; 
run;

NOTE: Convergence criterion (GCONV=1E-8) satisfied.
ERROR: Floating Point Zero Divide.
WARNING: Output 'ParameterEstimates' was not created.  Make sure that the output object name, label, or path is spelled correctly.  
         Also, verify that the appropriate procedure options are used to produce the requested output object.  For example, verify 
         that the NOPRINT option is not used.
ERROR: Termination due to Floating Point Exception
NOTE: The SAS System stopped processing this step because of errors.
NOTE: The PROCEDURE PHREG printed pages 157-158.
NOTE: PROCEDURE PHREG used (Total process time):
      real time           0.69 seconds
      cpu time            0.69 seconds

i can't give data, it may be the case that the event rate is low, but i'm not at all interested in that, i am only interested in how i can catch this error in a dataset

Re: how catch floating point error in an ods?

Quentin — Thu, 03 Aug 2023 12:34:41 GMT

When try to use Rick's solution, what happens? It's not clear to me what you mean by it does not work. His solution is storing the value of the automatic macro variable &SYSWARNINGTEXT. If you want to store the error message, you could change the macro call to:

%MonitorErrors(1, &SysErr, "&SysErrorText")

But as a more basic test, can you post the full log from running:

proc phreg data=...;
  class groupvar (ref=first) arm  / param=ref;
  model aval*cnsr(1,2)= groupvar arm  / risklimits=pl ties=exact; 
  ods output ParameterEstimates=.... ;
  estimate 'G1' groupvar 1 /cl exp; 
run;

%put &=SysErr &=SysWarningText &=SysErrorText ;

As long as PROC PHREG is appropriately setting SYSERR to a non-zero value, I think Rick's method should work. But if PHREG is not setting SYSERR, then you might have to resort to log scanning, or checking for the existence of the output ParameterEstimates data set. If you're running the code as a batch job, it's possible the error is causing the SAS session to abort, but there are options to change that. And that is not shown in the log messages you posted.

Re: how catch floating point error in an ods?

sbxkoenk — Thu, 03 Aug 2023 12:44:49 GMT

Hello,

I would first try what @Quentin has suggested 10 or 15 minutes ago.

But in case you need to revert to LOG-parsing

... you should know that a proper LOG-parser (LOG-analyzer) , that you have to write yourself, will also organize everything in a structured dataset.

I am sure you can find many SAS Log parsers (SAS Log analyzers) on internet.
You can start here :
https://www.lexjansen.com/search/searchresults.php?q=log%20parser

SAS Tip: Learn lexjansen.com
https://communities.sas.com/t5/SAS-Tips-from-the-Community/SAS-Tip-Learn-lexjansen-com/m-p/436336#M26

Koen

Re: how catch floating point error in an ods?

Tom — Thu, 03 Aug 2023 12:53:52 GMT

It seems to say in the log that the ODS output was not created. So if you use ODS OUTPUT to tell it make a dataset of that ODS output then you can later test if the dataset is created or not. Or perhaps if using BY group processing that it has all of the BY groups it should have.

Re: how catch floating point error in an ods?

pmbrown — Thu, 03 Aug 2023 14:39:54 GMT

apparently syserr is 0 because it's resulting in msg=OK for everything, ie i run the model for many scenarios and it always reports syserr=0 even when there is an error. Like this:

SYSERR=0 SYSWARNINGTEXT=Output 'ParameterEstimates' was not created.  Make sure that the output object name, label, or path is 
spelled correctly.  Also, verify that the appropriate procedure options are used to produce the requested output object.  For 
example, verify that the NOPRINT option is not used. SYSERRORTEXT=Termination due to Floating Point Exception

the solution then seems to be to collect syserrortext, as you suggested, regardless of the value of syserr, and then use the index function (or whatever) to scan the text. Edit: or Tom's suggestion re checking for the parametestimates dataset

Re: how catch floating point error in an ods?

pmbrown — Thu, 03 Aug 2023 14:42:08 GMT

thanks, yes i scan the log. The reason i want the info in a dataset is so i can write code to suppress unreliable model estimates from a summary table, or at least flag them.

Re: how catch floating point error in an ods?

Quentin — Thu, 03 Aug 2023 14:54:38 GMT

That's unfortunate that SYSERR is not being set when there is an error. I recommend you submit that to tech support as a bug.

The tricky thing about SYSERRORTEXT and SYSWARNINGTEXT is that they are not reset to blank at a step boundary. So you can't simply test the value of these variables. But probably you could use the existence of the parameterestimates output dataset as the switch instead of syserr. So something like:

%macro MonitorErrors(ModelID, ParameterEstimatesData, Text);
   data ErrorMsg;
      length msg $200;
      ModelID = &ModelID;
      %if %sysfunc(exist(&ParameterEstimatesData)) %then %do;
         msg = "OK";
      %end;
      %else %do;
         msg = &Text;
      %end;
   run;
   proc append base=Errors data=ErrorMsg force; run;
%mend;

With that approach, assuming you're running multiple PHREG steps, you would need to PROC DELETE the parameter estimates data after each step. You could do that with the above macro, or outside of it.

Re: how catch floating point error in an ods?

pmbrown — Thu, 03 Aug 2023 18:17:26 GMT

excellent, i will try this asap.

re they are not reset to blank, i just noticed this, ie there's is one model causing the problem, but the errors i capture suggest every model after this problem model is a problem, as if the value is retained

i think syserr has been useful for me when catching the non-poitive definite warning message, so maybe i need a combination of syserr and detecting if parameterestimatesdata exists

Re: how catch floating point error in an ods?

Quentin — Thu, 03 Aug 2023 18:54:24 GMT

@pmbrown wrote:

excellent, i will try this asap.

i think syserr has been useful for me when catching the non-poitive definite warning message, so maybe i need a combination of syserr and detecting if parameterestimatesdata exists

Yes, that's a good idea to check both. Something like:

%if &syserr=0 and %sysfunc(exist(&ParameterEstimatesData)) %then %do; 
...

Re: how catch floating point error in an ods?

data_null__ — Thu, 03 Aug 2023 22:38:17 GMT

You can sort of reset SYSERRORTEXT.

101  data _null_;
102     set notfound;
ERROR: File WORK.NOTFOUND.DATA does not exist.
103     run;

NOTE: The SAS System stopped processing this step because of errors.
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      cpu time            0.00 seconds


104  %put NOTE: &=syserrortext;
NOTE: SYSERRORTEXT=File WORK.NOTFOUND.DATA does not exist.
105  %put ERROR: %str( );
ERROR:
106  %put NOTE: &=syserrortext;
NOTE: SYSERRORTEXT=

Re: how catch floating point error in an ods?

Quentin — Fri, 04 Aug 2023 13:31:45 GMT

@data_null__ wrote:

You can sort of reset SYSERRORTEXT.

101  data _null_;
102     set notfound;
ERROR: File WORK.NOTFOUND.DATA does not exist.
103     run;

NOTE: The SAS System stopped processing this step because of errors.
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      cpu time            0.00 seconds


104  %put NOTE: &=syserrortext;
NOTE: SYSERRORTEXT=File WORK.NOTFOUND.DATA does not exist.
105  %put ERROR: %str( );
ERROR:
106  %put NOTE: &=syserrortext;
NOTE: SYSERRORTEXT=

Wow, thanks DN, that's just the kind of insightful tidbit that makes SAS communities so valuable. You never know what you will learn by participating in a discussion thread.

Re: how catch floating point error in an ods?

pmbrown — Thu, 10 Aug 2023 14:18:00 GMT

i accept this as the solution, although it turns out that when the floating point error occurs it terminates the program. I was hoping it would continue to run analyses with the problem analysis flagged, but i need to write code to detect when a problem analysis might arise and prevent it from running. For what it's worth i will continue using the macro to catch warnings

%macro monitorerrs(modelID,errornum,ParameterEstimatesData);

data errmsg&modelID;
  length msg $200;
*put model ID and syserr in dataset;
  modelID = &modelID;
  syserr = &errornum;
*put message in dataset;
  %if &errornum gt 0 %then %do;
      msg = "&SysWarningText"; /*eg, pos def, ridging etc*/
  %end;
  %else %if %sysfunc(exist(&ParameterEstimatesData)) %then %do; 
     msg = "OK";
  %end;
  %else %do;
      msg = "&SysErrorText";
  %end;
run;

%mend monitorerrs;

Re: how catch floating point error in an ods?

Quentin — Thu, 10 Aug 2023 17:22:01 GMT

You could try setting system option NOERRORABEND if you want the SAS session to keep running despite errors.