How to suppress warning from PROC NLIN

[rholowczak]

I have a very large data set (100GB) and need to run PROC NLIN on each set of observations.

The parameters from PROC NLIN were given to me and I am not able to change any of them.

In some cases, the models fails to converge which generates a set of lines to the log file:

NOTE: PROC NLIN grid search time was 0: 0: 0.
WARNING: PROC NLIN failed to converge.
NOTE: Negative model SS. Check model and initial parameters.
NOTE: The (approximate) Hessian is singular.
NOTE: The above message was for the following BY group: obs=420

I have done the following just before the call to PROC NLIN:

options nonotes ;
proc printto print='/dev/null' new;
run;

However the line "WARNING: PROC NLIN failed to converge." is still printed to the log.

Is there any way to suppress this warning (or all similar warnings)?

This is on SAS 9.3 TS1M2

[RW9]

In the documentation:

https://support.sas.com/documentation/cdl/en/statug/63033/HTML/default/viewer.htm#statug_nlin_sect02...

Unable to Improve

The method can lead to steps that do not improve the estimates even after a series of step halvings. If this happens, the procedure issues a message stating that it is unable to make further progress, but it then displays the following warning message:

   PROC NLIN failed to converge

Then it displays the results. This often means that the procedure has not converged at all. If you provided your own derivatives, check them carefully and then examine the residual sum of squares surface. If PROC NLIN has not converged, try a different set of starting values, a different METHOD= specification, the G4 option, or a different model.

[rholowczak]

Thanks for the pointers. Yes I have read this documentation many times. Unfortunately, the PROC NLIN and its parameters was determined by another programmer and I am not permitted to change it.  All I really need to do is prevent the WARNING from showing up in the log file. 

[PGStats]

Why? Those warnings are instructive and essential in interpreting the results. It would seem more reasonable to grab table ConvergenceStatus with ODS OUTPUT and reflect those warnings into the results. Ignoring them would just be irresponsible.

[rholowczak]

Ultimately the input file will be filtered for these condiutions. For now the results are being saved along with whether or not the model converged. So we are explicitly not ignoring the model results and warnings. What we are trying to do is complete some runs on the whole data set (tens of billions of records) without filling up the log file with 200GB of WARNING text.

[rholowczak]

'm posting this as an answer. It should have been obvious but in case others come across this. within a SAS program you can temporarily re-direct the output of warnings and errors to a different log file using the following syntax (on Linux/Unix):

proc printto print='/dev/null' log='/dev/null' new;
run;

I had been using the print= part but not the log= part. To return to regular logging just run:

proc printto
run;

[RW9]

Hi,

Well, glad you found something that will work for you.  However I would just add a final caution.  What you are doing is simply hiding the problem.  Thisto me, is worse than messages in the log as you have acknowledged there is a problem and are hiding it.  All warnings/errors/notes can be fixed and they are they for good reasons which may not be be visible immediately.  Hence if code came to me which had errors/warnings/certain notes, I fail that code, and if I saw re-directions of problem log output that would be something I would raise with QA as a problem.  If the NLIN is throwing issues, and someone has approved that it is correct and should not be changed, then just put that information into the log before the code run, e.g:

%put This code throws a problem, but the statistician AB 12NOV2014 confirmed this is correct and should be kept as is;

...the code...

%put End of known problem.;

This then shows traceability, and that you have checked the problem.

[rholowczak]

Thanks again for the reply. As I wrote previously, we are aware of the data issues leading to the NLIN warnings and are explicitly capturing the parameters and observations to be used for filtering in the future.  Initially, being able to complete a run through a data set was more important. Now that we have a way to do that we can take the next step to filter the data accordingly.

[Rick_SAS]

For tips and techniques for monitoring convergence in simulations and other situations, see the article

"Monitor convergence during simulation studies in SAS."