topic Re: Proc nlin method in Developers

Proc nlin method

herbman — Thu, 14 Jul 2016 14:05:22 GMT

Hey, when excuting proc nlin in SAS 9.3 and SAS enterprise guide 5.1 stored process using method Marquardt, I got two different iteration result. I am wondering was this really because my setting or ther is potential difference between this two platform for running proc nlin using Gauss or Marquardt methods?

Thank you!

Herbman

Re: Proc nlin method

Rick_SAS — Thu, 14 Jul 2016 17:15:14 GMT

Enterprise Guide 5.1 is just an interface that submists SAS programs to whatever SAS server it is connected to. Thus the real issue is what version of SAS id running the code (assuming that the data is the same).

You can submit the following SAS code and then view the Log to see the version of SAS that is running on the server:

%put SYSVLONG = &SYSVLONG;

You'll see something like

SYSVLONG = 9.03.01M2D082312

which is interpretted as SAS 9.3, maintenance 2.

Since you are getting different behavior, you will presumably find that you are comparing two different versions of SAS. Each version is SAS is compiled using potentially different compiler versions and compiler optimizations. It is not unusual to see slightly different results as the result of a statistical optimization problem (which PROC NLIN solves).

Re: Proc nlin method

herbman — Thu, 14 Jul 2016 17:32:20 GMT

Thank you, I ran %put SYSVLONG = &SYSVLONG; you sent, both the server and sas codes show: SYSVLONG = 9.03.01M2P081512. So this means they ran under exact the same enviroment. It is really confusing when I copied the same code into enterprise guide and SAS code did show susscessful converge when using proc nlin and method=marquardt but the stored process showed failure to converge.

Re: Proc nlin method

Rick_SAS — Thu, 14 Jul 2016 18:23:41 GMT

Well, if it's not the SAS version, then it must be the operating system or the bitness of the OS. Submit:

%put SYSSCP= &SYSSCP;
%put SYSSCPL = &SYSSCPL;

I'm guessing that you'll either see Windows vs Linux or 32-bit vs 64-bit.

The bigger issue is what to do about the problem. Look at the iteration history table to see why it is not converging. (If there is no iteration history, make sure that the NOITHIST option is not used.)

Some reasons why the convergence problems can occur are

1) The sample size of the data is small

2) The model does not fit the data

3) The initial guess for the parameters is not good

There's not a lot you can do about (1). (2) is a modeling problem. For (3) try using the optimal values for the program that converges as initial values on the PARMS statement in the program that does not converge. This should help the second program converge to the same optimal solution. You can also specify a grid of parameter values, if necessary.

If all this fails, look at the "Troubleshooting" section of the doc.

Re: Proc nlin method

herbman — Fri, 15 Jul 2016 18:13:41 GMT

Hi Rick,

Thank you. Yes, I ran the two macro and the sas server information is:

SYSSCP= LIN X64

SYSSCPL = Linux

and my local sas is:

SYSSCP= WIN

SYSSCPL = W32_7PRO

and when the 5 parameter proc nlin code run (based on previous 4 parameter code):

proc nlin data=nlinready outest=est5 method=marquardt maxiter=32000 maxsubit=100;

by byval;

parms / pdata=initialval5;

bounds c5>0;

model rlu=c1+(c2-c1)/(1+exp(-c3-c4*log(dose)-c5*dose));

_weight_=1/rlu**2;

ods output parameterestimates=params5 anova=stats5 convergencestatus=converge5;

run;

quit;

I noticed the number of observation difference of output EST5 between the server and local sas. and output data params5 has 5 obs and 9 variables in local SAS and 6 obs and 10 variable and 10 variables as indicated by the uploaded files (original is server log and the revision is local sas log).

Any further suggestions to make them consistent?

Thank you very much!

Herbman

Re: Proc nlin method

herbman — Fri, 15 Jul 2016 18:26:44 GMT

Now the real concern is the inconsistence between the server and local sas, as I replied below. Thank you!

Re: Proc nlin method

Rick_SAS — Fri, 15 Jul 2016 19:03:16 GMT

One system is 64-bit Linux. The other is 32-bit Windows. These systems use different compilers, compiler optimization flags, possibly different linear algebra BLAS and vendor-supplied math libraries,.... and the list goes on. Numerically speaking, there is no reason to expect that these systems will give exactly the same answers to a complex optimization problem.

I don't follow what you are trying to accomplish statistically, but if you are using the results of a previous run to provide initial estimates for a subsequent run, then these little differences can add up.

Based on what you've told me and what I've inferred, I don't see any way to guarrantee that the results are exactly the same for both runs. Check the Iteration History to see how/when the optimizations diverge from each other.