The NLPFDD function uses computes numerical first and second derivatives: http://blogs.sas.com/content/iml/2011/10/14/hints-for-derivatives/ ... View more

You can either expand the summary data, or use PROC FREQ to summarize the observed data. For this situation, I'd suggest summarizing the new data by running PROC FREQ on the new observations; see the 2nd example of http://blogs.sas.com/content/iml/2011/09/19/count-the-number-of-missing-values-for-each-variable/ Then you can use a short DATA step to consolidate the old summary and the new summary, and use PROC FREQ with a WEIGHT statement to take the frequencies into account when you do the chi-square test. ... View more

Two comments: 1) Statistically speaking, I'd be careful about using linear regression to extrapolate. In general, use regression to interpolate, and use time series forecasting to extrapolate. 2) If you decide to procede, there is still some work to do. Dan Heath's method will give you the points in the plot. However, you'll have to use two of those points to find the slope-intercept formula for the line of best fit. Once you have the equation, you can fit new points by using the DO loop technique that you mentioned in your post. Good luck, Rick ... View more

See also http://communities.sas.com/message/39698#39698 for the perspective of Time Series analysts. I have seen some people use PROC VARCLUS to group similar variables into clusters of variables with similar behavior. They then model a representative variable from each cluster, or construct the "average" of each cluster and model that.  ... View more

First, if this is a "very basic model" for you, then I'm not sure how much help I can offer, since it doesn't look basic to me! Second, I have to compliment you. You have clearly done your research. You are using NLPNRR to match the fact that PROC MIXED uses a ridge-stabilized Newton-Raphson algorithm. You are using the same ML formula in the PROC MIXED doc. So what is the source of the problem? I don't see anything obvious, but nonlinear optimization is complicated. The PROC MIXED documentation lists 14 different problems that can occur during the MLE, and many of those are general issues apply to general optimization problems. You say "that the model does not converge," but you don't give the reason. Does SAS report an error? (If so, what is it? Can you make the error go away by constraining sigma2>1e-8?) Is the gradient not zero at the final estimate (if so, allow more iterations or change the criteria that stop the algorithm).  One specific suggestion: Take the solution that PROC MIXED gives and plug it into the log-likelihood function that you have define.  Does your IML code give the same gradient and Hessian at that point as does PROC MIXED? If not, your function and PROC MIXED are computing different things.  (The LL is only defined up to a constant, so the LL might not match.) A related suggestion: if you use the solution from PROC MIXED as an initial condition, does the IML code converge?  If so, maybe the intial condition is bad. Rick PS. One small performance suggestion: You don't need a loop to eliminate the missing values. Use the LOC function to find the nonmissing values, then extract them. For example, I think your IF-THEN...DO WHILE code is equivalent to yobs = yi[loc(y^=.)]; For more on removing missing values, see http://blogs.sas.com/content/iml/2010/09/15/removing-observations-with-missing-values/ ... View more

We're getting a lot of posts recently. This is a good time to reiterate these points. ... View more

It's one way, if you are assuming a regression structure and want outliers with respect to the regression model. In general, you might not have a response variable, but you want to find outliers (for example, multivariate normal data with outliers). In the non-regression context, one approach is to compute a robust center and a robust covariance for the data. You can then regard an outlier as a point that is more than a certain Mahalanobis distance from the center. See the section "Robust Multivariate Outlier Detection" on p. 7-9 of http://support.sas.com/resources/papers/proceedings10/329-2010.pdf ... View more

Well, you obviously know that you can get various output data sets from the procedure. You can merge that data with your Weibull data, and then use SGPLOT (or the GTL, if it is really complicated) to overlay the curves on the same plot by using multiple SERIES statements. IF, for some reason, there is not an output data set that contains all of the LIFETEST results that you need, you can always use ODS OUTPUT to generate the data that underlies the K-M plot. For example,    ods output SurvivalPlot=plot; ... View more

The error is in your constraints. When lamda=0, your log-likelihood function is undefined. Even when lamda gets very small (-1.38E-17) in your example), log(lamda) numerically overflows. You can bound the constraint for lada away from 0.  Also, rewrite the LogLik function to eliminate the loop over the elements of  and Censor_Ind.  It might look something like this (untested): start LogLik(param) global (Y,CENSOR_IND);    lamda = param[1];    gamma = param[2];    a = CENSOR_IND # ( log(lamda)*gamma+log(gamma)+(gamma-1)*log(Y) );    b = -(lamda*Y)##gamma ;    f = sum(a + b);    return ( f ); finish; con={1e-8 0, 1 1}; ... View more

By the way, you don't need the loop over all elements of Y. start LogLik(param) global (Y);    lamda = param[1];    gamma = param[2];    f = sum( exp((-lamda*Y)##gamma) );    return ( f ); finish; ... View more

Probably a good general approach. In IML you don't even need to use a macro. In IML you can just wrap the code in a module definition and call the module at each step of an iterative method. For example, if you're trying to find a zero, see http://blogs.sas.com/content/iml/2011/08/03/finding-the-root-of-a-univariate-function/ Or, if you're trying to optimize some criterion, see http://blogs.sas.com/content/iml/2011/10/12/maximum-likelihood-estimation-in-sasiml/ ... View more