Yes, indeed!  Thanks for the correction. ... View more

In order to have different random efffect covariance values across levels of some group with the NLMIXED procedure, you need to include group-specific random effects in your linear predictor.  When constucting the covariance structure for the random effects, we assume that the group-specific random effects are independent of each other. For the example which you show above, suppose that period has just two values (1 and 2).  Then NLMIXED code which would fit the model specified by your MIXED code would be: proc nlmixed data=D;   /* construct linear predictor, including random effects */   eta = b0 + b1*time +         (u0_1 + u1_1*time)*(period=1) +  /* period 1 random effects */         (u0_2 + u1_2*time)*(period=2);    /* period 2 random effects */   /* Parameterize the covariance between u0_i and u1_i as */   /* a function of the correlation between u0_i and u1_i. */   /* Parameterize the correlation between u0_i and u1_i   */   /* through the inverse Fisher transformation.           */   rho_1 = (exp(2*z1) -1) / (exp(2*z1) + 1);   /* rho(u0_1, u1_1) */   rho_2 = (exp(2*z2) -1) / (exp(2*z2) + 1);   /* rho(u0_2, u1_2) */   /* Specify and fit the model */   model y ~ normal(eta, Vres);   random u0_1 u1_1 u0_2 u1_2 ~ normal([0,0,0,0],                                       [Vu0_1, rho_1*sqrt(Vu0_1*Vu1_1), 0, 0,                                               Vu1_1,                   0, 0,                                                      Vu0_2, rho_2*sqrt(Vu0_2*Vu1_2),                                                             Vu1_2])                                subject=SubjectID; run; Note that this can quickly become a very difficult estimation problem for NLMIXED because of the need to integrate over all random effects. ... View more

No!  Quite the opposite!  I am saying that with the volume of data that you have, there is every reason to hold out a validation sample of, say, 1 million records.  You would still have a very large sample (4 million records) for estimation.  Confidence intervals of predicted probabilities will not be much larger for a model constructed from 4 million observations compared to a model constructed from 5 million observations.  Thus, you don't lose much in the way of model estimation and you gain much by having a validation sample where you can test your model. ... View more

Ruth, Let me see if I can turn your thinking around here.  You have a very large data set.  Because it is large, you should have tight confidence limits on predicted probabilities obtained from a logistic regression model fitted to these data.  Now, suppose that you had just 4 million records instead of 5 million records.  Would you still say that you would expect tight confidence limits on predicted probabilities?  Probably so.  (If not, what are the limits on a large sample size?) Now, if 4 million records are going to produce tight confidence limits on predicted probabilities, then what would it hurt you to hold out a million records from the estimation set and use those 1 million observations for subsequent evaluation.  It would seem that you have every opportunity in the world to obtain and subsequently test a model.  What benefit would there be to you in using 5 million records for model estimation without model validation vs using 4 million records for model estimation followed by model evaluation? ... View more

Yes, a subject specification is required.  If there are no subject blocks and all data should be regarded as a single subject, then specify subject=intercept. ... View more

From the SAS online documentation, here is the definition of Cp:       Cp = [(SSEp)/(s2)] - (N - 2p)   where s2[=s**2] is the MSE for the full model, and SSEp is the   sum-of-squares error for a model with p parameters Since s2 is the MSE for the full model (which includes all candidate variables), then changing the set of candidate variables will change the value of s2. Hence, you can expect to get a different value for Mallow's Cp if you change the set of candidate variables. Now, if your restricted set of candidate variables includes all of the important variables, then the expectation for s2 in the restricted and complete variable sets should be the same. So, you might not see much difference in Mallows' Cp if the restricted variable list contains all of the important predictors. But if the restricted set results in the loss of important predictors, then E(s2) for the restricted variable set will be larger than E(s2) for the full variable set. In that case, Mallows' Cp should go down in the restricted variable set. ... View more

Ryan, Sorry, I didn't pay attention to the note about specification of the memsize option only in the SAS configuration file or at startup. If you locate your sas configuration file (it should have name sasv9.cfg), then you can edit it to have a line: -memsize 0 With regard to the error message "ERROR: The mean parameter is either invalid or at a limit of its range for some observations", take a look at: http://www.google.com/url?sa=t&source=web&cd=1&ved=0CBkQFjAA&url=http%3A%2F%2Fwww2.sas.com%2Fproceedings%2Fsugi26%2Fp247-26.pdf&rct=j&q=mean%20parameter%20invalid%20limit%20range%20some%20observations&ei=2NUATrvtDZGgsQP6-pW5DQ&usg=AFQjCNHdbKj8_JbBmkkry8OIKrNaQwNqtw&cad=rja The author indicates that this message indicates the need for exact Poisson regression when you encounter this error message. You must have SAS version 9.22 in order to conduct exact Poisson regression (or use other software as indicated in the above link). See the SAS documentation for version 9.22 for appropriate syntax for exact Poisson regression. (I would note that version 9.22 also allows exact logistic regression. I might think that exact logistic regression would be appropriate for your problem. But don't take that to be a recommendation. I would have to spend more time thinking about the issues and perhaps spend time with the data itself in order to make a specific recommendation.) It is possible that exact Poisson and exact logistic regression were implemented in version 9.2 as undocumented features. Finally, you might still run into memory issues even when the memsize option is specified as indicated above given the volume of data that you have. (In fact, since the memsize option default is -memsize 0 and if you have not previously modified your SAS configuration file to change the memsize specification, then editing your configuration file will probably not change the behavior of SAS at all.) Certainly, to perform exact logistic or exact Poisson regression, you will encounter additional performance issues on top of the problems you already have. Since you have only a single observation per subject, you can greatly reduce the data processing requirements by constructing a summary data set where each combination of the response and predictors is observed only once and you record a variable that indicates the number of times that combination occurs in your data set. You can get this as follows: proc sort data=dataset;   by smoke refGroup MultGest HighParity PreviousPreterm STD WtGnLT15 Inadequate MEDICAID; run; data summary;   set dataset;   by smoke refGroup MultGest HighParity PreviousPreterm STD WtGnLT15 Inadequate MEDICAID;   if first.MEDICAID then count=0;   count+1;   if last.MEDICAID then output; run; You can then name the data set summary to the GENMOD procedure and specify a FREQ statement naming the variable COUNT. Since you apparently have only binary predictors and a binary response, then the number of unique combinations of these 9 variables would be at most 2^9=512. Using the summary data set should produce great improvements in data processing. ... View more

Have you tried specifying the SAS option: options memsize -0; That will allow SAS to use the maximum available memory for subsequent analyses. You don't indicate how many observations you have (number of subjects and number of observations/subject). You also don't indicate whether the predictor variables vary over time within subject. You also don't tell us now much RAM you have or what kind of hardware you have to run these analyses on. There can be differences across operating systems in the efficiency that each is able to process large amounts of data. This is all critical information for understanding the scope of the problem. ... View more

Ryan, You do, indeed, have your independent and dependent variables turned around in your genmod code. Your regression model would suggest that the risk factor can affect whether a respondent is black or not. This is a strange notion indeed! You can get the RR of 1.3 with the following code: proc genmod data=test desc;   class RefPop (ref='1') /param=glm;   model RiskFactor = RefPop / link=log dist=bin;   estimate 'Beta RefPop' RefPop 1 -1/ exp; run; ... View more

Syntax with random personID; or random intercept / sub=personID; should produce the same log-likelihood and identical random effect variance estimates. I would note that while the results should be identical, the two models are parameterized differently. Because the models are parameterized differently, round-off errors could conceivably produce different results for the alternate syntaxes. If you were to observe any difference in results, such differences should be quite small. The second form in which you use a subject specification is generally more efficient in use of computer resources. I typically recommend use of the "random intercept / sub=personID;" form. ... View more

Kui, You might want to show how you have parameterized the zero-inflated beta-binomial distribution. Have you parameterized it in terms of two shape parameters or in terms of the mean and variance? I presume that you are fitting a beta-binomial regression model, in which case I would think that you would want to parameterize the beta-binomial density function in terms of mean and variance and then compute the shape parameters as functions of the mean, variance, and number of trials on which the response (number of successes) is based. But without seeing how you have parameterized the zero-inflated beta-binomial distribution, let's consider how many different types of overdispersion you are attempting to model here. The beta-binomial distribution represents an overdipsersed binomial distribution. Allowing for zero-inflation introduces a second type of overdispersion function. Adding random intercept and slope is yet another form of overdispersion. So, you are trying to fit a model that has three sources of overdispersion. It comes as no surprise that you are having trouble introducing random effects on top of an zero-inflated beta-binomial. Conceivably, it is possible introduce random subject effects to a zero-inflated beta-binomial distribution. But in practice, you may find it difficult to estimate such a model. ... View more

I'm not sure what the question is. The model which specifies random personID; fits a model with a random intercept term, whereas the model which specifies random intercept age diet / sub=personID; fits a model in which each person has their own age slope and their own diet slope. Are you wondering if the model with person-specific age and diet slope effects produces a model which has better fit to the data, then you could use a likelihood ratio test. Note that this second model would only seem to be valid if each person was observed on multiple diets. If each person receives only a single diet, then there would be no data that would allow you to compute a person-specific diet slope effect. Also, if each person receives only a single diet, then the subject effect should be nested within diet. ... View more