Proc Adaptivereg, Partition Statement, Out-of-Sample Testing

[davis]

Good morning,

I am interested in exploring out-of-sample testing using SAS EG proc adaptivereg (SAS EG, 7.15 (7.100.5.5850, 64-bit) and the partition statement.  However, I am running into a little difficulty.

My input data set is a "skinny" and unbalanced panel (i.e., time series, cross section), and the partition statement seems to operate on an observation, or row, as opposed to a group.  Applying the partition statement to the panel will ignore, it seems, the time dimension, which needs to be preserved.

If my understanding is clear, any thoughts on how to get the partition statement to operate on a group (e.g., unique member id) as opposed to an observation or on any other ways to potentially navigate this obstacle?

Thanks.

[StatDave]

I don't think ADAPTIVEREG can take into account the clustered nature of your data as you describe it. For such data, you might want to consider using procedures like GEE or GLIMMIX which can accommodate clustered data. In GLIMMIX, you can use the EFFECT statement to incorporate splines into the model if you need flexibility similar to (but not the same as) ADAPTIVEREG.

[MichaelL_SAS · Accepted solution]

I assume you are using the FRACTION option in the PARTITION statement. If so, you might try using ROLEVAR= option instead. This option allows you to specify a variable in input data whose values are used to assign observations to the training, test, or validation roles. Using this option would add the step of you first creating the role variable to satisfy the conditions you deem appropriate (ensuring observations from the same group are assigned to the same role, ect), but once created I think would enable you to do what you want. 

Quick edit based on seeing Dave's response. I should clarify that my comment deals only with how to get a bit more control over how the partition is created, not if the analysis is properly accounting for repeated measures.