5000 Models and Their AICs?

[vilike85]

I have a dataset which in a simplified view looks like this;

Index     Time     Var1     Var2     Var3 . . . Var7

1               1       a          b          c

1               2       d          e          f

1               3       g          h          i

1               4       j           k          l     

2               1       m         n          o

2               2       p          q          r

2               3       s           t         u

2               4       v          w          x    

.               .          .          .          .    

.

.

.

5000

It is hoped that it can be used to develop 5000 different non-liner regression models. I have generated a list of 7 random knot locations between 0 to 60 for 5000 models, these have then been used to calculate a truncated power series where the values are stored under Var1 to Var7.

What I need to know is how to do a proc reg on the values within the rows of data where index =   (i = 1 to 5000) and then obtain the AIC for each model. The LONG way would be to subset 5000 times and do each model individually and then compare AICs. Is there a way to automate this process?

Many Thanks

Graeme Torrance

University of St. Andrews

[art297]

Graeme, I'm not sure I understand because you indicated non-linear, but are using proc reg.

Regardless, if you already have the code to accomplish the desired calculations, one way would be to expand the dataset to account for the 5000 different scenarios, and then simply include a by variable.

[vilike85]

Hi Art,

Two Things;

1) Sorry the data is linear.... thats why I am using proc reg.

2) Unfortunately I do not have the code on SAS to do the truncation series. Ashamedly I used MS Excel to do the truncated series (because I am a SAS newbie).

I think I understand your suggestion but not sure how that would allow me to build number of models.

I'm sorry if this is not much more information.

[art297]

No reason to be ashamed about using Excel.  More people do, for various things, than anyone will admit.

You can always import that into Excel and use it to drive the creation of the needed large datafile that will include all of the replicates .. as long as they are identified by a series number.

Then, that series number, can be used as the by variable.

Without knowing exactly what you have, the above are my best guesstimates (and, yes, I used Excel to make the guesstimate ;).

[FriedEgg]

You might want to look into proc robustreg, using your original data.  It can imprelent the truncated power function and knots without preprocessing in Excel.  I cannot say your example or problem explanation give me enough information to really be helpful...

[Ksharp · Accepted solution]

You can use " by index ;" to get these 5000 model respectively.

But for the sake of your large number, i recommend to use macro to loop them,

And use "ods ouput" or "ods select" to select AIC statistical estimator, then compare them in a dataset as you wish.

Ksharp