DATA Step, Macro, Functions and more

5000 Models and Their AICs?

Accepted Solution Solved
Reply
New Contributor
Posts: 2
Accepted Solution

5000 Models and Their AICs?

I have a dataset which in a simplified view looks like this;

Index     Time     Var1     Var2     Var3 . . . Var7

1               1       a          b          c

1               2       d          e          f

1               3       g          h          i

1               4       j           k          l     

2               1       m         n          o

2               2       p          q          r

2               3       s           t         u

2               4       v          w          x    

.               .          .          .          .    

.

.

.

5000

It is hoped that it can be used to develop 5000 different non-liner regression models. I have generated a list of 7 random knot locations between 0 to 60 for 5000 models, these have then been used to calculate a truncated power series where the values are stored under Var1 to Var7.

What I need to know is how to do a proc reg on the values within the rows of data where index =   (i = 1 to 5000) and then obtain the AIC for each model. The LONG way would be to subset 5000 times and do each model individually and then compare AICs. Is there a way to automate this process?

Many Thanks

Graeme Torrance

University of St. Andrews


Accepted Solutions
Solution
‎12-08-2011 11:52 PM
Super User
Posts: 9,682

Re: 5000 Models and Their AICs?

You can use " by index ;" to get these 5000 model respectively.

But for the sake of your large number, i recommend to use macro to loop them,

And use "ods ouput" or "ods select" to select AIC statistical estimator, then compare them in a dataset as you wish.

Ksharp

View solution in original post


All Replies
PROC Star
Posts: 7,363

5000 Models and Their AICs?

Graeme, I'm not sure I understand because you indicated non-linear, but are using proc reg.

Regardless, if you already have the code to accomplish the desired calculations, one way would be to expand the dataset to account for the 5000 different scenarios, and then simply include a by variable.

New Contributor
Posts: 2

5000 Models and Their AICs?

Hi Art,

Two Things;

1) Sorry the data is linear.... thats why I am using proc reg.

2) Unfortunately I do not have the code on SAS to do the truncation series. Ashamedly I used MS Excel to do the truncated series (because I am a SAS newbie).

I think I understand your suggestion but not sure how that would allow me to build number of models.

I'm sorry if this is not much more information.

PROC Star
Posts: 7,363

5000 Models and Their AICs?

No reason to be ashamed about using Excel.  More people do, for various things, than anyone will admit.

You can always import that into Excel and use it to drive the creation of the needed large datafile that will include all of the replicates .. as long as they are identified by a series number.

Then, that series number, can be used as the by variable.

Without knowing exactly what you have, the above are my best guesstimates (and, yes, I used Excel to make the guesstimate Smiley Wink.

Trusted Advisor
Posts: 1,300

Re: 5000 Models and Their AICs?

You might want to look into proc robustreg, using your original data.  It can imprelent the truncated power function and knots without preprocessing in Excel.  I cannot say your example or problem explanation give me enough information to really be helpful...

Solution
‎12-08-2011 11:52 PM
Super User
Posts: 9,682

Re: 5000 Models and Their AICs?

You can use " by index ;" to get these 5000 model respectively.

But for the sake of your large number, i recommend to use macro to loop them,

And use "ods ouput" or "ods select" to select AIC statistical estimator, then compare them in a dataset as you wish.

Ksharp

☑ This topic is SOLVED.

Need further help from the community? Please ask a new question.

Discussion stats
  • 5 replies
  • 188 views
  • 3 likes
  • 4 in conversation