Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Options

- RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page

🔒 This topic is **solved** and **locked**.
Need further help from the community? Please
sign in and ask a **new** question.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Posted 10-20-2017 02:33 PM
(1416 views)

Hi all.

I'm using Warren Kuhfeld's marvellous experimental design macro, %choiceff() to find large choice designs from even larger candidate data sets, and when I ramp up to 16 alternatives and their 2-way cross availability effects + generic price I have 251 parameters in the design.

```
%let nupc=16;
%macro mk2clst(mynalts=);
%do i = 1 %to &mynalts.;
%do j = 1 %to &mynalts.;
%if
%eval(&i. NE &j.)
%then
%do;
c%trim(%eval(%eval(%eval(&i.-1)*&mynalts.)+&j.))
%end;
%end;
%end;
%mend;
%choiceff
(
data= perm.yogurt,
model= class(a1 - a%trim(&nupc.) %mk2clst(mynalts=&nupc.) / zero=first sta) identity(price),
nsets= %eval(144*32),
nalts= %trim(&nupc.),
seed= 09051958,
options= nodups,
rscale= alt,
maxiter= 1,
beta= zero
)
;
```

The macro has to evaluate the determinant as that is part of the all-important design D-efficiency statistic. This interesting thing about my error is that the problem size of 251 parameters completes normally when I run it inside the latest SAS University Edition VM (SAS 9.4 m4 64 bit Centos) , but it terminates with the error in the title when I run exactly the same analysis on my win x64 desktop SAS 9.4 M1. Is this just another 'linux is better than windows' example? I note that in other community posts Rick_SAS has explained that this error arises when the determinant of a large matrix exceeds the double precision floating point representation limit. I'm planning to scale up my analysis to 175 alternatives (!) so I guess I'll have to re-write the determinant calculation using Rick_SAS's suggestion of calculating log(det), then scale the log det term before exponentiating. Attempts to find designs with 17 alternatives and their 2-way cross availability effects fail in both environments with less informative errors but I assume the underlying problem is the same - matrix determinant too big for double floating point representation. Has anyone else struck this limit with Warren's %choiceff() macro? Any tips before I dive down that rabbit hole?

1 ACCEPTED SOLUTION

Accepted Solutions

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Hi Damian,

Thanks for the kind words! I requested a log determinant function in IML quite a few years back. It was delivered in 9.4M1. I have been using it ever since in several of my macros including ChoicEff. Are you sure that you are using a 9.4M1 version of the macro?

In case you are interested, here is how I get efficiency.

```
%macro _logdet(logdet, eff, xpx, npi);
holdlogdet = logabsdet(&xpx);
if holdlogdet[2] <= 0 then holdlogdet[1] = -1e50;
%if &logdet ne %then %do; &logdet = holdlogdet[1]; %end;
%if &eff ne %then %do; &eff = exp(holdlogdet[1] # &npi); %end;
%mend;
```

Best,

Warren

5 REPLIES 5

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

The LOGABSDET function was introduced in SAS 9.4M1. That was a long time ago, but if I recall correctly, I mentioned the LOGABSDET function to @WarrenKuhfeld sometime later and he updated his macros to use it. If so, it might not be a Window/Linux issue, but that the 9.4M1 macro uses different computations than the 9.4M4 macro.

Warren might chime in if he has something to say.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Hi Damian,

Thanks for the kind words! I requested a log determinant function in IML quite a few years back. It was delivered in 9.4M1. I have been using it ever since in several of my macros including ChoicEff. Are you sure that you are using a 9.4M1 version of the macro?

In case you are interested, here is how I get efficiency.

```
%macro _logdet(logdet, eff, xpx, npi);
holdlogdet = logabsdet(&xpx);
if holdlogdet[2] <= 0 then holdlogdet[1] = -1e50;
%if &logdet ne %then %do; &logdet = holdlogdet[1]; %end;
%if &eff ne %then %do; &eff = exp(holdlogdet[1] # &npi); %end;
%mend;
```

Best,

Warren

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Thanks for the explanations, Warren and Rick. You guys are the best. I checked and indeed, an older version had been re-installed on my desktop in error during an automated group software update programmed by my IT support staff. My bad for not checking the startup log! Back to the future (tm)? Forward to the past? Anyway,, I'm sure when they eventually install the latest version, you fantastic macros will all run just perfectly.

Warren, with the log det improvements since 9.4 M1, have you or other %choiceff() users encountered an upper parameter limit for designs? Can we now all reach out into the (big data) universe for our design-balanced choice model data?

Warren, with the log det improvements since 9.4 M1, have you or other %choiceff() users encountered an upper parameter limit for designs? Can we now all reach out into the (big data) universe for our design-balanced choice model data?

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Damien,

I emailed you the 9.4M1 version. I quit putting new versions on the web since we update SAS so often now days. Email me if you need a different version. There are no built in limitations on the size of the problem, but this is a highly iterative process, and run time does not scale linearly with problem size. I have found that usually you get most of the efficiency gain in the first iteration. Don't ask for a lot of iterations with a big problem.

Best,

Wrren

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

I guess in the meantime, while I wait for my IT support staff, I could just download the 9.4 M4 mr macros to overwrite the older versions, right?

Here is the code I use to copy the latest version of choiceff.sas from inside my current SAS University Edition:

```
filename macrout '/folders/myfolders/mymacros/';
data _null_;
infile sasautos(choiceff.sas);
file macrout(choiceff.sas);
input ;
put _infile_;
run;
```

and from there, a shared folder with my windows host operating system, I can copy it to overwrite my existing SAS 9.4 M0 for win for x64 supplied macro.

**Don't miss out on SAS Innovate - Register now for the FREE Livestream!**

Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.

Multiple Linear Regression in SAS

Learn how to run multiple linear regression models with and without interactions, presented by SAS user Alex Chaplin.

Find more tutorials on the SAS Users YouTube channel.