Help on %choiceff for discrete choice design

[calmac]

Hi all,
I'm designing a binary choice experiment in order to indirectly determine participants' willingness to pay for specific features of an online service. The proposed choice is between two general options A vs B.
I have 14 features of which 13 features are binary (i.e., 1=feature shown, 0=feature not shown) and 1 feature has 8 levels (i.e., price, 0-1-2-3-4-5-6-7 dollars). So, my design needs to be asymmetric and fractional factorial, possibly blocked.
As a first step I run

%mktruns (2 ** 13 8)

which tells me that a design size of 64 is a 100% efficient design.
So, as a second step, I run the MktEx macro:

%mktex(2**13 8, n=64, seed=205)

which gives me an efficient solution (i.e., 100).
After this I label variables and create the two options

proc format;
value Status_update 1 = 1 2 = 0;
value Notes 1 = 1 2 = 0;
value Comments 1 = 1 2 = 0;
value Wall_posts 1 = 1 2 = 0;
value Private_messages 1 = 1 2 = 0;
value Chats 1 = 1 2 = 0;
value Groups 1 = 1 2 = 0;
value Newsfeed 1 = 1 2 = 0;
value Like 1 = 1 2 = 0;
value Photos_videos 1 = 1 2 = 0;
value Events 1 = 1 2 = 0;
value Gaming 1 = 1 2 = 0;
value Fan_pages 1 = 1 2 = 0;
value Price 1 = $0 2 = $1 3 = $2 4 = $3 5 = $4 6 = $5 7 = $6 8 = $7;

run;

%mktlab(data=design, /* input data set */
vars=Status_update Notes Comments Wall_posts Private_messages Chats Groups Newsfeed Like Photos_videos Events Gaming Fan_pages Price, /* new attribute names */
int=f1-f2, /* create 2 columns of 1’s in f1-f2 */
out=final, /* output design add a format statement for the attributes */
stmts=format Status_update Status_update. Notes Notes. Comments Comments. Wall_posts Wall_posts. Private_messages Private_messages. Chats Chats. Groups Groups. Newsfeed Newsfeed. Like Like. Photos_videos Photos_videos. Events Events. Gaming Gaming. Fan_pages Fan_pages. Price Price.)

proc print; run;

So far everything seems fine to me and consistent with what I'm trying to achieve.
Now I run the choiceff macro, which is the step that is worring me:

%choiceff(data=final, /* candidate set of alternatives */
bestout=sasuser.facebookdes, /* choice design permanently stored */
/* model with stdz orthog coding */
model=class(Status_update Notes Comments Wall_posts Private_messages Chats Groups Newsfeed Like Photos_videos Events Gaming Fan_pages Price / sta) /
cprefix=0 /* lpr=0 labels from just levels */
lprefix=0, /* cpr=0 names from just levels */
nsets=64, /* number of choice sets to make */
seed=205, /* random number seed */
flags=f1-f2, /* flag which alt can go where, 2 alts*/
options=relative, /* display relative D-efficiency */
beta=zero) /* assumed beta vector */

The D-efficiency is poor, around 40, and the output table does not contain the variables as previously defined.

If I double the design size, from 64 to 128, the D-efficiency of %choiceff increases to 94 but still the output table contains variables that seem weird to me so I don't have much confidence in the resulting design.

I guess I'm missing something (big) here, but I couldn't figure this out by myself. If someone can help me, this would very much appreciated. Thank you

[WarrenKuhfeld · Accepted solution]

You are getting strange coded variable names because you asked for them.  Drop the CPREFIX= and LPREFIX= options and the names will be based on input variable names and levels.

choiceff(data=final, /* candidate set of alternatives */
bestout=sasuser.facebookdes, /* choice design permanently stored */
/* model with stdz orthog coding */
model=class(Status_update Notes Comments Wall_posts Private_messages Chats Groups Newsfeed Like Photos_videos Events Gaming Fan_pages Price / sta),
nsets=64, /* number of choice sets to make */
seed=205, /* random number seed */
flags=f1-f2, /* flag which alt can go where, 2 alts*/
options=relative, /* display relative D-efficiency */
beta=zero) /* assumed beta vector */
 

Now about the efficiency.  Do not assume that the efficiency value is on a 0 to 100 scale.  It is when you run MktEx in the context of a linear model. Or at least, it might be. The scale is 0 (linear dependencies) to 100% efficient relative to a (perhaps hypothetical) orthogonal design.  For many design situations, we do not know the optimal efficiency.  That is true for linear models for all but the smallest or most combinatorially special designs.  It is even more true for choice models.  I think your 40 or so efficiency is pretty good.  I think that because I wrote ChoicEff and I think it usually works pretty well. I do not think it based on any particular number.  2 might be pretty good. It might be that the maximum is 41 (or 2.01).  We usually have no way of knowing.

Now for the more meaningful comparison, your variances are quite small and look consistent and stable. Also, you are estimating 20 parameters with a design capable of estimating 64.  If you were estimating a bunch more parameters, you might start seeing pretty large variances.  You can estimate everything that you specified. Nothing you are doing seems wrong or even slightly weird (except for specifying those prefix options).  The only thing you need to realize is that you can only have two of eight prices in each choice set, whereas it is possible to have both levels for the other attributes.  That is not a problem, but it is a feature of what you are doing.  I hope this helps.

[calmac]

Thank you very much Warren, I couldn't ask for a more qualified response.