topic Re: Grouping similar competitors in Statistical Procedures

Grouping similar competitors

kurofufu — Tue, 28 Aug 2012 18:38:22 GMT

Suppose we want to look at which airlines have similar pricing strategies. The data set looks like this:

Variables: Flight Origination, Flight Destination, Airline1 Price, Airline2 Price, ....Airline10 Price.

Data:

Origination: A, Destination: B, Airline1 Price=100, Airline2 = 120, ...., Airline10=95

Origination: A, Destination: C, Airline1 Price=500, Airline2 = 450, ...., Airline10=505

......

I wonder if I can use PROC VARCLUS to accomplish this objective. But as I read, it seems PROC VARCLUS is not used for this purpose.

Re: Grouping similar competitors

ballardw — Tue, 28 Aug 2012 19:15:13 GMT

You will probably find that restructuring your data to: Origination, Destination, Airline and Price lends itself to better analysis in general.

Re: Grouping similar competitors

kurofufu — Wed, 29 Aug 2012 17:41:11 GMT

More info:

Expected outcome goes like this:

Airline1, Airline5, Airline10 have the same pricing strategies

Airline2, Airline3, Airline6 have the same pricing strategies

....

What I mean by the same pricing strategies is when Airline1 prices high on a origination and destination pair, Airline5 and Airline10 also prices high.

Re: Grouping similar competitors

Doc_Duke — Wed, 29 Aug 2012 18:19:43 GMT

If you restructure your data the way that ballardw says, then you can apply the various clustering procedures to get to the type of answer that you are looking for. This looks like a classic CART (Classification and regression tree) problem. SAS Enterprise Miner has a CART procedure.

You will probably need some additional information for a complete analysis (distance, number of stops, time of day, lead time to purchase, etc.).

Doc Muhlbaier

Duke

Re: Grouping similar competitors

kurofufu — Wed, 29 Aug 2012 19:01:58 GMT

The example is just intended for illustrating my question. I don't plan to go into a full analysis of airline industry. I don't see restructuring the data in ballarsw's way can solve my problem.

Another point is I want to group variables, not observations.

Can I create a correlation matrix of Airline1 to Airline10 and figure out the groupings?

Re: Grouping similar competitors

PGStats — Thu, 30 Aug 2012 01:03:03 GMT

Hi,

In principle, VARCLUS seems fine for your task, as long as every airline offers a flight between every O-D pair. O-D pairs with missing values will however be deleted from the analysis.

An alternative would be to transpose your dataset and use proc DISTANCE to get a distance matrix with perhaps DCORR as a distance measure. Then you could use that distance matrix as input into proc CLUSTER or proc MODECLUS to get airline clusters. You could also simply feed your distance matrix into proc MDS and get a 2-D representation of the distances.

Re: Grouping similar competitors

kurofufu — Thu, 30 Aug 2012 03:47:17 GMT

Thanks for the suggestion, PG.

I also would like to find out who price in the different direction than each other.

So two patterns are interested:

When A prices high/low, B prices high/low most of time => group A & B together

When A prices high/low, B prices low/high most of time. => group A & B together

I think only correlation matrix fits the bill.

Re: Grouping similar competitors

Ksharp — Thu, 30 Aug 2012 05:02:08 GMT

I don't think so.

Correlation matrix is suited for those variables which are independent for each everyone.

There are some Partial correlation for one single variable. That means someone variable will be associated with more than one variables. That is the reason why we should use proc varclus to consider such a question, it has considered the Partial Correlation .

Ksharp

Re: Grouping similar competitors

kurofufu — Thu, 30 Aug 2012 05:18:28 GMT

Ksharp,

Pattern1: When A prices high/low, B prices high/low most of time => group A & B together

Pattern2: When A prices high/low, B prices low/high most of time. => group A & B together

Can proc varclus group variables with Pattern2 together? Or it only can group variables with Pattern1?

Re: Grouping similar competitors

Ksharp — Thu, 30 Aug 2012 06:29:15 GMT

These two Patterns you are talking about is linear correlation (i.e. Pearson correlated coefficient ) which conform to Normal distribution (i.e. independent for each other). you can use correlation matrix, once you make sure that they are non-correlated (i.e. each airline make its decision is independent ) .

proc varclus is implementing multi-variables analysis, which is nothing with linear correlation. he use principal component analysis.

The first principal component is a weighted average of the variables that explains as much variance

as possible.

For example: if A and B belong to a group that means A and B can explain most variance of either the correlation or the covariance matrix .

KSharp