I have 68 2km x 2km plots for which the proportions of 5 land cover classes (proportions add to 1.0) have been determined by 2 methods. How can I determine that the two methods generate a significantly different result?
I can use a pair-wise t-test to determine if the proportions generated by the two methods in the same class are significantly different. What I am looking for is an overall assessment of whether or not the two methods generate a significantly different result across all 5 land cover classes.
The table to be analyzed here is a 5x5 table (method1 x method2) and each of the 68 plots falls into one of the 25 cells. As stated, it sounds like you want a test of marginal homogeneity. That is, whether the 5 row proportions (marginal proportions for one method) are the same as the 5 column proportions (marginals for the other method.
You can test for marginal homogeneity as shown in the example titled "Repeated Measures, 4 Response Levels, 1 Population" in the PROC CATMOD documentation. You will need to create a data set with one observation per plot and with METHOD1 and METHOD2 variables containing the plots' classes under each method. Then METHOD1 and METHOD2 are used like the RIGHT and LEFT variables in the example. See this usage note for additional interpretation:
The approach you suggest works for the case where the 2 classifications are intersected spatially in a geographic information system and the area common to each of the 5 land cover classes in the two methods is evaluated in a contingency table to calculate a user's and producer's accuracy and an overall accuracy.
In my case, I would like to test the similarity of the two sets of proportions as if they are describing two independent populations. In this case, there are two sets of proportions that add up to 1.0 in each plot. I want to know if the two sets of proportions are significantly different from one another.
You can use the MIXED procedure to test multiple correlated land cover class differences. First, set up your data as within-plot differences for the first four cover classes. (The fifth cover class is an exact linear combination of the first four, so it can be excluded from use. If you had just two cover classes, you would only use the within-plot difference of the first land cover class as your response variable.) You will need each within-plot difference as a separate result with a label indicating which land cover class the difference belongs to. That is, you want your data to be constructed as shown below: