@Dennisky wrote: That is, when the data follow a multivariate normal distribution, we conducted a repeated measure analysis on the data from healthy eyes and diseased eyes separately. There was no difference in variable X among healthy eyes, while variable X in diseased eyes showed a difference. Based on this, we concluded that our surgery is effective. I think that the method you wish to apply can solve the problem, namely reaching the conclusion on whether the surgery is effective. Another issue I would like to point out is about multiple comparison. In ANOVA for repeated measures, the researcher can perform pairwise comparison of the variable between any time points. I don't know the exact way of analyzing your data, so in your data analyzing process, you should be fully aware of the issue of multiple comparisons. If the data analyzing technique only yields results like "the differences of results of variable X at different time points are statistically significant", without pointing out which of them are different (especially on whether the level of variable X at the last time point is different from that at previous time point(s)), then this is not enough. Finally, I would like to refer to a book on analyzing data with repeated measures via SAS. You may take a look at it to find out the proper way of analyzing your data. ... View more

There are a few issues I would like to point out: (1) Normality test problem. 1) Difference between tests for univariate and multivariate normal distribution. You have stated that you tested normality for variable X at each time point. It should be noted that the test for multivariate normal distribution is somehow different from univariate normal distribution. More specifically, variable X at each time point following a normal distribution does not mean that variable X follows a multivariate normal distribution. A macro is now available for testing multivariate normality. But I am not that confident on your variable X following a multivariate normal distribution, as the variable X at some time points do not follow a univariate normal distribution. 2) Reliability of normality tests in large samples. According to central limit theorem, tests for univariate normality usually provide significant test results (i.e. the variable does not follow a normal distribution) given a large sample size. In my data analyzing experience, when the sample size exceeds 100, univariate normality tests tend to produce significant test results once the distribution of the sample is slightly skewed. In that case, normality results are not that reliable. You can also use the shape of histograms, P-P plots and Q-Q plots to aid your judgement. If your sample size is large and the normality test results and the results of histograms, P-P plots and Q-Q plots contradict each other, then believe in the results of histograms, P-P plots and Q-Q plots. (2) Issues on GEE. To the best of my knowledge, the independent variables of GEE should be categorical, not continuous. Since variable X is continuous, I think that GEE is not suitable for your problem. ... View more

So do you mean that the variable X does not follow a multivariate normal distribution? If that is the case, then sorry, I don't know the way of solving your problem. I previously thought that analysis of variance for repeated measures could solve your problem, but since the variable you are interested in does not follow a multivariate normal distribution, this method is not suitable. ... View more

Is the variable X categorical or continuous? ... View more

Thank you for your comments! As you have mentioned, the fact that principal component analysis takes only the X-matrix into account is a major inherent limitation of it. ... View more

Thank you for your comments! ... View more

@PaigeMiller wrote: I still don't see how this can be better from a logical point of view than Partial Least Squares, which goes ahead and finds vectors that are predictive, rather than starting with vectors that are not chosen because they are predictive, and then finding some that are predictive. I agree with your opinion on the disadvantages of that method. I am not an advocator of that method neither. I posted this method online and invited other people to make comments on that because I think that during the discussion and opinion-exchange process, one can deepen his or her understanding on principal component analysis, variable selection, and possibly other issues on statistics as well. That is good for all of the people participating in the discussion. Aside from the disadvantages I mentioned in my reply to @DanObermiller, a major limitation of the method is the arbitrariness of the definition of "drastic" decrease of eigenvalues. In statistics, not all situations have an objective criterion to distinguish "good" from "bad". The choose of the method for the selection of smoothing parameters and degrees of freedom of the spline effects in generalized additive models is one of the situations. Is there a number that can tell you which method is better, generalized cross-validation (GCV) and unbiased risk estimator (UBRE)? To the best of my knowledge, in many cases, the answer is "no". I learnt from experience of building generalized additive models that in many cases the two methods produce similar yet different results, both of which are useful and "valid" (no overfitting observed). If that is the case, both models are acceptable. However, in the case of variable selection, there are already statistical methods other than the method I mentioned in dealing with collinearity. These methods are based on objective criteria. So why should we abandon the objective one and embrace the subjective and defective one? @PaigeMiller wrote: But both the method mentioned by @Season and the modified method from @DanObermiller still assume that variable selection is an important step, and yet the paper from Tobias (and from hundreds or thousands of other authors) using PLS simply skips the variable selection step and they get useful models. Yes, I have been fully acknowledged that variable selection may not be a necessary process in regression. Yet, as I have mentioned, I am not an advocator of the variable selection method I mentioned. I am neither an advocator of "variable selection is a must in regression" theory now. ... View more

Thank you for your comments and the amended method you offer! @DanObermiller wrote: The biggest problem is that PCA is performed on the independent variables. There is nothing in PCA that says the lowest eigenvalues correspond to the worst predictors. So removing predictors based on the smallest eigenvalue does not mean you are keeping the best variables. I totally agree with your opinion. That is a major limitation of principal component analysis when it comes to regression. I have read articles discussing the arbitrariness of variable selection based on P-values, as they are computed from the data from a single sample. Therefore, resampling strategies like Bootstrap, Jackknife and cross-validation have been introduced to the variable selection process in some cases (e.g. building a prediction model). I think this method is worse than the variable selection based on P-values, as both the regression coefficients and their corresponding t-statistics from which P-values are computed take information both pertaining to the matrix of independent variables (X-matrix) and the vector of dependent variables into account when ordinary least squares is applied. However, only using the information of X-matrix, this method has the deficiency of, as you have mentioned, the lack of correlation between the largest load in the principal component with the smallest eigenvalue and insignificance in determining/predicting the dependent variable. I think with deductive reasoning that this method is only better than simply deleting professionally relatively insignificant or unimportant variables when dealing with collinearity, as the latter is based on nothing but the researcher's subjective perception on the significance and/or importance of the independent variables when it comes to determining/predicting the dependent variable. The method I mentioned takes data into account more or less, but that is not enough. The only advantage for the method I mentioned may be the resulting unbiased estimate of the regression coefficients. Linear regression based on principal components (also known as principal component regression) produces biased estimates of the regression coefficients. To the best of my knowledge, things are same (i.e. producing biased estimates of the regression coefficients) for ridge regression and LASSO. I do not know if that is also the case for partial least squares. But the sacrifice for an unbiased estimate of the regression coefficients may be huge- the researcher may have deleted statistically and professionally significant variables in the variable selection process without noticing the situation. The only circumstance I can think of in which this method is a good one is that the researcher only wants to know the relation between the dependent variable and the independent variables that remain after the variable selection method I mentioned. In other words, the researcher does not care about the effects of the variables deleted. If that is the researcher's goal, this method may be a good one. But I don't think that this situation is common. ... View more

Hello, everyone. I read a piece of information about PCA today, with a method of using PCA to perform variable selection during linear regression described in it: (1) Standardize all of the independent variables; (2) Calculate the eigenvalues of the correlation matrix of the original variables and the corresponding eigenvectors (since the original variables has been standardized); (3) Find the principal component with the smallest eigenvalue, namely the last one; (4) Find the variable whose absolute value of the coefficient is the largest in that (the last) principal component; (5) Delete that variable; (6) Repeat steps (2) to (5) until the last eigenvalue does not drastically decrease (the cut-off value for "drastic" was not mentioned); (7) Use the remaining variables to do linear regression. I wonder if you could give some comments on that method. ... View more

Thank you for your reply! @sbxkoenk wrote: @Season wrote: I wonder whether a composite (more than one) dependent variable, instead of solely one dependent variable is involved in the modeling equals "output block has only one response variable". Sorry for the mistake I made here. I did mean "output block has more than one response variable".  @sbxkoenk wrote: Multivariate Analysis (MVA) can mean two things: there are only input variables (correspondence analysis, PCA, Factor analysis, Multidimensional scaling (MDS) ...) if there is an output block, then Multivariate Analysis (MVA) means "solving problems where more than one dependent variable is analyzed" (all dependent variables are simultaneously explained / predicted in just one model). Thank you for your introduction to the difference between the two concepts! The statistics I learnt in school deal with at most one dependent variable. So I did not know these knowledge before. ... View more

Thank you very much, Koen, for your more detailed explanation! @sbxkoenk wrote: I would like to raise a brief question for the sake of selecting a possible "shortcut": do you think that in the situation I encounter, only "predictive Partial Least Squares regression", but not other kinds of PLS, is suitable for reaching the goal I previously mentioned (tackle collinearity and conducting variable selection at the same time in a multivariate linear regression)? If so, maybe I do not need to know every kind of PLS to reach my goal. If I am right your output block has only one response variable, so you are doing multiple regression analysis and NOT multivariate regression! I am not sure what the noun "output block" means, but the model I am attempting to build does contain merely one dependent variable. I know that Bayesian neural network and structural equation model can be used to deal with situations in which more than one dependent variable is modeled. These situations are far more complicated than the one I am encountering. I am building a regression model with only one dependent variable. I wonder whether a composite (more than one) dependent variable, instead of solely one dependent variable is involved in the modeling equals "output block has more than one response variable". But anyway, I appreciate your pointing out the subtle yet maybe significant difference of my nomenclature of the analysis I am attempting to perform. ... View more

OK, thank you for your previous work and kindly offering them to me! ... View more

Thank you very, very much for your kind help, including a more detailed description of PLS and its history! I really appreciate your blending humanities and statistics, with introductions on the history of statistical methods, including those of PLS and joint model for longitudinal and time-to-event data (which you had pointed out previously) attached. For me, scientific history is not just a record of what happened in the past, but also a record of the trajectories of the development of sciences, from which I can summarize the pattern of existing scientific knowledge and disciplines, raise questions or even propose theories. I firmly believe that questions are key elements of the improvement of science, as is the case of the development of elliptic integrals while astronomers tried to calculate the circumference of orbits. In a word (two words, to be exact😂), thank you! Still, I am still a green hand on PLS, so I hardly know anything about this method apart from its name. Your interpretation of PLS using "paths and blocks" somehow illuminates the method, but I am still not that clear about it. I am going to read more about PLS. I would like to raise a brief question for the sake of selecting a possible "shortcut": do you think that in the situation I encounter, only "predictive Partial Least Squares regression", but not other kinds of PLS, is suitable for reaching the goal I previously mentioned (tackle collinearity and conducting variable selection at the same time in a multivariate linear regression)? If so, maybe I do not need to know every kind of PLS to reach my goal. ... View more

😀Thank you for your explanation!👍 ... View more