Multiobjective regression modifications for collinearity

In this work we develop a new multivariate technique to produce regressions with interpretable coefficients that are close to and of the same signs as the pairwise regression coefficients. Using a multiobjective approach to incorporate multiple and pairwise regressions into one objective we reduce this technique to an eigenproblem that represents a hybrid between regression and principal component analyses. We show that our approach corresponds to a specific scheme of ridge regression with a total matrix added to the matrix of correlations. One of the main goals of multiple regression modeling is to assess the importance of predictor variables in determining the prediction. However, in practical applications inference about the coefficients of regression can be difficult because real data is correlated and multicollinearity causes instability in the coefficients. In this paper we present a new technique to create a regression model that maintains the interpretability of the coefficients. We show with real data that it is possible to generate a model with coefficients that are similar to easily interpretable pairwise relations of predictors with the dependent variable, and this model is similar to the regular multiple regression model in predictive ability.