Multivariate linear regression is a statistical technique used in machine learning to predict multiple dependent variables using multiple independent variables. It extends the concept of simple linear regression by incorporating multiple predictors.
Unlike Linear regression, Multivariate Linear Regression have multiple output variables instead of single variable Y. It may be written as,
Y = XB + E,
where Y is a matrix of dependent variables (outcomes), X is a matrix of independent variables that might be a design matrix (each column being a set of observations on one of the independent variables), B is a matrix containing parameters or coefficients that are usually to be estimated. E is a matrix of error terms. They represent the residuals in the regression model, indicating the difference between observed values from the predicted values.
Multivariate Linear Regression vs Multiple Linear Regression
In Multivariate regression there are more than one dependent variable (y) with different variances (or distributions). The predictor variables may be one or multiple.
In Multiple regression, there is just one dependent variable i.e. y. But, the predictor variables or parameters are multiple.
Also read:
Why use Multivariate Linear Regression?
Suppose, in a medical trial, predictors might be weight, age, and race, and outcome variables are blood pressure and cholesterol. We could, in theory, create two “multiple regression” models, one regressing blood pressure on weight, age, and race, and a second model regressing cholesterol on those same factors. However, alternatively, we could create a single multivariate regression model that predicts both blood pressure and cholesterol simultaneously based on the three predictor variables. The idea being that the multivariate regression model may be better (more predictive) to the extent that it can learn more from the correlation between blood pressure and cholesterol in patients. [1]
Further reading:
1- Normal Equation
2- Normal Equation for Linear Regression
Reference:
[1] Difference between Multivariate Linear Regression and Multiple Linear Regression