## Definition A **linear regression model** is used to explain a given variable $y$ based on one or more input variables. These input variables can be: - **Univariate Regression**: When the input is a single scalar variable, denoted as $x \in \mathbb{R}$. - **Multivariate Regression**: When multiple input variables are used, represented as $x \in \mathbb{R}^n$. ## Mathematical Formulation A linear regression model expresses the target variable $y$ as a **linear function** of the input variables. The general form of the model is: $ \hat{y} = \theta_0 + \theta_1 x_1 + \theta_2 x_2 + \dots + \theta_n x_n $ Or, using vector notation: $ \hat{y} = \theta^T x $ where: - $\theta$ is the parameter vector $[\theta_0, \theta_1, \dots, \theta_n]$, including the bias term $\theta_0$. - $x$ is the feature vector $[x_0, x_1, \dots, x_n]$, where $x_0 = 1$ to incorporate the bias term. This representation enables efficient computation of predictions using matrix operations. ## Example: Predicting Life Satisfaction A simple example of linear regression is the prediction of **life satisfaction** based on **GDP per capita**: $ \text{life\_satisfaction} = \theta_0 + \theta_1 \times \text{GDP\_per\_capita} $ Here, $\theta_0$ represents the **intercept term**, while $\theta_1$ is the **weight** associated with GDP per capita. This equation follows the standard linear model structure, where the prediction is obtained by computing a weighted sum of the input feature and a constant bias term. More generally, a linear model can be understood as an optimization problem where the goal is to determine the values of $\theta$ that minimize the prediction error on a given dataset. This is commonly achieved using **least squares estimation** or **gradient descent**. ## Key Properties of Linear Regression - **Simplicity & Interpretability**: Linear regression is widely used due to its easy interpretability and computational efficiency. - **Assumption of Linearity**: The model assumes a linear relationship between input features and the target variable. - **Scalability**: Can be extended to multiple variables, allowing for **multivariate regression**. - **Optimization via Mean Squared Error (MSE)**: The best-fit parameters are typically found by minimizing the mean squared error (MSE) between predicted and actual values.