MAE vs MSE vs RMSE vs R²: regression metrics explained

MAE tells you the average absolute miss, MSE and RMSE penalize large misses more heavily, and R² compares your model with a mean-value baseline. This guide explains the formulas, works through the same five predictions by hand, and gives practical rules for choosing a metric without hiding important errors.

What is the difference between MAE, MSE, RMSE, and R²?

MAE, or mean absolute error, is the average size of the prediction error without considering direction. MSE, or mean squared error, squares each error before averaging, so a few large misses have much more influence. RMSE, or root mean squared error, is the square root of MSE, which puts the result back in the target’s original units. R², or the coefficient of determination, measures how much better the model is than a baseline that always predicts the average target value.

As a rule of thumb, use MAE when a typical miss is easiest to explain in the business unit. Use RMSE when large errors are disproportionately painful and you still want an answer in that unit. Use MSE when optimizing a model or explicitly assigning extra penalty to large errors. Use R² as context for a baseline comparison, not as the only score. That rule fails when the business loss, data distribution, or evaluation target makes a different error shape more important.

What do the four regression metrics measure?

Let yᵢ be the observed value, ŷᵢ be the prediction, and n be the number of paired observations. The residual is the signed error, defined here as actual minus predicted:

A positive residual means the prediction was too low. A negative residual means it was too high. The sign is useful for finding bias, but it cancels out if you simply average residuals, which is why the common error metrics transform it before averaging.

MAE: average absolute error

MAE gives every observation a linear penalty. An error of 4 counts as four times an error of 1. If your target is measured in dollars, minutes, kilograms, or degrees, MAE uses that same unit. A MAE of 2.5 means the model misses by 2.5 target units on average, although it does not mean every prediction is within 2.5.

MSE: average squared error

MSE makes the unit less intuitive because the result is in squared target units. Its benefit is deliberate sensitivity to large errors: an error of 4 contributes 16, while an error of 1 contributes 1. That makes MSE useful when a large miss is more than four times as undesirable as a small miss, and it also gives optimization methods a smooth objective.

RMSE: square-rooted MSE

RMSE preserves the large-error emphasis of MSE but returns to the original target units. It is often easier to communicate than MSE. RMSE is always at least as large as MAE for the same observations, and the gap grows when the errors are uneven. If RMSE is much higher than MAE, inspect the largest residuals rather than assuming the model is uniformly poor.

R²: comparison with the mean baseline

Here, SSE is the sum of squared model errors, and SST is the total squared deviation of actual values from their mean:

R² is unitless. An R² of 1 means perfect predictions on the evaluated observations. An R² of 0 means the model has the same squared error as always predicting the actual mean. R² can be negative when the model is worse than that baseline. It is not a percentage accuracy, and it does not guarantee that individual predictions are close.

How do you calculate the metrics by hand?

Use five observations. The actual values are 2, 4, 5, 7, 10 and the predictions are 3, 3, 5, 8, 8. The values have no special domain; they are small enough to audit by hand and illustrate both ordinary errors and one larger miss.

The residual sum is not needed for these metrics; it is shown only to make the directional cancellation visible. The important aggregate sums are absolute error 5 and squared error 7.

There are n = 5 observations, so:

For R², first calculate the actual mean: ȳ = (2 + 4 + 5 + 7 + 10) / 5 = 5.6. The squared deviations from that mean are:

Therefore SST = 37.2 and SSE = 7:

These results tell a coherent story. The typical absolute miss is 1 target unit. The single error of 2 receives extra weight in RMSE, which rises above MAE to about 1.18. The model’s squared error is about 81.2% lower than the mean-prediction baseline on this small sample. That last statement is a baseline comparison, not a guarantee that 81.2% of predictions are correct.

When should you use MAE, MSE, RMSE, or R²?

The best practice is often to report a pair: MAE or RMSE for operational meaning, plus R² for baseline context. If the cost of an error is known, use that cost to define or supplement the metric rather than choosing because one score looks nicer. For example, a forecasting system may care more about under-predicting demand than over-predicting it; ordinary symmetric MAE and RMSE do not express that preference. A quantile or asymmetric loss may be more appropriate.

Also keep the evaluation scale fixed. MAE and RMSE can be compared across model versions when the target unit and test set are the same. Comparing an RMSE measured in dollars with one measured in thousands of dollars is meaningless unless you first normalize the units. R² is safer across unit conversions, but not across datasets with very different target variance or difficulty.

How can regression evaluation go wrong?

Training on one metric and reporting another without explanation. A model optimized for MSE may reduce large errors while making the median error less pleasant. That is not automatically bad, but your report should explain which behavior the product needs.

Using the mean of signed residuals as “average error.” In this example, the signed residual sum is 1, so the mean signed error is 0.2. That can reveal directional bias, but positive and negative errors cancel. It is not a substitute for MAE or RMSE.

Looking only at R². A high R² can coexist with errors that are too large for the use case, especially when the target naturally varies over a wide range. A low R² can occur in a narrow-range dataset even when absolute errors are acceptable. Always show at least one unit-based metric.

Letting leakage enter the test set. If a feature contains information that would only be available after the prediction time, the metric will be optimistic. Time-dependent problems need a time-aware split; randomly shuffling future observations into training can make a model look much better than it will be in production.

Deleting outliers without a policy. A large residual may be a data error, a rare but important event, or the exact failure the product needs to handle. Check the record, document the rule, and report sensitivity with and without the observation instead of silently removing it.

Calculating on mismatched pairs. Every prediction must line up with the actual value for the same observation and horizon. Sorting one series independently, dropping missing values from only one side, or mixing one-step and multi-step forecasts corrupts every metric.

What edge cases should you report?

If all actual target values are identical, SST = 0, so R² has no defined baseline denominator. In that case, report MAE, MSE, and RMSE and mark R² as undefined rather than manufacturing a value. The regression calculator displays a dash for this situation.

Negative predictions or targets are not a problem for these formulas. Zero targets are also valid, unlike percentage errors that divide by the actual value. If your target spans several orders of magnitude, consider whether a transformation or a scale-free metric fits the decision better, and evaluate after converting predictions back into the business unit.

For multiple outputs, state how you aggregated the scores: average the per-output metrics, weight outputs by business importance, or flatten every value into one pool. Those choices answer different questions. For small test sets, show the number of observations and a residual plot or error distribution when possible; one average can conceal a systematic failure in a subgroup.

Use the calculator and compare metric families

Try the MAE, MSE, RMSE, and R² calculator with the worked example above, then replace the values with a representative holdout sample. To compare classification evaluation, read accuracy vs precision vs recall vs F1; the same principle applies: choose the score that reflects the errors your system actually cares about.

When a model is part of an LLM product, combine quality metrics with operating cost. The LLM API cost guide explains how to forecast usage, while LLM API pricing explained covers cost per successful outcome and hidden failure costs.

FAQ