Normalized RMSE & Normalized MAE

To compare a model's performance across different datasets by scaling the error metrics, you can normalize both RMSE and MAE:

Normalized RMSE:

• using the mean of data set: $NRMSE=\frac{RMSE}{\overline{y}}$ (i.e., Coefficient of Variation of RMSE)

• using the difference between maximum and minimum in data set: $NRMSE=\frac{RMSE}{y_{max}-y_{min}}$,

• using the standard deviation of data set: $NRMSE=\frac{RMSE}{\sigma }$, or

• using the interquartile range of data set; $NRMSE=\frac{RMSE}{Q1-Q3}$, i.e. the difference between 25th and 75th percentile, of observations.

https://www.marinedatascience.co/blog/2019/01/07/normalizing-the-rmse/

Normalized MAE: 

• aka. Coefficient of Variation of MAE; Coefficient of Variation (CV) =  the ratio of the standard deviation ${\displaystyle \sigma }$ to the mean ${\displaystyle \mu }$, ${\displaystyle CV=\frac{\sigma }{\mu }.}$

• using the mean of data set: $N\mathrm{MA}E=\frac{\mathrm{MA}E}{\overline{y}}$ 

https://d2uars7xkdmztq.cloudfront.net/app_resources/38997/documentation/135539_en.pdf

===

What is the Coefficient of Variation (CV)?

The Coefficient of Variation (CV) is a standardized measure of the dispersion of a probability distribution or frequency distribution.

In plain English: It tells you how much variation there is relative to the mean.

The Formula

$$CV = \frac{\sigma}{\mu}$$

Where:

• $\sigma$ = Standard deviation (the measure of absolute spread)
  
  
• $\mu$ = Mean (the average)
  
  

(Note: It is often expressed as a percentage by multiplying the result by 100: $CV = (\frac{\sigma}{\mu}) \times 100\%$)

Why Do We Need It? (The Problem CV Solves)

Standard deviation ($\sigma$) is measured in the same units as your data. This makes it hard to compare the variability of two different datasets if they have different units or drastically different means.

Example: Elephants vs. Mice

Imagine you are a biologist measuring weights:

• Dataset A (Elephants): Mean weight $\mu = 4,000 \text{ kg}$, Standard deviation $\sigma = 400 \text{ kg}$.
  
  
• Dataset B (Mice): Mean weight $\mu = 20 \text{ g}$, Standard deviation $\sigma = 2 \text{ g}$.
  
  

If you just look at standard deviation, elephants look 200 times more variable ($400$ vs $2$). But that is misleading because elephants are massive.

Let's look at the Coefficient of Variation:

• CV for Elephants: $\frac{400}{4000} = 0.10$ (or 10%)
  
  
• CV for Mice: $\frac{2}{20} = 0.10$ (or 10%)
  
  

Once normalized by the mean, you realize that both populations have the exact same relative amount of variability.

Key Characteristics & Use Cases

1. Unitless: Because you divide a unit by the same unit (e.g., kilograms divided by kilograms), the CV has no units. This allows you to compare completely different types of data (e.g., comparing the volatility of stock prices against daily temperature fluctuations).
   
   
2. Finance & Risk Management: In portfolio management, CV is often used as the Sharpe ratio (or related risk-to-return metrics) to see how much risk (standard deviation) you are taking per unit of expected return (mean).
   
   
3. Machine Learning / Feature Engineering: CV is often used to check feature stability. If a numerical feature has a very low CV across your dataset, it is essentially a constant and might provide little to no predictive value for a machine learning model.
   
   

Limitations

• Unstable near zero: If your mean ($\mu$) is close to zero, the CV explodes to infinity or becomes wildly uninterpretable, making it useless for data centered around zero (like temperature in Celsius or profit/loss statements that swing between positive and negative).