There are many articles on regression evaluation metrics, such as MSE, MAE, RMSE, etc. These metrics are very important when we care about the mean or median prediction. However, when we want to train our models to focus on other locations in the distribution, we have to use a different metric, which is not so frequently described in data science blog posts.

In this article, we will explore the quantile loss, also known as the pinball loss, which is the go-to metric in quantile regression.

A few definitions to get us started

Before explaining the quantile loss, let’s quickly go through a few definitions to make sure we are on the same page.

Let’s start with a simple one. Algorithms that belong to the regression type predict a continuous variable, for example, they predict the temperature, the price of a stock, the demand for the latest iPhone, etc.

Now it is a time for a refresher from statistics. An α quantile is a value that divides a given set of numbers such that α × 100% of the numbers are less than or equal to this value, while the remaining (1 − α) × 100% of the…