The math and intuition behind the softmax function and its application in neural networks and softmax regression
—
The softmax function is one of the most important functions in statistics and machine learning. It takes a vector of K real numbers and converts it into a vector of K probabilities that sum to 1. Softmax is a generalization of the logistic function to more than two dimensions, and it can be used in softmax regression (also known as multinomial logistic regression) to address classification problems with more than two labels. The softmax function can be also used as the last activation function of a neural network in a multi-class classification problem. In this case, the neural network uses the softmax activation function to compute the probability of each possible class for the target.
This article provides a visual understanding of the softmax function, the intuition behind it, and the important mathematical properties that make it valuable in machine learning. We also discuss the relationship between the softmax and the logistic function and demonstrate how to perform a softmax regression using Python.
All the images in this article were created by the author.