Logistic Loss
What loss function do we use for logistic regression?
Answer
Logistic regression uses the logistic loss (cross-entropy loss) function. It is given by
\[
J(\beta) = -\frac{1}{n} \sum_{i=1}^{n} \left[ y_i \log(\hat{p}_i) + (1 - y_i)\log(1 - \hat{p}_i) \right],
\]
where \(\hat{p}_i = P(Y_i = 1 \mid X_i)\). The parameters \(\beta\) are chosen to minimize this loss.