Decision Threshold
How is the decision threshold chosen?
Answer
In logistic regression, the predicted target value \(\hat{Y}\) should be either 0 or 1. To make this classification, we use the modeled probability \(P(Y = 1 \mid X)\) and apply the following rule:
\[
\hat{Y} =
\begin{cases}
1, & \text{if } P(Y = 1 \mid X) \geq 0.5 \\
0, & \text{otherwise.}
\end{cases}
\]
This is equivalent to,
\[
\hat{Y} =
\begin{cases}
1, & \text{if } \beta^{T}X \geq 0 \\
0, & \text{otherwise.}
\end{cases}
\]
Notes and comments
Comment 1: If we had two predictors (features), the decision boundary would be a straight line in the 2D predictor space that separates the data points belonging to different classes. This line will of course depend on the estimated parameters \(\beta\).