← Back to Modules Directory

Module 08 - Logistic Regression

This study reference outlines the mathematical architecture of logistic regression classification models. It covers the progressive mapping pipeline that links bounded probability spaces to continuous log-odds equations via sigmoid transformations and maximum likelihood parameters.

1. The Probability-to-Odds Transformation Pipeline

To use a linear combination of features for binary classification, the bounded target must be mapped into an infinite continuous scale:

Probability: The baseline frequency index of an event materializing, calculated as the count of target observed outcomes divided by the total possible sample space outcomes. Bound strictly between 0 and 1.
Odds: The ratio comparing the probability of an event manifesting relative to the probability of that same event failing to occur ("what happened divided by what didn't happen"). This transforms the event score into a continuous range from 0 to +∞.
Log Odds: The natural logarithm computed over the raw odds ratio. Taking the log of the odds forces the variable space to become perfectly symmetrical around 0 and expand from -∞ to +∞, creating an infinite continuous scale that linear equations can estimate.

2. Functional Foundations & Link Inversions

Logistic architectures rely on inverse functional pairs to step between linear predictors and bounded probability outputs:

The Logit Function (The Inverse Link)

The Logit function is the mathematical logarithm of the odds ratio, mapping input probability values bounded between 0 and 1 across an open scale from -∞ to +∞. It defines the foundational framework of a Logistic Model (or logit model), establishing the log-odds of a binary event as a direct linear combination of independent input variables:

logit(p) = ln(p / (1 - p)) = b0 + b1*x1 + b2*x2 + ... + bn*xn

The Logistic / Sigmoid Function (The Activation Link)

The mathematical inverse of the logit mapping is the Sigmoid function (or inverse-logit function). It compresses an infinite coordinate space ranging from -∞ to +∞ back into a smooth, S-shaped cumulative probability distribution bounded strictly between 0 and 1 along the vertical axis, matching a clear midpoint probability of exactly 0.5 at x = 0.

The standard operational Logistic Function used to output the final conditional probability of an event based on calculated independent feature parameters is structured as:

p = 1 / (1 + e^-(b0 + b1*x))

Euler's Number (e): A mathematical constant tracking near 2.71828 that serves as the standard base for exponential growth functions and logistic activation transformations within machine learning algorithms. Its derivative function (y = e^x) equals itself.

3. Mapping Class Boundaries

Logistic Regression: A classification model that models the conditional probability of a discrete categorical outcome by passing a linear combination of independent explanatory features through a logistic sigmoid transformation.
Classification Threshold (Decision Threshold): The explicit probability boundary value used to map continuous model probabilities to a rigid binary category.
Example: If a spam classification script sets a decision threshold of 0.50, any incoming email message yielding a sigmoid probability score of 0.9995 safely maps to the positive binary classification label (Spam).