Logistic Regression

By Sitraka Forler · Lecturer, Durham Business SchoolUpdated 3 October 2026 About this site

Classification model that turns a linear score into a probability through the sigmoid.

Logistic regression models the log-odds of an event as a linear function of the features, then squashes that score through the sigmoid so the output lies between 0 and 1. It is fitted by maximum likelihood (minimising log-loss), not by least squares, and each coefficient becomes an odds ratio once exponentiated. In finance it is the standard baseline for default prediction and up/down market classification.

The intuition

A straight line can output any number, but a probability must sit between 0 and 1. Logistic regression keeps the linear score z = β₀ + β₁x and pushes it through the sigmoid σ(z) = 1 / (1 + e^(−z)), an S-curve that maps −∞ to 0, 0 to 0.5 and +∞ to 1.

Undo the sigmoid and the model is linear again, in log-odds: log(p / (1 − p)) = β₀ + β₁x. So e^(β₁) is an odds ratio: each extra unit of x multiplies the odds of the event by e^(β₁). That is the sentence to write in a report.

There is no closed-form solution. The coefficients maximise the likelihood of the observed 0/1 labels, which is the same as minimising log-loss (cross-entropy). That loss is convex, so gradient descent or Newton's method finds the unique optimum.

Formula / theory

All 32 formulas

P(y=1 | x) = σ(β₀ + β₁x) = 1 / (1 + e^(−(β₀ + β₁x))) ;  log(p / (1 − p)) = β₀ + β₁x

In Python

from sklearn.linear_model import LogisticRegression
clf = LogisticRegression().fit(X, y)
clf.predict_proba(X)[:, 1]   # probability of class 1
import numpy as np; np.exp(clf.coef_)  # odds ratios

Run code like this in your browser, free: Predict Market Direction with Logistic Regression · Machine Learning, 20 min

From score to probability, by hand

Model: β₀ = −1, β₁ = 0.5, and a borrower with feature x = 4 (say, debt-to-income in tens of percent).
Linear score: z = −1 + 0.5 × 4 = 1.
Probability: p = 1 / (1 + e^(−1)) = 1 / (1 + 0.3679) = 1 / 1.3679 ≈ 0.731.
Odds: p / (1 − p) = 0.731 / 0.269 ≈ 2.72 = e^1. Log-odds = 1 = z, as it must be.
Decision boundary: p = 0.5 exactly when z = 0, i.e. x = 2. Odds ratio per unit of x: e^(0.5) ≈ 1.649, so each extra unit of x multiplies the odds of default by about 1.65.

The model is a line in log-odds space and an S-curve in probability space. Threshold at 0.5 only if a false positive costs the same as a false negative; in credit and fraud it never does.

Common pitfalls

Judging by accuracy on imbalanced data: with 2 % defaults, predicting 'no default' for everyone scores 98 %. Use precision, recall, ROC-AUC and a confusion matrix weighted by real costs.
Reading coefficients as probabilities. They act on log-odds; exponentiate for odds ratios, or compute marginal effects at a given x.
Perfect separation: if a feature splits the classes perfectly, the likelihood has no maximum and coefficients run off to infinity. Regularise (sklearn's default C = 1 already does) or drop the feature.
Forgetting to scale features before a regularised fit, so the penalty hits large-unit features hardest.
Trusting predicted probabilities without a calibration check (reliability curve): a model that ranks well can still be badly calibrated.

Frequently asked questions

Why is it called regression if it classifies?

Because it regresses the log-odds of the class on the features. Classification only happens when you choose a threshold on the predicted probability.