The Confusion Matrix & the Cost of Being Wrong

The Confusion Matrix & the Cost of Being Wrong 💸

Tuesday, same quant desk. Yesterday you fixed the leakage and your logistic regression earned an honest ~58% on next-day direction - genuinely good for daily data, remember the 52–56% ballpark. You draft the go-live email. The risk manager reads it and asks one question: "Fifty-eight percent of WHAT? Show me the confusion matrix." She is not being pedantic. Hidden inside your 42% of errors are two very different kinds of mistake - and in a long-only book, one of them sends an invoice while the other only sends regret.

The four squares every trader should read

A binary classifier can be right or wrong in exactly four ways. For a long-only strategy - buy when the model says UP, stay in cash when it says DOWN:

scikit-learn hands you all four in one call, with an ordering quirk to burn into memory: rows = truth, columns = prediction, class 0 first - so the top-left square is TN, not TP as in many textbook drawings:

from sklearn.metrics import confusion_matrix
tn, fp, fn, tp = confusion_matrix(y_test, y_pred).ravel()

🧠 The mental model: not every error sends the same invoice

Accuracy treats the four squares as equal citizens: (TP + TN) / all, one vote each. Your P&L does not vote democratically:

• a false positive - you bought, the market fell → a real loss on a real position;
• a false negative - you stayed in cash, the market rallied → opportunity cost. Annoying, but nobody posts margin on a trade they never made.

Your desk prices the asymmetry at 3 : 1 - one losing trade hurts as much as three missed rallies:

Expected cost = 3 × FP + 1 × FN      (correct calls TP and TN cost 0)

The beauty of a cost matrix: every strategy can be priced with the same ruler - including doing nothing. The always-out baseline never buys: zero false positives by construction, but every true UP day becomes a missed rally at 1 point each. That sleepy benchmark is what your model must beat on cost - not on accuracy.

Why accuracy lies

A classic from credit: if 99% of borrowers repay, the model "approve everyone" is 99% accurate - and bankrupts the bank, because the 1% it waves through are exactly the errors that cost fifty times the margin earned on a good loan. Accuracy lies whenever classes are imbalanced or errors have asymmetric costs. Markets serve you the second condition every single day.

And the number a long-only trader actually feels is precision on UP = TP / (TP + FP): of the trades I took, what share made money? Every point of imprecision is a real losing trade - billed at 3× in our cost matrix.

Your Task

The starter reuses yesterday's exact lab - seed 42, n = 400, the honest next-day target, the temporal split - trains the same logistic regression, and prints the labelled confusion matrix. What it does not do yet is price the mistakes.

Predict before you run: the model's accuracy (~58%) comfortably beats the always-out baseline's (~46%). Under 3:1 costs, which one wins - the model, or doing nothing at all? Commit to an answer.

Then fill in the two 👉 YOUR TURN lines (price the model, price the baseline), run, and read the VERDICT. If it stings - good. The sting is the lesson.