// EcoFinLearn quiz questions: Machine Learning
// Source: https://ecofinlearning.com/learn/python-ml
// (c) Sitraka Forler. Teachers may use these with a class, with a link back: https://ecofinlearning.com/terms#reuse
// Import in Moodle: Question bank > Import > GIFT format.

$CATEGORY: $course$/EcoFinLearn/Machine Learning/ML Foundations

// ml-m0-l1: What Is Machine Learning, Really?
::ml-m0-l1 Q1::[plain]Your econometrics professor is delighted\: the regression has R² \= 0.92 and every p-value is below 0.01. An ML engineer looks at the same model and says "that tells me almost nothing yet." What is the engineer waiting to see?{
	=How well the model predicts data it was NOT trained on - out-of-sample performance is ML's scoreboard#A superb in-sample fit can come from memorising noise (overfitting). ML withholds unseen data and grades the model on it - that is the exam that counts.
	~A higher R² - 0.92 is considered too low in machine learning#ML does not demand higher in-sample R². In financial prediction, a near-perfect in-sample fit is usually a red flag for leakage or overfitting, not a virtue.
	~A p-value for every feature in the model#scikit-learn does not even report p-values by default. ML judges models by predictive performance, not by significance tests on coefficients.
	~A bigger model, such as a neural network#Model size is beside the point. A linear model that predicts well out-of-sample beats a deep network that does not.
}

// ml-m0-l1: What Is Machine Learning, Really?
::ml-m0-l1 Q2::[plain]What actually happens when you call model.fit(X, y) on a LinearRegression?{
	=The model estimates its coefficients (slope and intercept) from the training data - exactly OLS estimation#fit() solves the least-squares problem and stores the learned parameters in coef_ and intercept_. It is the "study the past" step - estimation, in econometric language.
	~It generates predictions for the rows of X#Predictions come from .predict(). fit() only learns the parameters; it returns forecasts for nothing.
	~It evaluates how accurate the model is#Evaluation is a separate step (e.g. .score() or error metrics), and it should ideally be done on data the model was NOT fitted on.
	~It downloads the training data from the internet#X and y are already in memory. fit() performs estimation on them - no data is fetched from anywhere.
}

// ml-m0-l1: What Is Machine Learning, Really?
::ml-m0-l1 Q3::[plain]You predict quarterly GDP growth from the change in unemployment. In ML vocabulary, which is which?{
	=Unemployment change is the feature; GDP growth is the target#Features are the inputs (what econometrics calls regressors); the target is what you predict (the dependent variable). Here growth is being predicted, so it is the target.
	~GDP growth is the feature; unemployment change is the target#Backwards - the target is the thing being predicted, and here that is GDP growth.
	~Both are features; the target is the model's coefficient#Coefficients are the parameters the model learns during fit(), not the target. The target is the variable you want to predict.
	~Both are targets; the features are whatever predict() returns#predict() returns forecasts OF the target. The inputs you feed the model are the features.
}

// ml-m0-l2: Overfitting: The Silent Portfolio Killer
::ml-m0-l2 Q1::[plain]A strategy shows a Sharpe ratio above 3 across a 10-year backtest, then loses money immediately and consistently once live. What is the most likely explanation?{
	=The model was overfit - it memorised noise in the historical data (or was the best of many variants tried on the same history)#A stellar backtest followed by immediate live failure is the classic signature of overfitting\: the fitted patterns were noise, so they vanished the moment new data arrived.
	~Markets became efficient on the exact day the strategy launched#Regimes do change, but the timing coincidence is not the parsimonious explanation - a model that only ever worked in-sample failing from day one points to overfitting.
	~The backtest code must have had an arithmetic bug#Possible, but a bug is not the systematic explanation. Overfitting produces exactly this outcome with perfectly correct code - that is what makes it so dangerous.
	~The fund used too little leverage to make the strategy profitable#Leverage scales returns up and down; it cannot turn a money-losing signal into a profitable one - it only makes the losses bigger.
}

// ml-m0-l2: Overfitting: The Silent Portfolio Killer
::ml-m0-l2 Q2::[plain]Which pattern of errors is the telltale signature of an overfit model?{
	=Very low training error together with much higher test error#Overfitting means the model bent itself around the noise in the training set - so it looks brilliant in-sample and falls apart on data it never saw.
	~High error on both the training and test sets#That is underfitting\: the model is too simple (or the features too weak) to capture the signal at all.
	~Low error on both the training and test sets#That is what a good model looks like - it generalises, performing similarly on seen and unseen data.
	~High training error but low test error#That pattern is usually luck on a small test set. Systematic overfitting shows the opposite\: great in-sample, poor out-of-sample.
}

// ml-m0-l2: Overfitting: The Silent Portfolio Killer
::ml-m0-l2 Q3::[plain]Why is standard shuffled k-fold cross-validation dangerous on financial time series?{
	=Shuffling lets folds train on the future to predict the past - look-ahead bias. Use TimeSeriesSplit or walk-forward validation instead#Once data is shuffled, a fold can contain 2023 observations while validating on 2019 - information leaks backwards in time and inflates every score. Time-ordered splits keep the exam honest.
	~k-fold cross-validation is too computationally slow for financial datasets#Speed is not the issue - k-fold is cheap. The problem is that shuffling breaks the arrow of time and leaks future information.
	~Financial datasets have too many rows for cross-validation to work#Cross-validation handles large datasets fine. The failure is about temporal ordering, not data volume.
	~It is actually fine - shuffling does not matter as long as each fold is large enough#Returns are not i.i.d. - volatility clusters and regimes persist - so a model trained on the future genuinely knows things about the past. Shuffled CV systematically overstates performance.
}

$CATEGORY: $course$/EcoFinLearn/Machine Learning/Feature Engineering

// ml-m1-l1: Feature Engineering for Alpha
::ml-m1-l1 Q1::[plain]Why do you shift features by 1 day (.shift(1)) before using them in a model?{
	=To ensure the model only uses data available at the time it makes a prediction (no look-ahead bias)#Features must be strictly older than the period they predict. Here the target is tomorrow's return, and the extra one-day lag is a defensive convention guaranteeing zero overlap even if the target definition later changes - the classification lesson shows the tighter but equally valid convention.
	~To improve model accuracy by smoothing the features#Shifting by 1 does not smooth anything - it just lags the series by one period. The purpose is strictly to prevent data leakage.
	~Because pandas requires shifted data for rolling operations#pandas has no such requirement. Shifting is a deliberate design choice to prevent look-ahead bias.
	~To align the DataFrame index with the target variable#While alignment is one side effect, the primary purpose is to ensure features are lagged relative to the target to avoid leakage.
}

// ml-m1-l1: Feature Engineering for Alpha
::ml-m1-l1 Q2::[plain]A 20-day momentum feature has a correlation of +0.04 with next-day returns. Is this useful?{
	=Yes - an IC of 0.04 is considered meaningful in finance#Equity markets are informationally efficient; even a 4% correlation between predictions and outcomes is exploitable at scale. Top quant funds target IC ≈ 0.03–0.08.
	~No - R² needs to be above 0.5 to be useful#A 0.5 R² in equity return prediction would mean the model explains 50% of variance - essentially impossible. Finance R² values of 0.01–0.05 are considered excellent.
	~No - a 0.04 correlation is effectively zero#In most fields 0.04 is negligible, but in finance with thousands of trades, even tiny predictable edges compound into large profits.
	~Only if the feature has zero correlation with all other features#Low inter-feature correlation is desirable but not required. Even a correlated feature can add value if it captures a different component of the signal.
}

// ml-m1-l1: Feature Engineering for Alpha
::ml-m1-l1 Q3::[plain]You build features on 200 days of prices using a 20-day rolling window, a 1-day shift, and a next-day target - then call dropna(). Roughly how many rows survive, and why not all 200?{
	=About 178 - the first \~21 rows are NaN warm-up (20-day window + 1-day lag) and the final row has no next-day target#Rolling windows need history before they produce a value, the shift adds one more day of lag, and the last day has no "tomorrow" to predict. dropna() removes exactly those warm-up and edge rows.
	~All 200 - rolling operations never create missing values#A 20-day rolling window has nothing to compute for the first 19 days - pandas fills those rows with NaN, and dropna() then removes them.
	~Exactly 100 - dropna() always removes half the rows#dropna() removes only rows containing at least one missing value; how many that is depends entirely on your window lengths and shifts.
	~0 - dropna() drops every row that has any value#It is the opposite\: dropna() drops rows with missing values. The starter printed (0, 6) only because two entire columns were None, which made every single row incomplete.
}

$CATEGORY: $course$/EcoFinLearn/Machine Learning/Linear Regression

// ml-m2-l1: Predict Asset Returns with Linear Regression
::ml-m2-l1 Q1::[plain]You are predicting next-day returns. Why must the train/test split follow time order instead of using sklearn's default random shuffle?{
	=A shuffled split trains the model on days that come after some test days - future information leaks into training, and the test score becomes one no live strategy could reproduce#This is look-ahead bias. A real model only ever knows the past; shuffling lets it interpolate between days it has already seen, inflating every metric.
	~Random shuffling changes the 80/20 proportion between train and test sets#The proportions stay exactly as requested. What changes is WHICH rows land where - test days scattered inside the training period are the leakage problem.
	~Time-ordered splits produce higher R² scores, which look better in reports#Usually the opposite\: an honest temporal split lowers the score dramatically. That drop is the point - it is the realistic estimate of live performance.
	~Shuffled splits are slower to compute on large financial datasets#Speed is irrelevant here - the issue is information leakage, not computation.
}

// ml-m2-l1: Predict Asset Returns with Linear Regression
::ml-m2-l1 Q2::[plain]The code runs StandardScaler().fit(X_train) and then transforms both X_train and X_test. Why not fit the scaler on all of X at once?{
	=Fitting on all of X bakes the test set's mean and variance into the preprocessing - a subtle data leak; the test set must stay completely unseen until evaluation#Every statistic estimated from data is part of the model. If the scaler saw the test period, the future contaminates training - the same crime as a shuffled split, just quieter.
	~Fitting the scaler on all of X would raise an error in scikit-learn#It runs perfectly - which is exactly why this bug is dangerous. Nothing crashes; your backtest is just silently too optimistic.
	~The test set does not need to be standardised at all#It does - the model was trained on standardised inputs, so the test features must be transformed with the same train-fitted scaler or the scales will not match.
	~StandardScaler can only handle a limited number of rows#There is no such limit. The reason is statistical hygiene, not capacity.
}

// ml-m2-l1: Predict Asset Returns with Linear Regression
::ml-m2-l1 Q3::[plain]Your test-set IC comes out at 0.04 - the correlation between predicted and actual next-day returns. What is the most professional reading?{
	=A small but potentially real edge - in liquid markets an IC of 0.02–0.06 is considered solid and can be profitable when applied across many assets and days#IC is the quant industry's standard yardstick. Tiny per-prediction correlations compound across thousands of bets - the intuition behind the Fundamental Law of Active Management.
	~The model is useless - any correlation below 0.5 is statistical noise#A daily IC of 0.5 would be beyond anything credibly documented. Finance lives on small, persistent edges; the 0.5 bar belongs to fields with far more predictable targets.
	~IC \= 0.04 means the model is right 4% of the time#IC is a correlation, not a hit rate. A positive IC means predictions and outcomes move together on average - the model can be directionally right far more often than 4% of days.
	~Re-run the model with different settings until IC exceeds 0.3#That is p-hacking the test set. Tweaking until the test metric looks good turns your test set into a training set - the score stops meaning anything.
}

$CATEGORY: $course$/EcoFinLearn/Machine Learning/Backtesting

// ml-m3-l1: Walk-Forward Backtest
::ml-m3-l1 Q1::[plain]You evaluate a daily trading model with standard 5-fold cross-validation that shuffles the rows before splitting. Why is the reported performance untrustworthy?{
	=Shuffling puts future days into the training folds, so the model trains on information it could never have had at prediction time#This is lookahead bias. Market data is ordered in time; any procedure that trains on later days to predict earlier ones inflates every metric. Walk-forward testing exists precisely to prevent this.
	~Five folds is too few - with ten folds the shuffle would be fine#The fold count is not the issue. Shuffled folds of any number leak the future into training; only time-ordered schemes (walk-forward, purged CV) respect causality.
	~Cross-validation only works for classification problems, not regression#Cross-validation works for both. The problem here is temporal leakage, not the type of model being fitted.
	~Nothing is wrong - more folds always give a more reliable estimate#For cross-sectional data, perhaps. For time series, shuffled folds systematically overstate performance because the model has effectively seen the future.
}

// ml-m3-l1: Walk-Forward Backtest
::ml-m3-l1 Q2::[plain]A fund pitches its best strategy\: Sharpe 2.4, selected after testing 500 variants on the same dataset. What does the "probability of backtest overfitting" framework (Bailey, Borwein, López de Prado and Zhu) say about this?{
	=The winner of many trials is largely selection luck - the more variants tried, the more likely the best backtest is overfit and its live Sharpe will collapse#PBO quantifies exactly this\: rank strategies in-sample, check them out-of-sample, and measure how often the in-sample winner disappoints. With 500 trials, an impressive best-of Sharpe is expected even on pure noise.
	~A Sharpe of 2.4 is high enough that the number of trials tested does not matter#The opposite\: the more trials you run, the higher the best Sharpe you would find on pure noise, so a big number after many trials carries very little evidence of skill.
	~Testing 500 variants proves robustness, since so many were examined#Only if performance is reported across ALL variants or adjusted for multiple testing (e.g. a deflated Sharpe ratio). Showing only the winner is the definition of selection bias.
	~The result is fine as long as each variant used walk-forward testing#Walk-forward removes lookahead within a single backtest, but choosing the best of 500 walk-forward backtests re-introduces overfitting at the selection level.
}

// ml-m3-l1: Walk-Forward Backtest
::ml-m3-l1 Q3::[plain]The price series in this lesson is a seeded random walk, yet your walk-forward backtest can print a healthy-looking Sharpe. What is the correct conclusion?{
	=There is nothing predictable by construction, so the number is drift plus single-path luck - expected live alpha is roughly zero, and one good backtest proves nothing#Exactly. Any single random path can trend by chance, and a strategy evaluated on one path inherits that luck. This is why quants demand many regimes, many paths and multiple-testing corrections before believing a Sharpe.
	~Ridge regression genuinely found alpha hidden in the random walk#A random walk with independent increments has no exploitable structure; no model can have real predictive alpha on it - only apparent, in-sample luck.
	~A positive Sharpe on random data means the code must contain a bug#Not necessarily - noise plus a small positive drift can produce a positive Sharpe honestly. The bug would be believing that number means skill.
	~Random walks cannot produce positive Sharpe ratios, so the data must not be truly random#Any finite sample of a random walk can show trends, streaks and a positive Sharpe purely by chance - that is precisely the trap backtests fall into.
}

$CATEGORY: $course$/EcoFinLearn/Machine Learning/Portfolio Optimisation

// ml-m4-l1: Mean-Variance Optimisation
::ml-m4-l1 Q1::[plain]The code minimises neg_sharpe(w) instead of maximising the Sharpe ratio directly. Why?{
	=scipy.optimize.minimize can only minimise - and minimising -f(w) is mathematically identical to maximising f(w)#The sign-flip is the standard idiom\: the w that minimises the negative Sharpe is exactly the w that maximises the Sharpe. You will use the same trick in maximum-likelihood estimation.
	~Sharpe ratios are always negative, so the minus sign makes them positive#Sharpe ratios can be positive or negative depending on whether the portfolio beats the risk-free rate. The minus sign is about converting a maximisation into a minimisation, not about fixing signs.
	~Minimisation converges faster than maximisation in scipy#There is no speed difference - maximising f and minimising -f are the same computation. scipy simply only ships a minimiser.
	~The minus sign prevents division by zero in the volatility term#The sign of the objective has no effect on the denominator. Volatility is strictly positive here because the covariance matrix is positive definite.
}

// ml-m4-l1: Mean-Variance Optimisation
::ml-m4-l1 Q2::[plain]DeMiguel, Garlappi & Uppal (2009) raced 14 sophisticated portfolio-optimisation models against the naive 1/N equal-weight rule, out-of-sample. What did they find?{
	=None of the 14 models consistently beat 1/N - estimation error in the inputs devoured the theoretical gains#Their Review of Financial Studies paper is the classic reference for why raw optimisation disappoints in practice\: the optimiser needs inputs far more precise than historical data can deliver.
	~Markowitz optimisation beat 1/N on every dataset, confirming the theory#The opposite - the whole reason the paper is famous is that the naive rule held its own against every optimised strategy they tested.
	~1/N lost badly because it ignores the covariance matrix#1/N does ignore the covariance matrix - and still was not consistently beaten, because the optimised models were sabotaged by noisy estimates of mu and Sigma.
	~The study proved diversification does not work#Quite the reverse\: 1/N is pure diversification and it performed admirably. The paper questions optimised weights, not diversification itself.
}

// ml-m4-l1: Mean-Variance Optimisation
::ml-m4-l1 Q3::[plain]Two stocks each have 20% annual volatility and zero correlation. You invest 50% in each. The portfolio volatility is roughly...{
	=14% - less than either stock alone\: the free lunch of diversification#Var \= 0.5^2 x 0.04 + 0.5^2 x 0.04 \= 0.02, so vol \= sqrt(0.02) \= 14.1% \= 20%/sqrt(2). Risk fell while expected return is just the plain average - nothing was given up.
	~20% - volatilities simply average out#That would require perfect correlation (+1). With any correlation below 1, the combined volatility drops below the weighted average - that is the entire point of diversification.
	~10% - half of each stock's volatility#Volatility does not add linearly. With zero correlation the VARIANCES add\: vol \= 20%/sqrt(2) \= 14.1%, not 10%.
	~0% - the two risks cancel completely#Full cancellation needs perfect NEGATIVE correlation (-1). At zero correlation the shocks are independent, so they dampen each other but never cancel.
}

$CATEGORY: $course$/EcoFinLearn/Machine Learning/Classification

// ml-m5-l1: Predict Market Direction with Logistic Regression
::ml-m5-l1 Q1::[plain]Your very first model predicts next-day market direction with 97% accuracy. What is the most likely explanation?{
	=Data leakage - information from the answer (like today's return) is inside the features, or the target is defined on the same day as a feature#Daily direction on liquid markets is close to a coin flip. A 97% score means the model can see the answer - audit when each column is known, starting with the target definition.
	~You discovered genuine alpha on your first try#If that were real you would be out-trading Renaissance's Medallion fund (right \~50.75% of the time) from a laptop. Markets are far too close to efficient for 97% daily-direction accuracy.
	~Logistic regression is simply a very powerful algorithm#No algorithm can conjure information that is not in the data. Swapping models changes how you fit - not what the features can legitimately know.
	~Accuracy is a broken metric, so the number means nothing#Accuracy has limits (class imbalance), but it is not this broken. 97% signals a data problem - leakage - not a metric problem.
}

// ml-m5-l1: Predict Market Direction with Logistic Regression
::ml-m5-l1 Q2::[plain]Your strategy only opens a position when the model predicts UP (long-only). Which metric tells you what share of your actual trades made money?{
	=Precision on the UP class - of all predicted UPs (trades you took), the fraction that were truly UP#Every predicted UP is a trade. Precision \= TP / (TP + FP) is exactly the winning share of those trades; false positives are the ones that lose money.
	~Recall on the UP class#Recall counts how many UP days you caught. Missing one (a false negative) costs opportunity, not cash - it does not describe the trades you actually took.
	~Overall accuracy#Accuracy mixes in the days the model said DOWN - days you did not trade at all - so it does not isolate your trade hit rate.
	~The F1-score#F1 blends precision and recall into one number, which hides the asymmetry a long-only trader cares about\: false positives cost money, false negatives only cost missed trades.
}

// ml-m5-l1: Predict Market Direction with Logistic Regression
::ml-m5-l1 Q3::[plain]Why does this lesson split with X[\:split] / X[split\:] instead of train_test_split(X, y, shuffle\=True)?{
	=Shuffling would train the model on days that come AFTER some test days - look-ahead bias, another form of leakage#With shuffled time series, the model effectively peeks at the future relative to the test rows, inflating every metric. Cutting by time keeps the test set strictly in the model's future, like real trading.
	~Shuffling is slower to compute#Speed is irrelevant here - the issue is temporal information flow, not performance.
	~sklearn raises an error if you shuffle time series#Unfortunately not - sklearn will happily shuffle a time series. Nothing warns you; the leakage discipline has to come from you.
	~Shuffling would change the class balance between train and test#Random shuffling roughly preserves class balance on average. The real problem is that future information contaminates training.
}

// ml-m5-l2: The Confusion Matrix & the Cost of Being Wrong
::ml-m5-l2 Q1::[plain]sklearn prints this confusion matrix for a long-only direction model (rows \= truth [DOWN, UP], columns \= prediction [DOWN, UP])\: [[80, 20], [30, 70]]. On how many days were you actually long while the market fell?{
	=20 - row "true DOWN", column "pred UP"\: the false positives, the trades that lost real money#FP lives at row 0 (truth \= DOWN), column 1 (predicted UP). You bought 20 times into a falling market - under a 3\:1 cost matrix, these 20 squares dominate your entire bill.
	~30 - the bottom-left square#Bottom-left is row "true UP", column "pred DOWN"\: the false negatives. Those are rallies you watched from the sidelines, in cash - missed opportunity, not losses.
	~70 - the biggest number in the UP row#70 is TP\: days you were long and the market rose. Those trades made money - no invoice there.
	~80 - the top-left square#80 is TN\: days you stayed in cash while the market fell - the quiet wins of doing nothing. Remember sklearn's ordering\: top-left is TN, not TP.
}

// ml-m5-l2: The Confusion Matrix & the Cost of Being Wrong
::ml-m5-l2 Q2::[plain]Your strategy only trades when the model says UP, and a losing trade costs far more than a missed rally. Which single metric deserves your closest watch?{
	=Precision on the UP class - the share of your actual trades that made money#Every predicted UP becomes a real position, so TP / (TP + FP) is exactly your hit rate on money at risk. When the false positive is the expensive error, precision is the metric that hurts when it drops.
	~Recall on the UP class#Recall counts the share of rallies you caught. Chasing it means trading more - which manufactures more of the expensive false positives you are trying to avoid.
	~Accuracy#Accuracy pools all four squares with equal weight, mixing days you never traded into your scoreboard. It is exactly the metric this lesson caught lying.
	~The F1-score#F1 averages precision and recall as if both errors cost the same - the 3\:1 asymmetry is precisely the information F1 erases.
}

// ml-m5-l2: The Confusion Matrix & the Cost of Being Wrong
::ml-m5-l2 Q3::[plain]Your model outputs P(UP), and false positives cost 3x false negatives. You keep the model unchanged but may move the decision threshold away from the default 0.5. Which way do you move it?{
	=Raise it (toward 0.6–0.75) - demand more conviction before buying, so you take fewer but safer trades#A higher bar converts borderline buys into stay-outs\: FP (expensive) falls while FN (cheap) rises - exactly the swap you want at 3\:1. Decision theory even gives the idealised cutoff\: 3 / (3 + 1) \= 0.75.
	~Lower it (toward 0.4) so you catch more rallies#Lowering the threshold buys recall by manufacturing extra false positives - the expensive error. That is the right move only when a miss costs more than a false alarm (e.g. screening for defaults).
	~Keep 0.5 - it is the statistically optimal cutoff#0.5 is only optimal when both errors cost the same. The optimal cutoff shifts with the cost ratio - with FP three times dearer, it moves up toward 0.75.
	~Thresholds cannot be changed without retraining the model#predict() is just predict_proba() plus a 0.5 cutoff. You can re-cut the same probabilities at any threshold with zero retraining - the cheapest knob in machine learning.
}

$CATEGORY: $course$/EcoFinLearn/Machine Learning/Ensemble Methods

// ml-m6-l1: Random Forest Feature Importance in Python
::ml-m6-l1 Q1::[plain]A single decision tree overfits, yet a forest of 100 trees - each trained on noisier, resampled data - generalises better. Why?{
	=Each tree makes different errors (different bootstrap sample, different feature subsets), and averaging many imperfectly correlated errors cancels most of the noise - the same mathematics as portfolio diversification#Exactly. Bagging is variance reduction through averaging\: the idiosyncratic mistakes diversify away while the shared signal survives, just like idiosyncratic risk in a diversified portfolio.
	~Each tree in the forest sees more training data than a single tree would#The opposite\: each bootstrap sample contains only \~63% distinct rows. The gain comes from averaging decorrelated trees, not from more data per tree.
	~Bagging makes every individual tree deeper and therefore more accurate#Individual forest trees are no better than a standalone tree - often slightly worse. The ensemble average is what improves.
	~The bootstrap removes outliers from the dataset before training#Outliers are not removed - resampling just means each tree weights rows randomly. Robustness comes from averaging, not filtering.
}

// ml-m6-l1: Random Forest Feature Importance in Python
::ml-m6-l1 Q2::[plain]What exactly is the out-of-bag (OOB) score?{
	=Each training row is predicted using only the trees whose bootstrap sample excluded it, giving a validation estimate without any hold-out set#Correct - every tree skips \~37% of rows, so every row has a jury of trees that never saw it. That makes OOB an honest, nearly-free estimate of generalisation.
	~The forest's accuracy on the rows it was trained on#That is training accuracy - optimistic by construction. OOB deliberately scores each row only with trees that never trained on it.
	~The accuracy on a separate test file that scikit-learn downloads automatically#Nothing is downloaded - OOB reuses your own training data, exploiting the rows each bootstrap left out.
	~The fraction of trees that agree with the majority vote#That would measure ensemble agreement (confidence), not accuracy against true labels.
}

// ml-m6-l1: Random Forest Feature Importance in Python
::ml-m6-l1 Q3::[plain]mom_5d and mom_20d are highly correlated and both genuinely predictive. What does impurity-based feature importance typically do?{
	=Split the credit between them - often arbitrarily favouring whichever a tree happens to pick first - so each can look weaker than the underlying signal really is#Once one of two correlated features is used in a split, the other adds little impurity reduction, so importance gets divided (and unstably). Permutation importance on held-out data is the standard cross-check.
	~Double-count the signal, inflating both importances#The bias runs the other way\: correlated features cannibalise each other's importance rather than inflating it.
	~Raise an error and force you to drop one of the features#scikit-learn happily trains on correlated features - no error. The distortion is silent, which is exactly why it is dangerous.
	~Nothing - feature importances are unaffected by correlation#Correlation is one of the best-documented pitfalls of impurity importance (Strobl et al., 2007). It very much affects the ranking.
}