// EcoFinLearn quiz questions: Data Manipulation
// Source: https://ecofinlearning.com/learn/python-pandas
// (c) Sitraka Forler. Teachers may use these with a class, with a link back: https://ecofinlearning.com/terms#reuse
// Import in Moodle: Question bank > Import > GIFT format.

$CATEGORY: $course$/EcoFinLearn/Data Manipulation/DataFrames

// pp-m1-l1: Pandas DataFrame Basics: describe(), head() & dtypes
::pp-m1-l1 Q1::[plain]What does df.describe() return for a numeric column?{
	=Count, mean, std, min, 25th/50th/75th percentiles, and max#These 8 statistics give a complete summary distribution of the column - essential for quickly checking if data looks reasonable.
	~Only the mean and standard deviation#describe() returns 8 statistics, not just 2. Use df["col"].mean() and df["col"].std() if you only want those.
	~The data types of each column#Data types come from df.dtypes. describe() focuses on the statistical distribution.
	~The first and last values in each column#Use df.head() and df.tail() for that. describe() is about the statistical summary.
}

// pp-m1-l1: Pandas DataFrame Basics: describe(), head() & dtypes
::pp-m1-l1 Q2::[plain]How do you select a single column from a DataFrame called df?{
	=df["close"] or df.close#Both work. df["close"] is preferred when the column name has spaces or conflicts with a DataFrame method name.
	~df.get("close")#DataFrames do have a .get() method but it returns None for missing keys rather than raising an error. Use df["close"] for standard access.
	~df[0]#df[0] looks up a COLUMN labelled 0 - it never selects by position. This frame has no column named 0, so it raises a KeyError; use df["column_name"] for column selection.
	~df.select("close")#There is no .select() method on pandas DataFrames. Use df["close"] or df[["close"]] for a DataFrame (double brackets).
}

// pp-m1-l1: Pandas DataFrame Basics: describe(), head() & dtypes
::pp-m1-l1 Q3::[plain]You load 10 days of prices, but df.describe() shows count as 9.00. What is the most likely explanation?{
	=One of the 10 values is missing (NaN) - count reports non-null values only#Real market data always has gaps (holidays, trading halts, vendor glitches). Comparing the count row against df.shape is the fastest missing-data check in pandas - exactly why the handshake starts with those two lines.
	~describe() always excludes the most recent row#describe() uses every non-null value in the column. It never drops rows by position or date.
	~The index row is subtracted from the count#The index labels the rows - it is not part of the column data and has no effect on count.
	~You must call df.info() first so pandas registers all rows#There is no registration step. df.info() is just another inspection tool - it reports the same non-null counts and would confirm the NaN gap.
}

// pp-m1-l1: Pandas DataFrame Basics: describe(), head() & dtypes
::pp-m1-l1 Q4::[plain]df holds 10 daily closes. How do you get the LAST close by position?{
	=df["close"].iloc[-1]#.iloc selects by integer position, so -1 is the last row whatever the index labels are. .iloc[0] would be the first.
	~df["close"].loc[-1]#.loc selects by index LABEL. Unless the index literally contains -1, this raises a KeyError. Use .iloc for positional access.
	~df["close"][-1]#On a Series whose index is not the default integer range this is ambiguous and often errors or warns. Be explicit\: .iloc[-1] for position, .loc[label] for a label.
	~df["close"].last()#.last() is a time-based offset selector (e.g. last("5D")), not "the last element". For the final value use .iloc[-1].
}

$CATEGORY: $course$/EcoFinLearn/Data Manipulation/Returns & Rolling Stats

// pp-m2-l1: Log Returns & 20-Day Volatility
::pp-m2-l1 Q1::[plain]A stock has a daily volatility of 1.5%. What is its approximate annualised volatility?{
	=About 23.8%#1.5% × √252 ≈ 1.5% × 15.87 ≈ 23.8%. This is a typical annual vol for a large-cap stock.
	~About 3%#Simply doubling or multiplying by 2 ignores the √252 scaling. Volatility scales with the square root of time, not linearly.
	~About 378%#1.5% × 252 \= 378% would be correct if volatility scaled linearly with time - but it scales with √time.
	~About 1.5%#That is the daily volatility. Annualising requires multiplying by √252 ≈ 15.87.
}

// pp-m2-l1: Log Returns & 20-Day Volatility
::pp-m2-l1 Q2::[plain]What does df["close"].rolling(20).std() compute?{
	=The standard deviation of the last 20 closing prices at each date#A rolling window of 20 observations moves one step at a time, computing std() over each window - producing a time series of local volatility.
	~The standard deviation of the entire price series divided by 20#That would be df["close"].std() / 20. Rolling applies the function to each 20-row window independently.
	~The 20-day simple moving average#That would be df["close"].rolling(20).mean(). .std() computes standard deviation, not mean.
	~The last 20 rows of the DataFrame#df.tail(20) gives the last 20 rows. rolling(20).std() computes a rolling standard deviation across the whole series.
}

// pp-m2-l1: Log Returns & 20-Day Volatility
::pp-m2-l1 Q3::[plain]Your vol_20 column shows exactly the same positive value on every row of the DataFrame. What most likely happened?{
	=std() was computed over the whole sample instead of a rolling window, and the scalar was broadcast to every row#df["log_ret"].std() returns a single scalar; assigning a scalar to a column stamps it on every row. A rolling estimate varies by date and has NaN warm-up rows at the start.
	~The stock price did not move during the period#A truly flat price would produce ZERO volatility, not a positive constant - and the log_ret column would be all zeros, which it clearly is not.
	~The window is larger than the dataset, so pandas silently reuses the full sample#If the window exceeded the available data, rolling().std() would return ALL NaN - pandas never silently substitutes the full sample.
	~np.sqrt(252) was forgotten#Forgetting annualisation changes the LEVEL of every value, but a rolling estimate would still vary day by day.
}

// pp-m2-l2: Resample Daily Prices to Monthly Returns
::pp-m2-l2 Q1::[plain]You have daily closes and want one MONTH-END price per month. Which is correct?{
	=close.resample('ME').last()#resample('ME') buckets the DatetimeIndex by calendar month-end; .last() takes each month's final close.
	~close.iloc[\:\:21]#A fixed 21-row step ignores that months have 18-23 trading days, so it drifts off the real month-ends and mislabels the returns.
	~close.groupby(close.index.month).last()#Grouping by month NUMBER (1-12) collapses every January across all years into one bucket. resample keeps each calendar month separate.
	~close.rolling(21).last()#rolling gives a value for EVERY day over a trailing window; it does not downsample to one row per month.
}

// pp-m2-l2: Resample Daily Prices to Monthly Returns
::pp-m2-l2 Q2::[plain]After monthly_close, how do you get month-over-month returns?{
	=monthly_close.pct_change().dropna()#pct_change() computes (this - prev)/prev between consecutive month-end levels; dropna() removes the first (undefined) month.
	~monthly_close.diff()#diff() gives the absolute price change in dollars, not the percentage return.
	~monthly_close.mean()#That collapses to a single average price, losing the per-month returns entirely.
	~np.log(monthly_close)#Taking the log of the level is not a return. A log RETURN would be np.log(monthly_close).diff().
}

$CATEGORY: $course$/EcoFinLearn/Data Manipulation/Multi-Asset Analysis

// pp-m3-l1: Correlation Matrix of Stock Returns in pandas
::pp-m3-l1 Q1::[plain]A colleague computes df.corr() on the raw closing PRICES of two tech stocks that both trended upward all year, gets 0.95, and declares them "almost perfectly co-moving". What is wrong?{
	=Price levels trend, so their correlation mostly measures shared drift - compute the correlation on daily returns instead#Two trending series always look highly correlated even if their day-to-day moves are unrelated - the spurious-correlation trap (Yule 1926, Granger & Newbold 1974). Returns strip out the trend and measure genuine co-movement.
	~Nothing - 0.95 on prices is a valid measure of co-movement#It is not. Two independent random walks with drift routinely show price-level correlations near ±1. The number says "both trended", not "they move together".
	~The correlation should have been computed on log prices instead#Log prices still trend, so the spurious-correlation problem remains. The fix is differencing - i.e. using returns (or log returns), which are approximately trend-free.
	~One year of daily data is too small a sample for any correlation#Roughly 250 daily observations is a perfectly workable sample for estimating an equity correlation. The problem is what is being correlated, not how much of it.
}

// pp-m3-l1: Correlation Matrix of Stock Returns in pandas
::pp-m3-l1 Q2::[plain]During September–October 2008, average pairwise correlations across large-cap US equities jumped from roughly 0.35 to 0.85–0.95. What is the key practical implication for a portfolio manager?{
	=Diversification fails exactly when it is needed most - a historical correlation matrix understates crisis risk#Correlation is conditional on the market regime. In a panic, nearly everything falls together, so risk models calibrated on calm-period matrices understate potential losses - the motivation for stress testing and regime-switching models.
	~It proves the portfolio was badly constructed in the first place#Even textbook-perfect diversified portfolios suffered\: the spike hit the whole equity market. The lesson is about regime-dependence of correlations, not about one manager's stock picks.
	~Correlations near 1 mean the market had become risk-free#The opposite\: high correlation means everything drops simultaneously, concentrating rather than eliminating risk.
	~The 2008 correlation estimates must have been computation errors#The spike is a well-documented empirical fact known as correlation breakdown under stress - a core limitation of static mean-variance analysis.
}

// pp-m3-l1: Correlation Matrix of Stock Returns in pandas
::pp-m3-l1 Q3::[plain]For the 4-asset matrix in this lesson (AAPL, MSFT, JPM, GS), how many DISTINCT pairwise correlations does it actually contain?{
	=6 - the matrix is symmetric with a diagonal of ones, leaving n(n-1)/2 unique pairs#The 16 cells break into 4 trivial diagonal ones (each asset with itself \= 1.0) and 12 off-diagonal cells that mirror each other, so 4×3/2 \= 6 unique numbers - precisely the six bars in the text heatmap.
	~16 - one for every cell of the 4×4 matrix#The diagonal is always exactly 1 and the matrix is symmetric (corr(A,B) \= corr(B,A)), so most cells are redundant.
	~12 - everything except the diagonal#Close, but each off-diagonal value appears twice (above and below the diagonal), so only half of the 12 are distinct.
	~4 - one per asset#Correlations are properties of PAIRS of assets, not of single assets. With 4 assets there are 4×3/2 \= 6 pairs.
}

// pp-m3-l2: GroupBy: Split-Apply-Combine Stats per Ticker
::pp-m3-l2 Q1::[plain]The book has 9 rows\: 3 each for AAPL, MSFT and JPM.\n\nstats \= df.groupby("ticker")["ret"].agg(["mean", "std", "count"])\n\nHow many ROWS does stats have?{
	=3 - one per group (ticker)#Right. Combine returns one row per distinct key. Nine input rows collapse into three groups, each carrying its own mean, std and count of 3.
	~9 - one per original row#That would be the input. The whole point of aggregating is to collapse each group into a single summary row, so 9 rows over 3 tickers becomes 3 rows.
	~1 - one blended summary row#That is what you get WITHOUT the groupby (the starter). Adding .groupby("ticker") produces one row per ticker instead of one for everything.
	~27 - 3 groups x 3 aggregations#The three aggregations become three COLUMNS, not rows. Shape is 3 rows (groups) x 3 columns (mean, std, count).
}

// pp-m3-l2: GroupBy: Split-Apply-Combine Stats per Ticker
::pp-m3-l2 Q2::[plain]A colleague reports the book's volatility as a single number by running df["ret"].std() on a long-format table of 40 tickers. What is wrong?{
	=It blends all 40 names into one series, so it measures cross-sectional spread, not any ticker's volatility#Exactly. Without grouping, the rows of every ticker are pooled, so the figure mixes differences BETWEEN names with variation within them. It is a real number answering a question nobody asked.
	~Nothing - .std() always handles groups automatically#Pandas has no idea the table is long-format; it just sees one column of numbers. You must state the grouping with .groupby("ticker") for per-name volatility.
	~It will raise an error because the table has multiple tickers#It runs perfectly happily - that is the danger. The result is plausible and silently wrong, which is why sanity-checking your group counts matters.
	~It returns 40 numbers, one per ticker#That is what .groupby("ticker")["ret"].std() returns. The ungrouped call collapses everything to one number.
}

// pp-m3-l3: CAPM Beta in Python: Regression on Market Returns
::pp-m3-l3 Q1::[plain]A stock's monthly returns have a standard deviation of 6%, the market's 4%, and their correlation is 0.5. What is the stock's beta?{
	=0.75\: correlation × (6% / 4%)#Beta \= correlation × stock volatility / market volatility \= 0.5 × 1.5 \= 0.75. The stock swings 1.5 times as much as the market, but only partly in step with it, so the slope of the stock on the market is 0.75.
	~1.5\: the ratio of the two volatilities#That is relative volatility, the starter's mistake. It would equal beta only if the correlation were exactly 1, so that every move in the stock came with the market.
	~0.5\: the correlation itself#Correlation says how tightly the dots hug the line, not how steep the line is. Beta scales it by the volatility ratio\: 0.5 × 1.5 \= 0.75.
	~0.33\: correlation × (4% / 6%)#That is the slope of the reverse regression, market on stock. CAPM regresses the stock (y) on the market (x), so the market's variance goes in the denominator.
}

// pp-m3-l3: CAPM Beta in Python: Regression on Market Returns
::pp-m3-l3 Q2::[plain]On 12 monthly returns, you compute beta as s.cov(m) / np.var(m), where s and m are pandas Series. It comes out about 9% higher than the np.polyfit slope. What went wrong?{
	=np.var divides by n (ddof\=0) while pandas .cov() divides by n - 1 (ddof\=1), inflating the ratio by 12/11#Right. 12/11 is about 1.09, the 9% gap. The regression slope has no ddof at all, because the n - 1 cancels, so cov / var matches it only when both halves use the same convention. Use m.var() or np.var(m, ddof\=1).
	~np.polyfit is less accurate than cov / var on small samples#polyfit solves the least-squares problem directly. With a consistent ddof the two routes agree to floating-point precision, which is what the lesson's cross-check tests.
	~Beta should be computed on price levels, not returns#The opposite. Beta is a relationship between returns; trending price levels produce spurious co-movement, as the correlation lesson showed.
	~pandas only approximates the covariance when there are fewer than 30 observations#pandas computes the exact sample covariance of whatever it is given. A small sample makes the estimate noisy, not the arithmetic wrong.
}

// pp-m3-l3: CAPM Beta in Python: Regression on Market Returns
::pp-m3-l3 Q3::[plain]With a risk-free rate of 3% and a market risk premium of 5%, what expected return does CAPM give a defensive stock with a beta of 0.6?{
	=6.0%\: 3% + 0.6 × 5%#CAPM adds beta times the premium to the risk-free rate\: 3% + 3% \= 6%. A beta below 1 carries less market risk, so it is priced below the market's expected 8%.
	~3.0%\: the risk-free rate#That is the CAPM return for a beta of 0, an asset with no market risk. Any positive beta adds part of the premium.
	~8.0%\: every stock earns the market return in equilibrium#Only a beta of exactly 1 is priced at the market's expected return (3% + 5% \= 8%). A beta of 0.6 earns less.
	~4.8%\: 0.6 × (3% + 5%)#That scales the whole market return by beta and so shrinks the risk-free part too. In the CAPM formula the risk-free rate enters in full for every asset; only the premium is scaled by beta.
}

$CATEGORY: $course$/EcoFinLearn/Data Manipulation/Technical Indicators

// pp-m4-l1: SMA, EMA & RSI in Python
::pp-m4-l1 Q1::[plain]Before your fix, the starter stores the raw RS ratio in the rsi_14 column and the census brands 99 of 100 days "Oversold" - even though the price actually drifted upward. Why?{
	=RS is an unbounded ratio that hovers near 1, so the x < 30 test fires on every day - the 70/30 thresholds are only meaningful on the rescaled 0–100 RSI dial#Exactly. RS lives on 0→∞ and typical values sit around 1. Comparing that against 30 always says "oversold". Wilder's transform 100 − 100/(1+RS) exists precisely so fixed thresholds mean the same thing everywhere.
	~The stock genuinely was oversold for 99 days and the census is correct#The seeded price series drifts upward over the sample - a market that is rising is not "oversold" for 99 straight sessions. The census is a units bug, not a market fact.
	~ewm() needs adjust\=True for the gains and losses, otherwise RSI cannot exceed 30#adjust changes early-sample smoothing details by a little; it does not cap anything at 30. The bug is that the raw ratio was never rescaled onto the 0–100 dial.
	~The random seed produced an unusually bearish sample#With seed 1 the sample actually trends up (the last closes are \~166 vs 150 at the start). No plausible sample makes a ratio near 1 exceed a threshold of 70 or behave like a 0–100 index.
}

// pp-m4-l1: SMA, EMA & RSI in Python
::pp-m4-l1 Q2::[plain]Price jumps sharply after an earnings surprise. Why does the EMA-20 react to this before the SMA-20 does?{
	=EMA weights recent prices more heavily - the newest close gets weight α \= 2/21, while in the SMA it gets the same 1/20 vote as a day three weeks ago#Correct. The EMA's exponentially decaying weights concentrate influence on the most recent observations, so new information shifts it immediately; the SMA dilutes the jump equally across all 20 days.
	~EMA uses fewer data points than SMA, so it has less to average#The recursive EMA effectively uses the entire price history - more points than the SMA's 20, not fewer. What differs is the weighting, not the count.
	~EMA is computed on returns while SMA is computed on prices#Both are computed on the same close prices in this lesson. The difference is purely in how the observations are weighted.
	~SMA is defined with a one-day reporting delay#There is no built-in delay in either definition - rolling(20).mean() at time t includes the close at t. The responsiveness gap comes entirely from the weights.
}

// pp-m4-l1: SMA, EMA & RSI in Python
::pp-m4-l1 Q3::[plain]Per the lesson's reading of the academic literature, what is the current consensus in the Fama-vs-practitioners war over technical signals?{
	=Some signals - notably momentum and short-horizon mean reversion - show statistically significant predictive power, but the edge is small and transaction costs absorb most of it in standard implementations#That is the documented middle ground (Jegadeesh & Titman 1993, Lo & MacKinlay 1990)\: the signals are not random noise, but they are also not free money once you pay to trade them.
	~The EMH was fully vindicated - every technical signal is statistical noise#Momentum and short-horizon mean reversion have survived decades of out-of-sample scrutiny in academic journals - the pure "all noise" position did not hold.
	~Technical analysis reliably beats the market as long as you combine at least two indicators#No such guarantee exists in the literature. Combining indicators does not conjure an edge that survives costs; most documented effects are small and regime-dependent.
	~The debate ended when Fama conceded that markets are inefficient#Fama never conceded that; he shared the 2013 Nobel with Shiller, whose view differs - the prize itself acknowledged the debate rather than settling it.
}

$CATEGORY: $course$/EcoFinLearn/Data Manipulation/Data Cleaning

// pp-m5-l1: Cleaning Messy Financial Data
::pp-m5-l1 Q1::[plain]A vendor feed shows NaN for a stock's close on a quiet day when no trades printed. What is the standard fix for the price column, and why?{
	=Forward-fill\: carry the last traded price forward - a stock that did not trade still has its last price#Exactly the last-price rule. No trade means no NEW price, not no value\: the last print stands until the next one arrives, so ffill() reconstructs what the market itself would say.
	~fillna(0) - no trades means no value that day#A zero price fabricates a −100% return followed by a massive rebound. Zero-fill belongs to volumes and flows, where "nothing happened" truly equals 0 - never to prices.
	~dropna() - keep only real observations#Dropping the row breaks the calendar\: rolling windows shorten and dates misalign across tickers, corrupting every merge downstream. The row structure is worth preserving.
	~Fill with the column mean - the statistically unbiased choice#The column mean is computed from prices that come AFTER the gap - future information (lookahead bias) - and it fabricates a price no market ever quoted.
}

// pp-m5-l1: Cleaning Messy Financial Data
::pp-m5-l1 Q2::[plain]You flag outliers with mean ± 3σ on ten daily closes, one of which is an absurd 9999 print. Nothing gets flagged. What happened?{
	=Masking\: the outlier inflates the very mean and std used to detect it - with 10 points, no value can even reach 3 sample standard deviations#With n points the largest possible z-score is (n−1)/√n ≈ 2.85 for n \= 10, so mean ± 3σ can never flag anything. Rank-based rules like Tukey's IQR fences are immune, because quartiles barely move when one point explodes.
	~The 9999 must be a genuine price, so the rule is right to keep it#A 65× jump on a $155 stock with no news is a data error (misplaced decimal, test tick). The rule missed it for a statistical reason - masking - not because the print is real.
	~pandas computes std incorrectly on small samples#The std is computed correctly. The problem is the method\: the outlier contaminates the statistics used to hunt for it.
	~The data must be sorted by date before computing the std#Mean and standard deviation are order-independent - sorting changes nothing. The failure is masking, not ordering.
}

// pp-m5-l1: Cleaning Messy Financial Data
::pp-m5-l1 Q3::[plain]The 9999 print sits in a series that feeds 20-day rolling averages. Why does the lesson clip it to the fence instead of dropping the row?{
	=Clipping repairs the bad value while preserving the row structure - rolling windows and date alignment stay intact#In time-series work the calendar is sacred\: dropping a row shortens every rolling window that touches it and misaligns dates across tickers. Clipping fixes the value and keeps the structure.
	~Rows can never be dropped once a DataFrame has dates in it#dropna() and boolean filtering work fine on dated frames. Dropping is possible - it is just the wrong tool here because it damages the time-series structure.
	~Clipping is computationally faster than dropping#Both are effectively instant at this scale. The argument is about preserving row structure for time-series operations, not speed.
	~Dropping would change the column mean, but clipping never does#Clipping changes the mean too - that is the point\: it replaces an absurd value with a plausible one. The real advantage is keeping the calendar intact.
}

$CATEGORY: $course$/EcoFinLearn/Data Manipulation/Merging Datasets

// pp-m6-l1: Merging Price & Fundamentals Data
::pp-m6-l1 Q1::[plain]price_df has 5 tickers; fundamentals_df has those same 5 plus GS. How many rows do how\='left' and how\='outer' each return, and why?{
	=5 and 6 - outer adds a GS row with NaN in every price column#A left join is anchored to the left table\: 5 rows in, 5 rows out (when right keys are unique). An outer join takes the union of keys, so the right-only orphan GS gets a row, with NaN wherever price data should be.
	~6 and 6 - merge always returns the union of the keys#Only how\='outer' takes the union. A left join keeps exactly the left table's rows; GS never appears because it has no price row.
	~5 and 5 - GS is dropped either way since it has no price#Keeping key-less orphans is the entire purpose of an outer join\: GS survives with NaN close - which is precisely why the starter's screen misbehaves.
	~5 and 11 - outer stacks the two tables on top of each other#Stacking rows is pd.concat, a different operation. merge aligns rows BY KEY, so the 5 shared tickers each appear once, plus one row for GS\: 6, not 11.
}

// pp-m6-l1: Merging Price & Fundamentals Data
::pp-m6-l1 Q2::[plain]The fundamentals table now holds 4 quarterly rows per ticker (20 rows for your 5 tickers). You left-merge your 5-row price snapshot onto it with on\='ticker'. How many rows come back?{
	=20 - each price row is duplicated once per matching quarterly row#merge produces one output row per key MATCH, not per left row. Each of the 5 price rows finds 4 partners, so 5 × 4 \= 20. This silent row explosion is the classic merge bug - guard with len() checks or validate\='many_to_one'.
	~5 - a left join always returns exactly the rows of the left table#Only when the right-side key is unique. With duplicate keys on the right, every match becomes a row - the left rows are multiplied, silently.
	~100 - the full Cartesian product of 5 × 20 rows#merge is not a blind cross join\: it pairs rows WITHIN each key. One price row × 4 quarters \= 4 rows per ticker, so 20 total - bad enough, but not 100.
	~It raises an error because the join key is duplicated#By default pandas merges duplicated keys without complaint. You must opt in to strictness with validate\='one_to_one' or 'many_to_one' to get an error instead.
}

// pp-m6-l1: Merging Price & Fundamentals Data
::pp-m6-l1 Q3::[plain]GOOGL tops the screen (0.22 / 24.8 ≈ 0.0089) ahead of AMZN (0.35 / 60.3 ≈ 0.0058) even though AMZN grows faster. What is value_score actually rewarding - and which research legitimised signals like it?{
	=Growth per unit of price paid - cheap growth. Fama & French (1992) showed fundamentally cheap stocks earned a persistent premium#Dividing by P/E penalises paying up\: AMZN's 35% growth costs 60× earnings, GOOGL's 22% costs only 25×. The Fama-French value factor (HML) made exactly this family of fundamental signals academically respectable.
	~The fastest EPS growth, as CAPM recommends#Pure growth would rank AMZN first. And CAPM recommends no fundamental signal at all - it claims market beta alone should explain returns, which is what Fama-French refuted.
	~The lowest share price in dollars#The close price never enters the formula. A 1,000-dollar stock can be cheap relative to its earnings; a 5-dollar stock can be wildly expensive.
	~Recent price momentum#No return series is involved - the score uses only fundamentals. Momentum is a different (and philosophically opposite) factor family from value.
}

$CATEGORY: $course$/EcoFinLearn/Data Manipulation/Econometrics for Your Thesis

// pp-m7-l1: How to Interpret a statsmodels OLS Summary
::pp-m7-l1 Q1::[plain]A coefficient has coef \= 0.60 and std err \= 0.25 with 100 residual degrees of freedom. What is its t-statistic, and is it significant at 5%?{
	=t \= 2.40\: yes, it is above the critical value of about 1.98#t \= coef / std err \= 0.60 / 0.25 \= 2.40. The two-sided 5% critical value of a t distribution with 100 df is about 1.98, so the null of a zero coefficient is rejected.
	~t \= 0.15\: no, it is far below 2#That multiplies the two numbers. The t-statistic divides the estimate by its standard error\: 0.60 / 0.25 \= 2.40.
	~t \= 0.42\: no, the standard error is too large#That divides the standard error by the coefficient, upside down. t measures how many standard errors the estimate sits from zero\: 2.40.
	~It cannot be known without the R-squared#The t-test on one coefficient needs only the estimate, its standard error and the degrees of freedom. R-squared describes the fit of the whole model.
}

// pp-m7-l1: How to Interpret a statsmodels OLS Summary
::pp-m7-l1 Q2::[plain]Your summary header reads "R-squared (uncentered)\: 0.893" and the coefficient table has no const row. What happened?{
	=X was passed without sm.add_constant, so the model has no intercept#sm.OLS fits only the columns it receives. Without a column of 1s the line is forced through the origin, statsmodels switches to the uncentered R-squared, and there is no alpha to test.
	~The data contain missing values, which statsmodels silently dropped#Missing values raise an error or are dropped with missing\="drop"; they do not remove the intercept or change the R-squared label.
	~statsmodels always reports an uncentered R-squared for financial data#The label depends only on whether the model has a constant. With sm.add_constant the header reads plain R-squared.
	~The F-statistic was too large to report#The F-statistic is printed in both cases. The missing const row is the tell.
}

// pp-m7-l1: How to Interpret a statsmodels OLS Summary
::pp-m7-l1 Q3::[plain]In a regression with a single regressor, the slope has t \= 4.0. What F-statistic does the summary print?{
	=16.0, because with one regressor F \= t²#The F-test of "all slopes are zero" and the t-test of "this slope is zero" are the same test when there is only one slope, and F equals t squared\: 4.0² \= 16.0. In this lesson 22.369² \= 500.4.
	~4.0, the same as t#F is on a squared scale. With one regressor it equals t², so 16.0.
	~2.0, the square root of t#The relationship runs the other way\: F \= t², not √t.
	~It depends on the intercept's t-statistic too#The overall F-test leaves the intercept out\: it tests the slopes only. With one slope, F \= t² of that slope.
}

// pp-m7-l2: Newey-West (HAC) Standard Errors in Python
::pp-m7-l2 Q1::[plain]You switch a regression from classic OLS standard errors to Newey-West. What happens to the coefficient estimates?{
	=Nothing\: only the standard errors, t-statistics, p-values and confidence intervals change#HAC is a different estimate of the VARIANCE of the coefficients. The coefficients themselves are still the OLS ones\: -2.862 in every row of this lesson's table.
	~They shrink towards zero to account for autocorrelation#That would be a different estimator (such as GLS or a shrinkage method). Newey-West keeps OLS coefficients and only corrects their standard errors.
	~They are re-estimated on non-overlapping data#HAC uses the full sample. Thinning the data to non-overlapping observations is an alternative to HAC, not what HAC does.
	~They change sign whenever the residuals are positively autocorrelated#Autocorrelation affects how precise the estimate is, not its value. The coefficient is unchanged.
}

// pp-m7-l2: Newey-West (HAC) Standard Errors in Python
::pp-m7-l2 Q2::[plain]You regress 6-month overlapping returns, sampled monthly, on a predictor. What is the minimum number of Newey-West lags the overlap calls for?{
	=5, because neighbouring 6-month windows share up to 5 months#An h-period overlapping return is correlated with the next h - 1 observations by construction, so use at least L \= h - 1 \= 5.
	~6, one per month in the window#Observations 6 months apart no longer share any month. The mechanical correlation runs up to h - 1 \= 5 lags.
	~1, since only adjacent windows overlap#Windows 2, 3, 4 and 5 months apart also share months (4, 3, 2 and 1 of them). All of those lags carry correlation.
	~0, because monthly sampling removes the overlap#Sampling monthly is exactly what creates the overlap\: each 6-month window starts one month after the previous one.
}

// pp-m7-l2: Newey-West (HAC) Standard Errors in Python
::pp-m7-l2 Q3::[plain]With L \= 3 Newey-West lags, what Bartlett weight does the lag-2 term receive?{
	=0.50, since w \= 1 - 2 / (3 + 1)#Bartlett weights are w_l \= 1 - l / (L + 1). For L \= 3 they are 0.75, 0.50 and 0.25 for lags 1, 2 and 3.
	~0.33, since w \= 1 - 2 / 3#The denominator is L + 1, not L. That is what keeps the last included lag at a small positive weight rather than zero.
	~1.00, every lag counts fully#Equal weights are exactly what Newey and West avoided\: they can make the estimated variance negative. The Bartlett weights fade linearly.
	~0.67, since w \= 2 / 3#The weight falls as the lag grows. Lag 2 of 3 gets 1 - 2/4 \= 0.50.
}

// pp-m7-l3: ADF Test in Python: Are Prices or Returns Stationary?
::pp-m7-l3 Q1::[plain]An ADF test on a bond yield series returns a statistic of -1.9 and a p-value of 0.34. What is the correct conclusion at 5%?{
	=We cannot reject the null of a unit root\: treat the series as non-stationary#The p-value is above 0.05 and the statistic is less negative than the -2.86 critical value. The data are consistent with a unit root, so work with changes rather than levels, or test further.
	~The series is stationary, because the p-value is above 0.05#That reads the test backwards. The null is the unit root; a large p-value means it is not rejected.
	~The series has been proven to have a unit root#Failing to reject is not proof. The test may simply lack power, which is why careful work pairs ADF with KPSS.
	~The test is invalid because the statistic is negative#ADF statistics are usually negative; rejection requires them to be MORE negative than the critical value.
}

// pp-m7-l3: ADF Test in Python: Are Prices or Returns Stationary?
::pp-m7-l3 Q2::[plain]Why is the 5% critical value of the ADF test about -2.86 rather than the -1.65 of an ordinary one-sided t-test?{
	=Under the unit-root null the statistic does not follow a t distribution; its distribution sits further to the left#Dickey and Fuller showed that when the true series is a random walk, the t-ratio on the lagged level has a non-standard distribution with a fatter left side. Using -1.65 would reject a true unit root far too often.
	~Because the test is two-sided#The ADF test is one-sided (the alternative is γ < 0). The difference comes from the non-standard distribution under the null.
	~Because financial data have fat tails#The critical values come from the random-walk null itself, not from the shape of the shocks.
	~Because statsmodels applies a small-sample correction#Even in very large samples the 5% critical value with a constant is about -2.86. It is a property of the test, not a correction.
}

// pp-m7-l3: ADF Test in Python: Are Prices or Returns Stationary?
::pp-m7-l3 Q3::[plain]You regress the price of one stock on the price of an unrelated stock over ten years and get R² \= 0.71 and t \= 14. What should you suspect first?{
	=A spurious regression between two non-stationary series#Two independent random walks routinely produce large t-statistics and high R² in levels (Granger and Newbold, 1974). Test for unit roots and regress returns on returns, or test for cointegration.
	~A genuine economic link, since t \= 14 is overwhelming#With non-stationary regressors the usual t distribution does not apply, so t \= 14 is not evidence of anything.
	~Heteroskedasticity, which robust standard errors would fix#Robust standard errors do not repair a regression of one random walk on another. The problem is non-stationarity.
	~Too few observations#Ten years of daily data is plenty. More data makes a spurious regression look even more significant, not less.
}

// pp-m7-l4: Event Study in Python: Abnormal Returns and CAR
::pp-m7-l4 Q1::[plain]A stock with an estimated α \= 0.0% and β \= 0.8 returns +3.0% on an announcement day when the market returns +2.5%. What is its abnormal return under the market model?{
	=+1.0%, since 3.0 - (0.0 + 0.8 × 2.5) \= 1.0#Expected return \= α + β × Rm \= 0.8 × 2.5 \= 2.0%. The abnormal return is the actual minus the expected\: 3.0 - 2.0 \= +1.0%.
	~+0.5%, since 3.0 - 2.5 \= 0.5#That is the market-adjusted shortcut, which assumes β \= 1. With β \= 0.8 the stock was expected to rise only 2.0%.
	~+3.0%, the raw return#The raw return mixes the market-wide move with the firm-specific news. The point of the event study is to strip out the part the market explains.
	~-1.0%, because the stock underperformed its beta#The stock did better than the market model predicted (3.0% against 2.0%), so the abnormal return is positive.
}

// pp-m7-l4: Event Study in Python: Abnormal Returns and CAR
::pp-m7-l4 Q2::[plain]With σ_AR \= 2.0% from the estimation window, what is the t-statistic of a CAR of +6.0% over a 4-day window?{
	=1.5, since 6.0 / (2.0 × √4) \= 6.0 / 4.0#Under the null the variance of a sum of L independent abnormal returns is L × σ², so its standard deviation is σ × √L \= 2.0 × 2 \= 4.0. t \= 6.0 / 4.0 \= 1.5\: not significant at 5%.
	~3.0, since 6.0 / 2.0#That treats a 4-day sum as if it were a single day. Summing four days of noise doubles its standard deviation.
	~0.75, since 6.0 / (2.0 × 4)#The standard deviation grows with the square root of the number of days, not linearly\: σ × √4, not σ × 4.
	~12.0, since 6.0 × 2.0#A t-statistic divides the estimate by its standard error; it never multiplies them.
}

// pp-m7-l4: Event Study in Python: Abnormal Returns and CAR
::pp-m7-l4 Q3::[plain]Why does the estimation window end at day -11 rather than at day -1?{
	=So that the event, and any leak just before it, cannot contaminate the model of normal returns#If pre-announcement trading or a leak sits inside the estimation window, the fitted α and σ partly absorb the event, biasing AR towards zero and weakening the test.
	~Because the market model needs exactly 120 observations#Common estimation windows run from about 100 to 250 days. The gap before the event is about contamination, not a magic sample size.
	~Because returns are not available in the days before an announcement#They are available, and they enter the event window. They are kept out of the ESTIMATION window on purpose.
	~To make the t-statistics larger#The goal is an unbiased test, not a larger statistic. A clean estimation window gives honest α, β and σ.
}

$CATEGORY: $course$/EcoFinLearn/Data Manipulation/From Excel to pandas

// pp-m8-l1: VLOOKUP, SUMIFS and Pivot Tables in pandas
::pp-m8-l1 Q1::[plain]Your lookup sheet has 4 tickers; your blotter has 10 trades, 2 of them in a ticker missing from the sheet. How many rows do an inner merge and a left merge return?{
	=8 and 10\: the inner merge drops the 2 unmatched trades, the left merge keeps them with NaN#An inner merge keeps only keys found in both tables. A left merge keeps every row of the left table, which is what VLOOKUP does, and fills the missing sector with NaN.
	~10 and 10\: merge never drops rows#how\="inner" is the default for merge, and it does drop rows without a match. That is the trap in this lesson.
	~4 and 10\: the inner merge returns one row per ticker#merge returns one row per matching pair, not per key. The 8 matched trades all survive the inner merge.
	~10 and 12\: the left merge adds rows for the unmatched tickers#A left merge never adds rows for keys only in the RIGHT table. That would be an outer merge, and it would add the sheet-only ticker, not the blotter-only one.
}

// pp-m8-l1: VLOOKUP, SUMIFS and Pivot Tables in pandas
::pp-m8-l1 Q2::[plain]Which pandas expression is the equivalent of \=SUMIFS(notional, desk, "Macro", sector, "Financials")?{
	=df.loc[(df["desk"] \=\= "Macro") & (df["sector"] \=\= "Financials"), "notional"].sum()#Each condition is a boolean Series; & combines them element by element, and the brackets are required because & binds more tightly than \=\=.
	~df.loc[df["desk"] \=\= "Macro" and df["sector"] \=\= "Financials", "notional"].sum()#Python's and cannot combine two Series and raises "The truth value of a Series is ambiguous". Use & with brackets.
	~df.groupby("desk")["notional"].sum()#That sums per desk over all sectors\: one condition, not two.
	~df["notional"].sum(desk\="Macro", sector\="Financials")#.sum() does not take filter arguments. The filtering happens before, with a mask.
}

// pp-m8-l1: VLOOKUP, SUMIFS and Pivot Tables in pandas
::pp-m8-l1 Q3::[plain]After a left merge, two trades have sector \= NaN. You run pd.pivot_table(df, index\="sector", values\="notional", aggfunc\="sum", margins\=True). What happens to those two trades?{
	=They are dropped from the pivot, so the grand total is short by their notional#pivot_table groups by the index column and, like groupby, drops NaN keys by default. Label them first with fillna, or the total no longer ties to the blotter.
	~They appear in a row labelled NaN#That is what you might expect from Excel's (blank) row, but pandas drops NaN group keys unless told otherwise.
	~pivot_table raises an error on missing keys#It does not complain at all, which is what makes the trap dangerous.
	~They are added to the largest sector#pandas never reassigns values to another group. They simply vanish from the aggregation.
}

$CATEGORY: $course$/EcoFinLearn/Data Manipulation/Risk and Backtests

// pp-m9-l1: Value at Risk in Python: Historical, Parametric, Monte Carlo
::pp-m9-l1 Q1::[plain]A portfolio worth EUR 2m has daily returns with mean 0 and standard deviation 1.5%. What is its one-day 95% parametric VaR?{
	=About EUR 49,350\: 1.645 × 1.5% × 2,000,000#With μ \= 0, VaR \= -z × σ × V \= 1.645 × 0.015 × 2,000,000 \= EUR 49,350.
	~About EUR 58,800\: 1.96 × 1.5% × 2,000,000#1.96 is the two-sided 95% value. VaR is one-sided, so the 95% quantile is 1.645.
	~About EUR 69,780\: 2.326 × 1.5% × 2,000,000#2.326 is the 99% one-sided quantile. At 95% use 1.645.
	~EUR 30,000\: 1.5% × 2,000,000#That is a one-standard-deviation move, which a normal variable exceeds on the downside about 16% of the time, not 5%.
}

// pp-m9-l1: Value at Risk in Python: Historical, Parametric, Monte Carlo
::pp-m9-l1 Q2::[plain]Returns have fat tails. Compared with historical VaR, what does the normal (parametric) VaR typically do at 99%?{
	=It understates the loss, because the normal distribution puts too little probability in the far tail#In this lesson the parametric 99% VaR is EUR 25,801 against a historical EUR 29,285. Fat tails matter most at high confidence levels.
	~It overstates the loss, because the normal distribution is symmetric#Symmetry is not the issue. At 99% the thin normal tail is what makes the parametric number too small.
	~It matches exactly, since both use the same standard deviation#The historical method does not use the standard deviation at all\: it reads the quantile straight from the data.
	~It cannot be computed when returns are not normal#It can always be computed. It is just less accurate when the normality assumption fails.
}

// pp-m9-l1: Value at Risk in Python: Historical, Parametric, Monte Carlo
::pp-m9-l1 Q3::[plain]A one-day 99% VaR is EUR 20,000. Under the square-root-of-time rule (zero mean, independent days), what is the 10-day 99% VaR?{
	=About EUR 63,200\: 20,000 × √10#If daily returns are independent, variance scales with time and volatility with its square root\: 20,000 × 3.162 \= EUR 63,246.
	~EUR 200,000\: 20,000 × 10#That assumes every one of the ten days is a 99% worst day, perfectly correlated. Independent days partly offset.
	~EUR 20,000\: VaR does not depend on the horizon#A longer horizon allows larger cumulative losses, so VaR grows with the horizon.
	~About EUR 6,300\: 20,000 / √10#Dividing would make longer horizons safer than shorter ones. The square root multiplies.
}

// pp-m9-l2: Backtest a Moving-Average Strategy in pandas
::pp-m9-l2 Q1::[plain]Your daily backtest computes strategy \= signal * ret, where signal on day t is based on the close of day t and ret is the return from day t-1 to day t. What is wrong?{
	=Look-ahead bias\: the position earns the return that helped create its own signal; it should be signal.shift(1) * ret#The signal is only known at the close of day t, after ret_t has happened. Holding it from day t+1 means multiplying by the next return, which is what shift(1) arranges.
	~Nothing, as long as transaction costs are included#Costs do not fix timing. A strategy that trades on information from the future stays invalid however much it pays.
	~The returns should be log returns instead of simple returns#The choice of return type changes the numbers slightly but not the bias. The problem is when the position is taken.
	~It should use signal.shift(-1), so the signal and the return line up#shift(-1) pulls tomorrow's signal into today, which adds even more look-ahead.
}

// pp-m9-l2: Backtest a Moving-Average Strategy in pandas
::pp-m9-l2 Q2::[plain]A position goes 0, 1, 1, 0, 1 over five days, with costs of 5 basis points per unit of turnover. How much does the strategy pay in costs?{
	=15 bp\: three changes of position (0 to 1, 1 to 0, 0 to 1) at 5 bp each#Turnover is position.diff().abs()\: 1, 0, 1, 1, so 3 units in total. 3 × 5 bp \= 15 bp.
	~15 bp\: three days in the market at 5 bp each#The number is right but the reasoning is not. Costs are paid on CHANGES of position; holding costs nothing extra here.
	~25 bp\: 5 bp every day#Costs depend on turnover, not on the number of days. Days with no change pay nothing.
	~5 bp\: one round trip#There are three changes of position, each paying 5 bp\: 15 bp.
}

// pp-m9-l2: Backtest a Moving-Average Strategy in pandas
::pp-m9-l2 Q3::[plain]On a price series that is a pure random walk with drift, what should an honest moving-average backtest show relative to buy-and-hold, over a long sample?{
	=No reliable edge\: after costs it should match or trail buy-and-hold on a risk-adjusted basis#A random walk has no predictable component for the rule to exploit. Spending time out of the market forgoes the drift, and costs subtract a little more. Any large edge is a bug or luck.
	~A higher Sharpe ratio, because moving averages filter noise#Filtering noise in the past does not predict the next independent return.
	~Exactly the same returns, since the rule is always in the market#The rule is out of the market whenever the close is below its 50-day average, so its returns differ.
	~A lower drawdown in every sample#In this lesson the honest rule's drawdown (-32.7%) is worse than buy-and-hold's (-18.1%). Whipsaws and costs can deepen drawdowns.
}