Pandas DataFrame Basics: describe(), head() & dtypes

DataFrames - Your Financial Spreadsheet on Steroids

A DataFrame is the central data structure in quant finance. Think of it as a SQL table in memory - with 1000× faster operations than Excel.

📜 pandas Was Born on a Trading Floor

2008 - AQR Capital Management, New York. Wes McKinney was a 23-year-old quantitative analyst at AQR (founded by Cliff Asness, a former Goldman Sachs researcher). He needed to analyse large financial time-series in Python, but no suitable library existed. Excel crashed on datasets larger than ~1 million rows. R had data.frames but was slow for production systems. MATLAB was expensive and proprietary.

McKinney started writing pandas on weekends and evenings. He released pandas 0.1 in mid-2009 to PyPI, specifically designed for the needs of a quant analyst:

• A date-indexed DataFrame where rows are timestamps and columns are assets - the exact structure of a price matrix
• Time-zone aware datetime indices - critical for comparing assets across global markets
• Built-in resampling (daily → weekly → monthly) and rolling windows - the bread and butter of financial analysis
• NaN handling - because real market data always has gaps

pandas was not designed for general-purpose data science. It was designed for financial time-series. Everything you are about to use was shaped by the practical needs of professional quants.

"I essentially built pandas to scratch my own itch. I was tired of the alternatives." - Wes McKinney, PyCon 2012

Today, pandas is the most downloaded data library in Python, used across every quantitative finance team in the world.

🧠 The mental model: Series glued to a shared index

A DataFrame is not a grid of dumb cells like a spreadsheet. It is a bundle of Series (the columns) welded onto one shared index (the rows) - here, trading dates. That one design decision is why finance adopted it overnight:

• Select a column - df['close'] - and you get a Series that carries the index with it: every price stays stamped with its date.
• Operations are index-aligned: combine two columns and pandas matches them date-by-date, never by accidental row position.
• The index is the timeline, the columns are the fields - a price matrix, exactly the way a trading desk pictures its data.

And the first thing an analyst does with any new dataset is the handshake: df.shape for the dimensions, then df.describe() for the 8-statistic summary (count, mean, std, min, the 25/50/75% quartiles, max). Ten seconds, and you know whether the data is sane - right number of rows, no absurd prices, volatility in a plausible range.

Creating a Price DataFrame

import pandas as pd

data = {
    'date':  pd.date_range('2023-01-01', periods=5, freq='B'),
    'close': [150.0, 152.3, 151.8, 154.2, 153.7],
}
df = pd.DataFrame(data).set_index('date')

Essential Inspection Methods

Selecting rows: .loc vs .iloc

Later lessons slice rows, so learn the two selectors now:

• df.iloc[0] selects by integer position (the first row), like a Python list. df.iloc[-1] is the last row; df.iloc[20::10] takes every 10th row from the 21st.
• df.loc["2023-01-05"] selects by label (the index value, e.g. a date); df.loc[df["close"] > 175] filters by a condition.

Rule of thumb: iloc for where a row sits, loc for what it is called. So df["close"].iloc[-1] is the latest close.

Your Task

The starter code builds a 10-day AAPL price DataFrame and prints its shape - but where the summary statistics belong, it merely peeks at the first rows with head(). Your job is the analyst's handshake: swap that one line for the full 8-statistic summary, rounded to 2 decimals.

Predict before you run: you loaded 10 closing prices ranging from 174.55 to 179.23. Write down what the summary table must show - what is count? What are min and max? And will std be closer to 1.5 or to 15? Now run the starter as-is, make your fix, and check the table against your predictions. That ten-second sanity check is exactly how professionals greet every new dataset.