pandas Fundamentals: Series, DataFrames, the DatetimeIndex, Selecting and Filtering, and Resampling Daily Bars to Weekly

What you will learn

• What a Series and a DataFrame are and why the DatetimeIndex is the backbone of price analysis.
• The difference between label-based loc and position-based iloc, and how boolean masks filter rows.
• How to build weekly or monthly bars from daily bars with resample and the correct aggregation per column.
• Which pandas defaults changed in version 3.0 and how to write code that behaves the same on 2.2 and 3.x.

Explanation

A Series is a one-dimensional labelled array: a column of closes with a date attached to each value. A DataFrame is a table of Series that share one index: open, high, low, close and volume side by side. The index holds the row labels; for price data it should be a DatetimeIndex, a specialised index of timestamps that enables date slicing, resampling and time-zone handling. Convention: import pandas as pd.

Create a frame with a dictionary of columns and a date index:

idx = pd.bdate_range("2025-03-03", periods=10)

df = pd.DataFrame({"open": [...], "high": [...], "low": [...], "close": [...], "volume": [...]}, index=idx)

bdate_range produces weekdays only and knows nothing about exchange holidays; real data comes from files (lesson 4). Inspect with df.head(), df.info() (types and missing values) and df.describe(). df.index.dtype must show datetime64; if it shows object, the dates are still text.

Selecting. df["close"] returns a Series; df[["open", "close"]] (a list) returns a DataFrame. df.loc[row_labels, columns] selects by label: df.loc["2025-03-05":"2025-03-07", "close"] gives the closes for those three dates, and with loc both ends of a slice are included, unlike Python lists. df.iloc[0:3] selects by integer position (first three rows, end excluded) and df.iloc[-1] is the latest bar.

Filtering. A comparison on a Series produces a Series of booleans, a mask: up = df["close"] > df["open"]. Passing the mask selects the matching rows: df[up]. Combine masks with & (and), | (or) and ~ (not), each condition in its own brackets: df[(df["close"] > df["open"]) & (df["volume"] > 1200)]. up.sum() counts True values, and up.mean() is the fraction of up days.

Adding columns is assignment: df["range"] = df["high"] - df["low"]. Operations are vectorised, applied to whole columns at once, which is why pandas is fast. shift(1) moves values down one row (yesterday's close on today's row) and pct_change() gives percentage changes; both leave NaN (not a number) in the first row.

Resampling. resample groups rows into fixed time buckets. Each column needs its own rule: the weekly open is the first daily open, the high is the maximum, the low the minimum, the close the last, and volume the sum:

rules = {"open": "first", "high": "max", "low": "min", "close": "last", "volume": "sum"}

weekly = df.resample("W-FRI").agg(rules)

The code "W-FRI" labels each week by its Friday. Monthly bars use "ME" (month end). Older tutorials write "M"; pandas 2.2 deprecated that alias in favour of "ME" (likewise "QE" and "YE"), and code written with the new aliases runs on both 2.2 and 3.x.

pandas 3.0 changes worth knowing. Copy-on-Write is now the only mode: a filtered frame is an independent copy, so df[mask]["close"] = 0 never modifies df; write df.loc[mask, "close"] = 0. Text columns now have dtype str rather than object.

Worked example

Hypothetical example: ten daily bars from Monday 3 March to Friday 14 March 2025, prices in euros, no holidays.

Resampling gives two rows. Week ending 7 March: open = first open = 100; high = max(102, 104, 104, 105, 106) = 106; low = min(99, 100, 101, 101, 103) = 99; close = last close = 105; volume = 1,000 + 1,200 + 900 + 1,100 + 1,500 = 5,700. Week ending 14 March: open 105, high 110, low 103, close 107, volume = 1,300 + 1,000 + 1,400 + 1,200 + 1,600 = 6,500.

Filtering: (df["close"] > df["open"]).sum() returns 7, because only 5, 10 and 14 March closed below their open. df.loc["2025-03-05":"2025-03-07", "close"] returns 102, 104, 105.

Common mistake

Practice activity

Using the ten-bar table above and working on paper: (a) write the pandas expression that selects the bars from 10 to 12 March inclusive; (b) state what df.iloc[-3:]["close"].tolist() returns; (c) compute the single monthly bar that df.resample("ME").agg(rules) would produce; (d) explain what goes wrong if someone writes df.resample("ME").last() for all columns.

Key takeaways

• Price data lives in a DataFrame with a DatetimeIndex; check df.index.dtype before anything else.
• loc selects by label and includes the slice end; iloc selects by position and excludes it.
• Boolean masks combined with &, | and ~ replace loops for filtering, and .sum() on a mask counts matches.
• Resample OHLCV with first/max/min/last/sum, use the "W-FRI" and "ME" aliases, and remember pandas 3.0's Copy-on-Write when assigning.

Next step

Lesson 4 brings in real data: reading CSV files and free APIs, understanding adjusted versus unadjusted prices, and handling missing values and time zones.