Skip to main content

Transform

Create new variables, recode, reshape, aggregate.

Create / Modify variables​

Compute Variable​

What it is. Creates a new variable from an expression (e.g., income_log = log(income + 1)).

Options / Parameters

  • Expression — Python/pandas expression using existing variable names.
  • New name — Name of the column to create.
tip

Use np.where(cond, a, b) for conditionals.

Recode​

What it is. Maps existing values to new ones (e.g., 1,2 → 'low'; 3,4 → 'high').

When to use. To collapse categories or convert strings to numeric codes.

Options / Parameters

  • Rules — List of old-value → new-value pairs.
  • Default — Value for unlisted cases.

Standardize (Z-scores)​

What it is. Transform a variable to mean 0 / SD 1, or to min-max [0,1].

When to use. Before PCA, clustering, or whenever variables have very different scales.

Options / Parameters

  • Method — z-score / min-max / robust (median + IQR).

Categorize / Bin​

What it is. Convert numeric into categories (quartiles, percentiles, or custom breaks).

Options / Parameters

  • Quantiles — Quartiles (4), quintiles (5)... each bin gets ~equal n.
  • Equal width — Bins of equal numeric width.
  • Custom breaks — Edges you specify (e.g. 0,18,35,65,120 for age bands).

Create Dummy Variables​

What it is. Expand a categorical variable into multiple 0/1 indicators.

When to use. For regression where predictors must be numeric.

Options / Parameters

  • Drop first — Omits the first category (avoids perfect collinearity).

Lag / Lead​

What it is. Shift values k periods forward (lead) or backward (lag).

When to use. Time series — build lagged predictors for ARIMA, VAR, etc.

Rank​

What it is. Convert values to ranks (1,2,3...).

Options / Parameters

  • Average — Ties get the average of their ranks (default).
  • Min/Max — Ties get the lowest / highest rank in the tie group.
  • Dense — No gaps after ties.
  • Ordinal — First observed gets the lower rank.

Split Column​

What it is. Split a text column into multiple columns (e.g., 'John_Smith' → 'John' | 'Smith').

Options / Parameters

  • Separator — Character to split on ('_', ',', ' ', etc.).

Concatenate Columns​

What it is. Join 2+ columns into one (e.g., 'First' + 'Last' → 'FullName').

Reshape / Aggregate​

Reshape Long​

What it is. Convert wide → long. e.g., columns year1, year2, year3 become one 'year' column + one 'value' column.

When to use. For longitudinal time series, repeated-measures analyses.

Pivot Longer (Complex)​

What it is. Advanced wide → long using a regex to split column names into multiple output columns.

Options / Parameters

  • names_pattern — Regex with groups. e.g., (.)_(.) splits 'year_2020' into 'year' + '2020'.
  • names_to — List of names for the extracted groups.

Reshape Wide​

What it is. Long → wide. Spread one column across several.

Aggregate​

What it is. Group rows and compute summaries (mean/sum/count/...) per group.

When to use. Summaries by country, month, category, etc.

Merge Datasets​

What it is. Combine two datasets by a common key (SQL-like join).

Options / Parameters

  • How — inner / left / right / outer — which keys to keep.
  • On — Column(s) to join on.

Crosstab Counts​

What it is. Produce a counts matrix from two categorical variables.