ml-001 · Train/validation/test splitting — R

Problem statement

Open this practice in GitHub Codespaces runs in your browser; open this folder under practices/ after it starts

Generate a synthetic imbalanced binary-classification dataset (n = 1500, about 15% positive, 4 informative + 6 noise features, plus one high-cardinality categorical that is pure noise), all in-script with a fixed seed. Then:

  1. Write a stratified 60/20/20 train/validation/test split by hand and verify each split preserves the 15% positive share.
  2. Show that standardizing before vs after splitting (scaler fit on all rows vs on training rows only) leaves a glm logistic regression’s validation accuracy unchanged.
  3. Show that target-encoding the noise categorical with all rows (peeking) makes it look predictive on validation, while a train-only encoding correctly shows no signal. Use AUC alongside accuracy.
  4. Fit an unconstrained rpart tree vs a regularized one, report the train/val/test accuracy of each, then tune maxdepth on validation and evaluate the winner on the test set exactly once.

You know it is correct when the splits partition the rows with preserved class balance, the leaky encoding’s validation AUC is far above 0.5 while the proper one sits near 0.5, and the overfit tree shows a much larger train-val gap than the regularized one.

Learning objectives

After completing this practice you should be able to:

  • explain why held-out data exists and what each of the three splits is for
  • implement a stratified multi-way split from scratch in base R
  • distinguish harmless preprocessing-before-split (feature scaling) from catastrophic leakage (target encoding on pooled labels), and demonstrate the difference empirically
  • read a train/val/test accuracy table and diagnose overfitting from the gap
  • run an honest tuning loop with rpart: select on validation, touch test once

Concepts and definitions

See concepts for the shared background. R-specific notes: an unpenalized glm(..., family = binomial()) is exactly affine-invariant in the features, so the standardize-before-vs-after difference is exactly zero here (in Python, sklearn’s default L2 penalty makes it merely near-zero). For the trees, grow an intentionally overfit rpart with rpart.control(cp = 0, minsplit = 2, minbucket = 1) — the defaults are already regularized, which is easy to forget.

Input / output description

Item Description
Input Nothing external — make_synthetic_data(seed = 42) builds X (1500 x 10 matrix), y (1500 ints, 15% ones), group (1500 ints in 1..400, independent of y) in memory
Output Printed report: split sizes and positive shares; validation accuracy with scaler fit on all vs train rows; validation accuracy + AUC for proper vs leaky target encoding; train/val/test accuracy for the overfit and regularized trees; tuned depth and the single test-set evaluation. run_experiments() also returns all numbers as a named list

Before-coding checklist

My attempt

Work here. Run cells interactively; nothing below overwrites this section.

# Milestone 1: generate the data (X, y, group) with a fixed seed and
# confirm the class imbalance is ~15%.
# Milestone 2: stratified 60/20/20 split by hand; check the three index
# sets are disjoint, cover everything, and preserve the positive share.
# Milestone 3: standardization experiment — scaler fit on ALL rows vs on
# train only; compare glm validation accuracy.
# Milestone 4: target-encoding experiment — encode `group` with all labels
# vs train labels only; compare validation accuracy AND AUC.
# Milestone 5: overfit vs regularized rpart (train/val/test table), then
# tune maxdepth on validation and touch the test set exactly once.

Hint 1

For the stratified split, loop over the classes: sample(which(y == cls)) shuffles that class’s row indices, and slicing off round(0.6 * n_cls) / round(0.2 * n_cls) / the rest gives per-class allocations. Concatenating the per-class slices preserves the class balance in every split automatically.

Hint 2

Write target_encode(group, y, fit_idx) so it only ever reads y[fit_idx]: tapply(y[fit_idx], group[fit_idx], mean) gives the per-group means, index them with as.character(group) to map onto all rows, and replace the resulting NAs (groups unseen in fit_idx) with the global mean. The leak is then a one-argument change: fit_idx = train_idx is proper, fit_idx = seq_along(y) is peeking. Evaluate with a rank-based AUC as well as accuracy — with 15% positives, accuracy hides the leak because predicted probabilities rarely cross 0.5.

Complete solution

The official, tested script lives at solutions/solution.R. Run it with:

Rscript practices/ml-001-train-validation-split/solutions/solution.R

Core code (the script adds a full report and returns every number):

stratified_three_way_split <- function(y, train_frac = 0.6, val_frac = 0.2,
                                       seed = 42) {
  set.seed(seed)
  train_idx <- integer(0); val_idx <- integer(0); test_idx <- integer(0)
  for (cls in sort(unique(y))) {
    idx <- sample(which(y == cls))
    n_cls <- length(idx)
    n_train <- round(train_frac * n_cls)
    n_val <- round(val_frac * n_cls)
    train_idx <- c(train_idx, idx[seq_len(n_train)])
    val_idx <- c(val_idx, idx[n_train + seq_len(n_val)])
    test_idx <- c(test_idx, idx[(n_train + n_val + 1):n_cls])
  }
  list(train = sort(train_idx), val = sort(val_idx), test = sort(test_idx))
}

target_encode <- function(group, y, fit_idx) {
  fit_group <- group[fit_idx]; fit_y <- y[fit_idx]
  means <- tapply(fit_y, fit_group, mean)
  enc <- unname(means[as.character(group)])
  enc[is.na(enc)] <- mean(fit_y)
  as.numeric(enc)
}

# The two encodings differ by ONE argument:
enc_proper <- target_encode(group, y, tr)            # leak-free
enc_leaky <- target_encode(group, y, seq_along(y))   # peeking

Key output (seed 42): splits 900/300/300 all at 15.0% positive; standardize-before vs -after validation accuracy 0.913 vs 0.913 (difference exactly 0.000 — glm is affine-invariant); target encoding on the pure-noise categorical gives validation AUC 0.484 proper vs 0.846 leaky; overfit tree train/val/test = 1.000/0.863/0.857 (gap 0.137) vs regularized 0.931/0.910/0.920 (gap 0.021); tuned maxdepth = 4 scores 0.910 on validation and 0.920 on the once-touched test set.

Line-by-line explanation

  • make_synthetic_data fixes the imbalance exactly (sample(c(rep(1L, 225), rep(0L, 1275)))), then shifts the informative features by outer(y, shifts) with shifts 0.8-1.6. sample.int(400, n, replace = TRUE) draws group without looking at y, which is what makes any apparent predictive power of its encoding provably fake.
  • stratified_three_way_split shuffles within class and slices 60/20/20; idx[n_train + seq_len(n_val)] is the base-R idiom for “the next n_val elements after position n_train”.
  • standardize(fit_x, apply_x) gets its mu/sd from fit_x only, and the double sweep subtracts then divides columnwise.
  • target_encode builds per-group means with tapply, maps them via named-vector indexing (means[as.character(group)]), and fills NA (groups unseen in fit_idx) with the global mean.
  • auc_score is the Mann-Whitney statistic from rank(score) — R’s rank averages ties by default, which is exactly what AUC needs.
  • Experiment 2 fits glm(y ~ enc, family = binomial()) on the encoded feature alone; a no-signal feature reaching validation AUC 0.846 is the whole lesson in one number.
  • The overfit tree uses cp = 0, minsplit = 2, minbucket = 1, maxdepth = 30 — you must disable all three brakes, because rpart’s defaults (cp = 0.01, minsplit = 20) already prune.
  • The tuning loop fits one tree per maxdepth 1..12 (keeping minbucket = 20 fixed), keeps the best validation accuracy, refits, and only then predicts on te — the single test-set touch.
  • The autorun guard on the last line lets the test suite source() the file without triggering main().

Design decisions

  • Base R + rpart only — no caret/tidymodels, so every statistic’s provenance (train rows or all rows) is visible in the code.
  • A pure-noise categorical for the leak demo. Because group is independent of y by construction, any validation lift is proof of leakage.
  • AUC alongside accuracy. With 15% positives, the leaky model’s accuracy barely moves (0.867 vs a 0.850 baseline) while its AUC leaps to 0.846; rank metrics expose leaks that threshold metrics hide.
  • One target_encode with a fit_idx argument rather than two functions: it makes “the leak is just a wrong argument” literal.
  • glm for the scaling experiment precisely because it is unpenalized: the exact-zero difference makes the point sharper than sklearn’s near-zero.
  • xval = 0 in every rpart.control so rpart’s internal cross-validation never runs — we are doing our own validation, and the runs stay fast and deterministic.

Common mistakes

  • Splitting after target-encoding “because the encoder is just feature engineering” — the encoder is part of the model and must be fit on train.
  • Forgetting that rpart’s defaults already regularize: with cp = 0.01 and minsplit = 20 you cannot demonstrate overfitting, so the “overfit” tree needs cp = 0, minsplit = 2, minbucket = 1.
  • Indexing means[group] instead of means[as.character(group)] — the former is positional indexing into the tapply result and silently returns wrong means when some group ids are missing.
  • Reporting the best validation accuracy as the result instead of the once-touched test accuracy.
  • Using accuracy alone to check for leakage on imbalanced data — here the leak moves accuracy by ~2 points but AUC by ~36 points.
  • Comparing factor predictions to integers without converting through as.character first (factor levels are not their labels).

Alternative solution

The rsample/tidymodels idiom handles stratification and guards against preprocessing leaks structurally:

library(rsample)
library(recipes)

set.seed(42)
split1 <- initial_split(df, prop = 0.8, strata = y)   # 80 / 20 test
holdout <- testing(split1)
split2 <- initial_split(training(split1), prop = 0.75, strata = y)
train <- training(split2)                              # 0.75 * 0.8 = 60%
val <- testing(split2)                                 # 20%

rec <- recipe(y ~ ., data = train) |>
  step_normalize(all_numeric_predictors()) |>
  prep(training = train)   # scaler stats come from train only
bake(rec, new_data = val)

Prefer this in real projects — prep()/bake() make it impossible for a step to see held-out rows. The hand-rolled version remains better for learning because nothing is hidden. (For leak-resistant target encoding, the embed package’s step_lencode_glm cross-fits internally.)

Extensions and harder variants

  • Repeat the split with 50 different seeds and plot the distribution of validation accuracy — how much of the train-val gap is split luck?
  • Replace the 60/20/20 split with 5-fold cross-validation for tuning (write the fold loop by hand) and keep the same untouched test set.
  • Make the leak subtler: target-encode with additive smoothing ((sum + m * prior) / (count + m)) and find the smoothing m at which the leaky validation AUC stops being distinguishable from 0.5.
  • Prune the overfit tree with printcp()/prune() at the cp minimizing cross-validated error and compare with the validation-tuned depth.
  • Shrink the positive class to 3% and observe stratification going from nice-to-have to mandatory.

Review questions

Answer out loud before expanding.

glm fits an unpenalized logistic regression, and affine transforms of the features are exactly absorbed by the coefficients and intercept — the fitted probabilities are identical, so accuracy matches to the last digit. sklearn’s LogisticRegression penalizes coefficient norms by default (C = 1.0), and the penalty is not scale-invariant, so scaling changes the solution slightly. Both land on the same conclusion: this is not the leak that hurts.

Restricting to fit_idx is the entire leak control: the per-group means may only read labels the model is allowed to see. The NAs are groups that occur in validation/test but never among the fitting rows — with 400 groups and 900 training rows this happens regularly. They must be filled with the global training mean (a neutral value), not dropped and not computed from the held-out rows themselves.

From the validation labels themselves. With ~4 rows per group, one row’s label moves its group mean by ~0.25, so encoding with all rows bakes each validation row’s own label into its feature value. The glm then reads the answer partially out of the feature. On genuinely new data the signal cannot exist — which the train-only encoding correctly shows (AUC 0.484, i.e., chance).

The unconstrained tree keeps splitting until leaves are pure, so late splits fit noise: with minbucket = 1, single odd rows get their own leaves. Those noise-driven regions misclassify validation rows that fall into them, while the regularized tree’s 20-row leaves average the noise away. The gap is the direct price of variance; the regularized tree gives up 7 points of (meaningless) training accuracy to gain ~5 on validation.

No — it means both numbers are noisy estimates of the same quantity, and with 300 rows each a 1-point difference is well within sampling noise (the standard error of an accuracy near 0.9 on n = 300 is about 1.7 points). Validation is not systematically above test unless you tuned aggressively; here only one hyperparameter was tuned over 12 values, so the selection optimism is small. The protocol’s value shows up when you tune hard, not in any single draw.

Personal reflection

2026-08-10 — first pass. rpart’s silently regularizing defaults cost me time: the “overfit” tree would not overfit until cp, minsplit, and minbucket were all disabled. Worth remembering that demonstrating a pathology can take more code than avoiding it.