Everyone Says SMOTE. I Ran 240 Experiments to Find Out if That's True.

Every ML tutorial handles class imbalance the same way. Dataset imbalanced? Apply SMOTE. Done. Next topic. Nobody tests it. Nobody asks whether SMOTE actually helps or whether it just feels like the responsible thing to do. It's become one of those default moves people make without thinking — like adding dropout to every neural network or scaling features before every model. I got annoyed enough to actually test it. What I built A benchmark. 4 classifiers, 4 sampling strategies, 3 real datasets, 5-fold cross validation on every combination. 240 runs total. Every result stored in PostgreSQL. Every claim tested with Wilcoxon signed-rank and Friedman tests before I wrote it down. Classifiers: Logistic Regression, Random Forest, XGBoost, KNN Sampling strategies: Nothing (baseline), SMOTE, ADASYN, Random Undersampling Datasets: Credit Card Fraud — 284,807 transactions, 0.17% fraud Mammography — 11,183 samples, 2.3% malignant Phoneme — 5,404 samples, 9.1% minority class Three different imbal