Handling Class Imbalance with SMOTE in Python

What Is Class Imbalance?

Class imbalance occurs when the target variable in a classification task contains a significantly higher number of observations in one class than in the others. For example, a fraud-detection dataset might contain 0.3 % fraudulent transactions and 99.7 % legitimate ones. Traditional machine-learning algorithms assume roughly equal class distributions and tend to be biased toward the majority class, leading to poor recall for the minority class.

Enter SMOTE

SMOTE—short for Synthetic Minority Over-sampling Technique—is a resampling method that synthetically generates new minority-class samples instead of simply duplicating existing ones. By interpolating between a minority sample and its k nearest minority neighbors, SMOTE produces realistic but slightly varied data points that help the learner better understand the minority decision region.

How SMOTE Works Under the Hood

1. Select a minority sample.
2. Find its k nearest minority neighbors (often k=5).
3. Randomly choose one neighbor.
4. Synthesize a new sample along the line segment that joins the two samples, using linear interpolation.
5. Repeat until the desired class balance is achieved.

Popular SMOTE Variants

Borderline-SMOTE Focuses on samples near the decision boundary.
SMOTE-Tomek Combines SMOTE with Tomek links for cleaning overlapping examples.
SMOTE-ENN Applies Edited Nearest Neighbors after SMOTE to remove noisy points.
ADASYN Adaptive approach that creates more samples where the class imbalance is most severe.

Why Handling Class Imbalance Is Critical

Ignoring imbalance leads to misleading metrics. A model that predicts “not fraud” 100 % of the time achieves 99.7 % accuracy but 0 % recall on the minority class. In regulated industries—healthcare, finance, cybersecurity—missing rare but crucial events is unacceptable. Techniques like SMOTE directly address this issue, improving recall and F₁ scores without sacrificing too much precision.

Implementing SMOTE in Python

Prerequisites

Install the imbalanced-learn library, which sits on top of scikit-learn:

pip install imbalanced-learn scikit-learn

Basic Example

from imblearn.over_sampling import SMOTE
from collections import Counter
from sklearn.datasets import make_classification

X, y = make_classification(n_classes=2, class_sep=2,
weights=[0.05, 0.95], n_informative=3,
n_redundant=1, flip_y=0, n_features=20,
n_clusters_per_class=1, n_samples=5000,
random_state=42)
print('Original class distribution:', Counter(y))

smote = SMOTE(random_state=42)
X_res, y_res = smote.fit_resample(X, y)
print('Resampled class distribution:', Counter(y_res))

Integrating SMOTE into a scikit-learn Pipeline

from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.metrics import classification_report
from sklearn.ensemble import RandomForestClassifier
from imblearn.pipeline import Pipeline
from imblearn.over_sampling import SMOTE

X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.2,
stratify=y,
random_state=42)
pipe = Pipeline(steps=[('smote', SMOTE(random_state=42)),
('clf', RandomForestClassifier(random_state=42))])

param_grid = {
'clf__n_estimators': [200, 400],
'clf__max_depth': [None, 20]
}

model = GridSearchCV(pipe, param_grid=param_grid, scoring='f1', cv=3)
model.fit(X_train, y_train)
y_pred = model.predict(X_test)
print(classification_report(y_test, y_pred))

Evaluating Model Performance

Always compare metrics before and after resampling. Focus on recall, precision-recall AUC, and the F₁ score rather than accuracy alone. Cross-validate within a pipeline so that SMOTE is applied only to training folds, avoiding data leakage.

Best Practices

1. Resample after train-test split to keep the test set representative.
2. Use pipelines to avoid leakage during cross-validation.
3. Try multiple ratios; sometimes a slight imbalance gives better precision.
4. Couple SMOTE with under-sampling of the majority class to control dataset size.
5. Monitor overfitting; synthetic samples can make minority decision regions unrealistically smooth.

Common Mistakes to Avoid

Applying SMOTE Before Splitting Data

This leaks synthetic minority information into the test set, inflating performance metrics.

Leaving Default Ratios Unchecked

Generating a perfectly balanced dataset may hurt precision—experiment with sampling_strategy.

Ignoring Class Boundary Noise

Synthesizing samples in noisy regions amplifies errors. Consider Borderline-SMOTE or SMOTE-ENN.

Real-World Use Case: Credit-Card Fraud Detection

In production at many fintech companies, SMOTE (or its variants) is applied within model-training pipelines to ensure that rare fraud patterns are captured. Teams monitor the precision-recall curve in live A/B tests to confirm that the uplift in recall justifies any increase in false positives.

Alternatives and When Not to Use SMOTE

If your minority class has categorical, non-ordinal features with few levels, SMOTE may create implausible samples. In these cases, try:

• Targeted under-sampling of the majority class
• Class-weighting in algorithms like XGBoost or LightGBM
• GAN-based data augmentation for complex feature spaces

Conclusion

SMOTE is a powerful, easy-to-implement technique that often yields dramatic improvements in recall for imbalanced datasets. When combined with proper cross-validation and evaluation metrics, it helps data engineers and data scientists build more robust, fair, and production-ready models.