Feature Engineering
Why Feature Engineering?
Features are the raw material of ML models. Better features → better performance, often more than choosing a fancier algorithm. Feature engineering is the process of creating, transforming, and selecting features to improve model accuracy.
Handling Missing Values
• Drop rows/columns (if < 5% missing and random) • Mean/Median/Mode imputation (simple, often fine) • Model-based imputation (KNNImputer, IterativeImputer) • Add indicator column: 'feature_was_missing' — lets model learn missingness signal
Encoding Categorical Variables
Label Encoding — Map categories to integers. Only for ordinal data (Small→0, Medium→1, Large→2). One-Hot Encoding — Binary column per category. Good for nominal data with few categories. Target Encoding — Replace category with mean of target. Great for high-cardinality columns. Risk: leakage — use cross-val encoding. Embedding — Neural network learned representations. Best for very high cardinality (user IDs, product IDs).
Feature Engineering Pipeline
Always build preprocessing into a Pipeline to prevent data leakage and make deployment safe.
Finished reading? Mark it complete to earn your XP.