ML Interview Questions
Q: What is the bias-variance tradeoff?
Model error = Bias² + Variance + Irreducible Noise. Bias: error from wrong assumptions → underfitting. Model too simple. Variance: error from sensitivity to training data → overfitting. Model too complex. Fix high bias: more complex model, more features, reduce regularization. Fix high variance: more data, regularization (L1/L2/dropout), simpler model, cross-validation.
Q: Explain gradient descent and its variants.
Gradient descent minimizes loss by updating weights in the negative gradient direction: w ← w - α∇L. Batch GD: uses full dataset. Accurate but slow. SGD: one sample per update. Fast but noisy. Mini-batch: batch of 32-512 samples. Best of both — used in practice. Adam = momentum + adaptive learning rates. Best default optimizer for deep learning.
Q: How do you handle class imbalance?
Data level: • Oversample minority (SMOTE — Synthetic Minority Oversampling Technique) • Undersample majority (random or Tomek links) • Generate synthetic data Algorithm level: • Class weights in loss function (class_weight='balanced' in sklearn) • Focal Loss — down-weights easy examples • Threshold tuning (instead of default 0.5) Evaluation: never use accuracy. Use F1, AUC-ROC, or PR-AUC.
Q: What is regularization and why do we need it?
Regularization adds a penalty to the loss function to prevent overfitting by discouraging large weights. L1 (Lasso): penalty = λ·Σ|wᵢ|. Drives some weights to exactly zero → feature selection. Sparse models. L2 (Ridge): penalty = λ·Σwᵢ². Shrinks all weights toward zero, keeps all features. Smoother models. Elastic Net: L1 + L2 combined. Best of both. Dropout: randomly zeroes neurons. Neural network specific. Forces redundant representations.
Q: Explain cross-validation. Why is it important?
A single train/test split is unreliable — result depends on the luck of the split. K-Fold CV: 1. Split data into K equal folds 2. Train on K-1 folds, evaluate on the remaining fold 3. Repeat K times (each fold is the val set once) 4. Average K scores Result: much lower variance estimate of true model performance. K=5 or K=10 is standard. Always use StratifiedKFold for classification.
Q: What is the difference between precision and recall?
Precision = TP / (TP + FP) — Of everything I labeled positive, what fraction is actually positive? Measures false alarm rate. Recall = TP / (TP + FN) — Of all actual positives, what fraction did I catch? Measures how much I miss. Precision-Recall tradeoff: increasing one decreases the other. Choose based on the cost of each error: • Medical screening → maximize recall (don't miss cancer) • Spam detection → maximize precision (don't block real emails) • Both matter → use F1 = harmonic mean
Finished reading? Mark it complete to earn your XP.