Machine Learning Engineer Mock Interview

Describe a time you improved the performance of an ML model in production—what was the hypothesis, how did you measure success, and what trade-offs (latency, cost, accuracy, robustness) did you make?

How would you design an end-to-end ML pipeline for a supervised classification problem, including data ingestion, feature engineering, training/validation strategy, model monitoring, and retraining triggers?

A model’s offline metrics look great, but online performance drops after deployment. Walk me through your investigation steps and how you would determine whether the issue is data drift, label delay, feature leakage, or something else.

Explain how you would handle class imbalance and calibration for a fraud/spam detection use case; include what metrics you’d optimize (e.g., PR-AUC vs ROC-AUC), and how you’d validate that probability outputs are well-calibrated.

You’re given a small dataset and a strong baseline model; you need to deliver improvements within two weeks. What approach would you take (e.g., transfer learning, augmentation, ensembling, Bayesian optimization), and how would you decide which direction is most likely to work?