ML fundamentals interview questions
40 ML fundamentals interview questions with answers, from junior to mid level.
Foundations
- What is the difference between a model's parameters and its hyperparameters?
- Why do you keep a test set that the model never sees during training?
- Which of these tasks is a regression problem?
- You have purchase histories for 50,000 customers but no labels, and you want to find groups of similar customers. What kind of learning is this?
- A model scores 99% on its training data and 70% on new data. What is this called?
- A table lists houses with the columns size, rooms, neighbourhood and price. You want to predict the price. Which one is the label?
Core
- Both the training error and the validation error are high and close to each other. What does that suggest?
- Only 0.5% of card payments are fraud. A model that always predicts "not fraud" has what accuracy?
- A screening model looks for a serious disease, and missing a sick patient is much worse than a false alarm. Which metric matters most?
- A spam filter flags 100 emails and 80 of them really are spam. The inbox held 200 spam emails in total. What are precision and recall?
- You standardize every feature using the mean and standard deviation of the whole dataset, and only then split it into train and test. What is wrong?
- Which regularization penalty can push some coefficients of a linear model to exactly zero?
- What does 5-fold cross-validation do?
- You tried 50 hyperparameter combinations and reported the best score on the test set. What is the problem?
- Why does a single, very deep decision tree tend to overfit?
- What helps a random forest generalize better than one deep decision tree?
- Explain the bias-variance trade-off and how you react to each symptom.
- Only 1% of your examples are positive. What is a sensible way to start?
Intermediate
- Positives are very rare (0.1%). Why can ROC-AUC look excellent while the model is poor at finding them?
- A binary classifier has a ROC-AUC of 0.5. What does that mean?
- A model outputs the probability of fraud. You raise the decision threshold from 0.5 to 0.8. What usually happens?
- A churn model has an offline AUC of 0.99 but only about 0.6 in production. One of its features is
days_since_account_closed. What is the most likely cause? - You must forecast next week's demand using three years of weekly data. Which validation scheme fits?
- Each patient has many images in the dataset. Random k-fold gives 0.97 accuracy, but at a new hospital the model scores 0.80. What should you change?
- What happens when you increase the penalty strength
alphain ridge regression? - Which model is most affected if you forget to scale numeric features that have very different ranges?
- How does gradient boosting build its ensemble?
- A random forest ranks a customer ID column among its most important features when using the default impurity-based importance. What is a better check?
- You pick the 20 features most correlated with the label on the full dataset, then run cross-validation on those 20. Why is the score too optimistic?
- What does
class_weight="balanced"do in scikit-learn classifiers such asLogisticRegression? - A model says "fraud, 90% probability", but only 70% of such cases are really fraud. What is this problem called?
- How would you choose evaluation metrics for a fraud detection model?
Advanced
- A shop's app starts attracting a younger audience and the model gets worse, although how age relates to buying has not changed. Which kind of drift is this?
- The labels for your model's predictions arrive about 60 days later. How do you monitor the model in the meantime?
- A feature is computed with SQL for training but re-implemented in the serving code, and the online values differ slightly. What is this called and what is the best fix?
- A new ranking model improves the offline metric, but click-through falls in the online A/B test. What is the most plausible explanation?
- When and why do you use nested cross-validation?
- A missed fraud costs 50 and a false alarm costs 1. If the model's probabilities are well calibrated, at which predicted probability of fraud should you start flagging payments?
- A model had a great cross-validation score but disappoints in production. What do you check?
- A lender trains a default model only on past loans that were approved, then uses it to score all applicants. What is the main risk?