Role interview guide
Machine learning engineer interview questions and what to expect
Machine learning engineer interviews usually combine software engineering (coding and system design) with machine learning: how models are trained, evaluated, deployed and monitored. Candidates commonly report an ML system design round, questions on deep learning or classical models depending on the team, and for newer AI roles, questions on building with large language models, retrieval and evaluation.
What candidates commonly report; it varies by employer, level and country. Last reviewed 18 October 2026.
The usual stages
- 1Recruiter or HR screen
- 2Coding interview, often similar to a software engineer's
- 3Machine learning fundamentals: models, training, evaluation and the maths behind them
- 4ML system design: a recommendation, search or classification system end to end
- 5Project deep dive on a model you trained or shipped
- 6Hiring manager or behavioural round
What interviewers check
- Sound ML basics: loss functions, optimisation, regularisation and evaluation
- Engineering skill: clean code, data pipelines, serving and testing
- Thinking about the whole system: data, training, serving, monitoring and retraining
- Trade-offs between accuracy, latency and cost
- Rigorous evaluation, both offline and with live users
- Honesty about what a model can and can't do
Behavioral questions
Tell me about a model you took from experiment to production. What was harder than you expected?
A good answer covers
- The problem, the data and the model you chose
- What changed on the way to production: latency, data mismatches, scale
- How it performs now, and how you know
Describe an experiment you believed in that didn't work. What did you do next?
A good answer covers
- The hypothesis and why it looked promising
- How you found out it didn't work, and whether you ruled out a bug first
- What you learned and what you tried instead
Tell me about a time you had to explain a model's limits to a product team that expected more.
A good answer covers
- What they expected and what the model could actually do
- How you showed it with examples or numbers
- What you agreed: a narrower scope, a fallback, more data
Tell me about a time you found a bug in a training or data pipeline.
A good answer covers
- How it showed up: metrics too good, too bad or inconsistent between runs
- How you traced it
- The fix, and the check you added so it couldn't return
Questions about the job
Walk me through how gradient descent trains a neural network.
A good answer covers
- A loss function, gradients from backpropagation, and updating the weights
- Learning rate, batches and optimisers such as Adam
- Signs of trouble: loss not falling, exploding or vanishing gradients
How would you design a system that recommends products or videos to users?
A good answer covers
- Candidate generation, then ranking, and the signals each step uses
- Offline metrics, then an online A/B test
- New users and new items, freshness, feedback loops, and serving latency
How do you serve a model with low latency at high traffic?
A good answer covers
- Batch versus real-time predictions, and which the product needs
- Smaller or quantised models, caching, and batching requests
- Monitoring latency, and a fallback if the model is slow or down
A model performs well offline but worse once it's live. What could explain it?
A good answer covers
- Features computed differently in training and in serving
- Training data that included information not known at prediction time
- Shifts in the data or feedback effects, and how you'd check each
What would you monitor for a model in production?
A good answer covers
- Input data distributions and missing features
- Prediction distributions and, once labels arrive, live accuracy
- Latency, errors and cost, with alerts and a plan for retraining
Explain regularisation. How do L1, L2 and dropout differ?
A good answer covers
- Penalising complexity to reduce overfitting
- L1 pushes some weights to exactly zero; L2 shrinks all weights
- Dropout in neural networks, and when each helps
How do transformers handle sequences, and why did they replace recurrent networks for most language tasks?
A good answer covers
- Attention, letting each token look directly at every other token
- Training in parallel instead of step by step
- The cost: compute and memory grow quickly with sequence length
How would you build an assistant that answers questions from a company's internal documents using a large language model?
A good answer covers
- Retrieval: splitting documents, embeddings, a vector index and ranking
- Putting the retrieved text in the prompt, citing sources, and saying 'I don't know' when unsure
- Evaluating answers, and making sure people only see documents they're allowed to
How would you evaluate a generative model's outputs when there's no single right answer?
A good answer covers
- A test set of realistic cases with clear criteria for a good answer
- Human review, automated checks, and model-based grading used with care
- Tracking failures such as made-up facts or unsafe output across versions
How do you make training experiments reproducible?
A good answer covers
- Versioning code, data and configuration together
- Fixed random seeds and recorded environments
- Experiment tracking and saved model artefacts
You have very little labelled data for a new classification task. What are your options?
A good answer covers
- Fine-tuning a pre-trained model, or transfer learning
- Labelling more data cleverly: active learning, weak labels
- Data augmentation, and a simple baseline to compare against
Motivation and fit
Why machine learning engineering rather than research or data science?
A good answer covers
- Enjoying building models that run reliably for real users
- An example from your work or projects that shows it
- The balance of engineering and modelling you want
Which recent development in AI interests you most, and how might it be useful in our product?
A good answer covers
- A specific development, explained in plain words
- A realistic use in the company's product
- Its risks, and how you'd test it before relying on it
Tips
- Prepare to code like a software engineer; many ML loops include a standard coding round.
- Structure ML design answers: problem and metric, data, model, serving, monitoring. Saying the structure first helps the interviewer follow.
- Know the maths behind what you claim; be ready to explain backpropagation or a loss function from scratch.
- For each project, know the baseline, the metric and the gain over the baseline; a bare accuracy figure invites hard follow-ups.
- If you've built with large language models, be ready to say how you evaluated the results, not just which models you used.
Practise it out loud
A mock interview asks what machine learning engineer interviews usually test, follows up on your answers and gives you a report. The warm-up is free, needs no account and checks five of these questions as you answer.
About this guide
Written by Orbit from what candidates and interviewers commonly describe for this kind of job. The questions are Orbit's own practice questions, not questions taken from any employer's interviews. Processes change, so check the job post and what your recruiter tells you.