AI Engineer Interview Guide

An AI engineer interview checks whether you can build software around a model that is sometimes wrong, and prove how often. Few employers publish their loop, so the best preparation guide is the posting itself. Its requirement lines tell you what the rounds will probe.

This guide reads current postings for those signals, groups them into five areas and ends with a practice set. Postings were read on September 26, 2026. Where an employer does not publish its process, nothing here claims to know it. The practice prompts are original, not questions from any employer's loop.

Confirm which job you are interviewing for

AI engineer covers a product engineering job, a customer-facing deployment job and, sometimes, a modeling job. Cognition's Applied AI Engineer sits in Customer Engineering. Brex's AI Engineer, Product says "the majority of your work will be backend." Those two loops will not look alike.

The comparison of AI, ML and software engineer jobs shows how to tell them apart from a posting. For a customer-facing role, the FDE interview guide covers discovery rounds in more depth. The rest of this guide assumes a product-facing role.

Check the rules on AI tools first

Employers differ on whether you may use AI assistants during the process, and it is worth asking before a take-home. Anthropic publishes its own guidance for candidates, last updated July 10, 2025. It asks candidates to complete take-home assessments without Claude "unless we indicate otherwise," and says of live interviews, "This is all you." Other employers allow or expect coding assistants. Do not assume either way.

The five areas a loop tends to probe

These areas come from requirement lines in current postings. They are an inference about what gets tested, not a published loop.

1. Ordinary engineering

Baseten's AI Engineer asks for "strong software engineering fundamentals and the ability to clear a real technical bar, not just prompt well." Brex lists "system design, data modeling, API shape." Expect a coding round that has nothing to do with models.

Practice practical code, not only puzzles: parse and validate a messy JSON payload, call a flaky API with a timeout and bounded retries, write the tests. Treat it as a normal software round, because it is one.

2. System design for a model-powered feature

Expect a prompt shaped like "design an assistant that answers questions from our help center" or "design an agent that files expense reports." A strong walkthrough names each stage, what it stores and how it fails:

Scope. Who asks, what counts as a correct answer, and when the system should decline.
Data. Which documents, who may see each one, how versions and deletions propagate.
Retrieval. Filter by permission and status before ranking. Say what you would try first and why.
Generation. What goes in the context, what output shape is required, how the output is validated.
Actions. If the model can change anything, which calls need approval, and how a retry avoids doing it twice.
Evaluation. The cases you would build first, and which scores you keep separate.
Operation. Latency and cost budgets, traces, a rollback path when a model or prompt change goes wrong.

The interviewer will push on the edges: a restricted document that is the best keyword match, a deleted page still in a cache, a model upgrade that changes answers, a request that costs ten times the average. The RAG lesson and the tool-calling lesson are good rehearsal for stages 3 to 5.

3. Evaluation

This is where AI engineer loops differ most from software loops. Clay's Software Engineer, Applied AI asks the person to "build and run evals that measure whether an agent actually completed the task correctly - not just whether the output looked plausible." Samsara's AI Engineer wants success criteria and evaluation frameworks defined "before you write the first line of code," and test suites that "cover real-world scenarios, edge cases, and known failure modes." Anthropic's Applied AI Engineer lists "evaluation frameworks" and "transcript analysis" among its requirements.

Be ready to explain, with an example:

How you build a case set: representative cases, rare high-cost cases, and a held-back set you did not tune on.
Why format, evidence and correctness are separate scores, and why a timeout stays in the denominator.
When a model judge is appropriate, and how you would check it against human labels first.
How a production incident becomes a new case without silently changing an old score.

The evaluation lesson works through each of these on ten cases.

4. Debugging a bad answer

Expect a scenario: users report wrong answers since last week. A strong answer does not start with the prompt. Reproduce one failure with the same input and configuration. Then check each stage separately. Was the right document eligible? Was it retrieved? Did it reach the model's context? Did the output contradict it, or did validation let a bad citation through? Only then propose a fix, and a regression case that fails before it and passes after.

Clay's posting describes this work directly: "Analyze real failures in production and systematically improve robustness." Traces make it possible, which is why the monitoring guide is worth reading before this round.

5. Product judgment and your project

Ramp's Applied AI Engineer asks for "a track record of working on full-stack AI projects, particularly those involving production use cases of LLMs." Expect a deep dive on something you built. The interviewer wants to hear what you measured, what failed, what you changed and what you chose not to build.

Brex adds a product angle: "Talk to customers and reviewers directly, bring what you learn back into the product, and prioritize what to build next on the team." Prepare to explain one decision where a simpler system, or no model at all, was the better answer.

How to prepare, in order

Classify the posting with the comparison guide. Write down which of the five areas it stresses.
Finish one evaluated project and write its failure report. The how to become an AI engineer guide specifies what it should contain. It will come up in most rounds.
Practice ordinary coding until it feels routine again: HTTP clients, validation, tests, small data transformations.
Rehearse the system design walkthrough above in twenty minutes, aloud, including three failure cases.
Prepare three stories: a feature you shipped that used a model, a failure you diagnosed, and a time you argued for a simpler design.
Save the posting you applied to, with the date. Postings change, and you want to discuss the version you read.

If you come from software engineering, spend more time on areas 3 and 4. If you come from data science, spend more on 1 and 2.

Practice set

These are original practice prompts, not questions from any employer's loop.

Prompt	What a strong answer shows
Design a help-center assistant for a product with free and enterprise tiers.	Scopes the users and the decline case, filters documents by tier before ranking, validates citations, and names the first ten eval cases.
A model upgrade improved a public benchmark but your assistant got worse.	Reproduces changed answers on frozen inputs, compares retrieval, citation and answer scores separately, and says why the benchmark measured a different task.
Build an eval set for a feature that has no labeled data yet.	Writes a labeling policy first, mixes representative and rare cases, keeps a held-back set, and explains how the set grows from incidents.
Your eval score went from 82 to 88 percent. Is the change better?	Asks for the denominator, the case count and which cases flipped in each direction, and treats a few flips on a small set as inconclusive.
The agent sometimes calls the refund tool twice.	Finds the missing idempotency key, separates proposal from execution, adds approval for irreversible calls and a replay test.
Latency doubled after adding retrieval. What do you do?	Measures each stage, separates cold and warm runs, considers fewer or smaller passages and caching, and states the quality tradeoff.
When would you not use a model for this feature?	Names a rule-based alternative, compares error cost and maintenance, and picks the simpler system when it meets the requirement.
Walk us through a project where the model was wrong in a way you did not expect.	Describes the evidence, the root cause by stage, the fix, the regression case and what the eval set was missing.

Answer each one aloud in under three minutes, then check your answer against the right-hand column. If you skipped a failure case or a measurement, answer again.

Questions to ask them

Who owns the eval set, and does a failing eval block a release?
Which model providers do you use, and who decides when to change one?
What did the last person in this role ship, and is it still in production?
Is the work mostly for customers, internal teams or a platform other engineers build on?
How is the posted range structured: base salary, total cash, or across several levels?

The answers tell you whether you will build and measure product features, or spend most of your time on something the title did not say.

Next steps

Follow the AI engineer career path: reading order, employer-posted pay and practice prompts.
Build the portfolio project in the AI engineer career guide.
Compare evaluation tooling in the agent evaluation tools comparison before a take-home.