AI Engineer Interview Guide

An AI engineer interview checks one thing over and over: can you build software around a model that is sometimes wrong, and show how often it's wrong?

Few employers publish their interview process. So the best prep guide is the job posting. Its requirement lines tell you what the rounds will dig into.

This guide reads current postings for those lines, groups them into five areas, and ends with practice prompts to answer out loud. Postings were read on September 26, 2026. Where an employer doesn't publish its process, nothing here pretends to know it. The practice prompts are original, not questions from any employer.

Confirm which job you're interviewing for

"AI engineer" can mean a product engineering job, a customer-facing job where you deploy the product at client companies, or sometimes a job training models.

Cognition's Applied AI Engineer sits in Customer Engineering. Brex's AI Engineer, Product says "the majority of your work will be backend." Those two interviews won't look alike.

The comparison of AI, ML and software engineer jobs shows how to tell them apart from the posting. For a customer-facing role, the FDE interview guide covers those rounds in more depth. The rest of this guide assumes a product role.

Check the rules on AI tools first

Employers differ on whether you can use AI assistants during the process. Ask before you start a take-home.

Anthropic publishes its own guidance for candidates, last updated July 10, 2025. It asks candidates to complete take-home assessments without Claude "unless we indicate otherwise," and says of live interviews, "This is all you." Other employers allow or even expect coding assistants. Don't assume either way.

The five areas a loop tends to test

These come from requirement lines in current postings. They're a reasonable guess at what gets tested, not anyone's published loop.

1. Ordinary engineering

Baseten's AI Engineer asks for "strong software engineering fundamentals and the ability to clear a real technical bar, not just prompt well." Brex lists "system design, data modeling, API shape." Expect a coding round that has nothing to do with models.

Practice practical code, not just puzzles. Parse and check a messy JSON payload. Call a flaky API with a timeout and a capped number of retries. Write the tests. Treat it like a normal software round, because it is one.

2. Designing a model-powered feature

Expect a prompt like "design an assistant that answers questions from our help center" or "design an agent that files expense reports." A strong answer walks through each stage, what it stores, and how it breaks:

Scope. Who asks, what counts as a correct answer, and when the system should say "I don't know."
Data. Which documents, who's allowed to see each one, and how edits and deletions reach the system.
Finding documents (retrieval). Remove what this user isn't allowed to see before ranking what's left. Say what you'd try first and why.
Writing the answer. What the model gets to see, what shape the answer must take, and how you check it before it goes out.
Actions. If the model can change anything, which actions need a person to approve them, and how a retry avoids doing the same thing twice.
Testing. The cases you'd write first, and which scores you keep separate.
Running it. Speed and cost limits, logs of each request, and a way to roll back when a model or prompt change goes wrong.

The interviewer will push on the edges. A restricted document that's the best keyword match. A deleted page still sitting in a cache. A model upgrade that changes answers. A request that costs ten times the average.

The RAG lesson and the tool-calling lesson are good practice for stages 3 to 5.

3. Testing the model's work

This is where AI engineer interviews differ most from software interviews. The test sets are usually called evals.

Clay's Software Engineer, Applied AI asks the person to "build and run evals that measure whether an agent actually completed the task correctly - not just whether the output looked plausible." Samsara's AI Engineer wants success criteria and evaluation frameworks defined "before you write the first line of code," and test suites that "cover real-world scenarios, edge cases, and known failure modes." Anthropic's Applied AI Engineer lists "evaluation frameworks" and "transcript analysis" among its requirements.

Be ready to explain each of these with an example:

How you build a test set: typical cases, rare cases where a mistake is expensive, and a set you held back and never tuned on.
Why "is the format right," "does it cite real evidence" and "is it correct" are separate scores. And why a request that timed out still counts against you instead of quietly dropping out of the total.
When it's fine to have a model grade the answers, and how you'd check that grader against human judgment first.
How a production incident becomes a new test case without quietly changing an old score.

The evaluation lesson works through each of these on ten cases.

4. Debugging a bad answer

Expect a scenario. Here's a made-up one to practice on.

Users of a help-center assistant say it's been giving wrong answers since last week. The interviewer asks what you'd do.

A weak answer starts rewriting the prompt. A strong answer starts by reproducing one failure, with the same question and the same settings. Then it checks each stage on its own:

Was the right document allowed for this user?
Did search actually find it?
Did it make it into what the model saw?
Did the model's answer contradict it, or did a bad citation slip past the check?

Say the document was found but never reached the model, because last week someone lowered the number of passages passed along. Now you have a cause. Propose the fix, and a test case that fails before the fix and passes after.

Clay's posting describes this work directly: "Analyze real failures in production and systematically improve robustness." Request logs make it possible, which is why the monitoring guide is worth reading before this round.

5. Product judgment and your project

Ramp's Applied AI Engineer asks for "a track record of working on full-stack AI projects, particularly those involving production use cases of LLMs." Expect a deep dive on something you built. The interviewer wants to hear what you measured, what failed, what you changed, and what you chose not to build.

Brex adds a product angle: "Talk to customers and reviewers directly, bring what you learn back into the product, and prioritize what to build next on the team." Have one story ready where a simpler system, or no model at all, was the better answer.

How to prepare, in order

Sort the posting with the comparison guide. Write down which of the five areas it leans on.
Finish one project you've tested, and write up how it failed. The how to become an AI engineer guide says what it should include. It'll come up in most rounds.
Practice ordinary coding until it feels routine again: HTTP clients, input checks, tests, small data transformations.
Rehearse the design walkthrough above in twenty minutes, out loud, including three ways it breaks.
Prepare three stories: a feature you shipped that used a model, a failure you tracked down, and a time you argued for a simpler design.
Save the posting you applied to, with the date. Postings change, and you want to discuss the version you read.

Coming from software engineering? Spend more time on areas 3 and 4. Coming from data science? Spend more on 1 and 2.

Practice set

These are original practice prompts, not questions from any employer's loop.

Prompt	What a strong answer shows
Design a help-center assistant for a product with free and enterprise tiers.	Defines the users and when to decline, filters documents by tier before ranking, checks citations, and names the first ten test cases.
A model upgrade improved a public benchmark but your assistant got worse.	Reproduces changed answers on the same saved inputs, compares search, citation and answer scores separately, and says why the benchmark measured a different task.
Build a test set for a feature that has no labeled data yet.	Writes down how cases will be labeled first, mixes typical and rare cases, keeps a held-back set, and explains how the set grows from incidents.
Your eval score went from 82 to 88 percent. Is the change better?	Asks how many cases there are and which ones flipped in each direction, and treats a few flips on a small set as inconclusive.
The agent sometimes calls the refund tool twice.	Spots that nothing marks the refund as already done (an idempotency key), separates deciding from doing, adds approval for actions that can't be undone, and adds a test that replays the call.
Latency doubled after adding retrieval. What do you do?	Times each stage, separates first runs from cached runs, considers fewer or smaller passages and caching, and states what it costs in answer quality.
When would you not use a model for this feature?	Names a rule-based alternative, compares the cost of mistakes and upkeep, and picks the simpler system when it does the job.
Walk us through a project where the model was wrong in a way you did not expect.	Describes the evidence, which stage caused it, the fix, the new test case and what the test set was missing.

Answer each one out loud in under three minutes. Then check your answer against the right-hand column. If you skipped a way it breaks or a measurement, answer again.

Questions to ask them

Who owns the test set, and does a failing test block a release?
Which model providers do you use, and who decides when to switch?
What did the last person in this role ship, and is it still in production?
Is the work mostly for customers, internal teams, or a platform other engineers build on?
How is the posted range structured: base salary, total cash, or across several levels?

The answers tell you whether you'll build and measure product features, or spend most of your time on something the title didn't mention.

Next steps

Follow the AI engineer career path: reading order, employer-posted pay and practice prompts.
Build the portfolio project in the AI engineer career guide.
Compare testing tools in the agent evaluation tools comparison before a take-home.