- Step 1 of 10 in the AI Engineer path
AI Engineer: A Practical Career and Portfolio Guide
Zarif
Picture a company's help assistant telling a customer the export limit is 100 items. That was true for the old version. The current manual says 500. Nobody notices until the customer complains.
An AI engineer's job is to stop that from happening. They build the software around a model so it finds the right document, answers from it, says "I don't know" when it should, and can prove how often it gets things right.
This guide covers what the job looks like in real postings, what to learn first, and a portfolio project you can build to show you can do it.
What the job actually produces
"AI engineer" covers a few different jobs. One person ships an assistant that cites the correct document. Another makes the servers that run models fast and easy to monitor. A research-leaning role might test new model methods.
They overlap, but a portfolio should show the specific thing the posting asks for.
Two examples. Baseten's AI Engineer role sits in its Training Product team, building customer features and internal automation with research engineers. Clay's Software Engineer, Applied AI role spans agent products and shared platform work.
Both involve shipping software. That doesn't make every AI engineer opening an agent job. Read what the role is responsible for, which team it sits on, and how much experience it asks for before you pick a project.
Here's how each kind of responsibility shows up in a portfolio:
| Responsibility | What to show |
|---|---|
| How the app behaves | A written description of the task, real-looking examples, and outputs anyone can inspect |
| Finding the right data | Users only get documents they're allowed to see, documents have versions, and citations get checked |
| Testing | A starting score, a set of questions you didn't tune on, and a written breakdown of what failed |
| Running it | Timeouts, a cap on retries, speed measurements, and a way to roll back a bad change |
| Product judgment | A clear rule for when the system should decline or hand off to a person |
Learn the basics before picking a framework
Use the AI foundations guide to see what you need to learn. Then tie each stage to something you build. Skip what you already know. You still need to show the work it makes possible.
| Stage | Practice | You're ready to move on when you have |
|---|---|---|
| Software and data | Python or TypeScript, HTTP, SQL, Git and tests | A small API that checks its input, reads a database and returns a clear error |
| How models behave | What the model sees, structured outputs, uncertainty, what data it can reach | A task description that covers good answers, "I don't know" answers and broken responses |
| Finding documents and testing | Picking documents, citations, starting scores, reading failures | Separate checks for "did it find the right document" and "was the answer right," on a dataset with versions |
| Running a product | Timeouts, speed, feedback, controlled changes | A before-and-after comparison anyone can rerun, a log of one incident and a rollback procedure |
People get stuck in different places. A software engineer may need more practice with output that's sometimes wrong, and with designing tests for it. A data scientist may need more practice keeping an API running, handling who can see what, and running a product.
Neither needs to learn every framework before building a first project.
When your project hits a specific failure, look it up in the engineering blogs directory. Open a repo from the agent starter directory only after you can follow your own simpler version end to end.
If you want a serving or infrastructure role, build something different. Profile a workload, explain what limits it, and measure how it performs under a stated load. A document-search demo alone doesn't show you can run infrastructure at scale.
A portfolio project: the Harbor docs assistant
This project is fictional. Harbor is a made-up product. Everything below is a spec for you to build and test yourself.
The setup
Harbor's version 1 manual allows 100 items per export. Version 2 allows 500. There's also an admin-only page about account recovery.
Your assistant has to answer from the product version the user asks about, using only documents that user is allowed to see. If it can't find support for an answer, it says so.
Write six short documents:
- Export instructions for version 1
- Export instructions for version 2
- A current plan comparison
- A public troubleshooting page
- An admin-only recovery page
- An old page that's been deleted
Give each document an ID, a version, an audience and an active-or-deleted status. Keep those labels outside the document text. Otherwise a document could contain a line like "anyone may read this" and change its own access rules.
The first version
Start with plain keyword search and a fixed answer template. Nothing fancy.
Ask it "What is the export limit in version 2?" It should say 500 items and cite the version 2 document.
Now have a regular user ask how to recover an admin account. They must not get the admin page, even if its wording is the closest match.
For every request, record which documents the user was allowed to see, which ones search returned, which evidence was used, the answer, and how long it took.
The eight test cases
These cases define the exercise. Run them and publish your own results.
| Case | Expected behavior | Failure it exposes |
|---|---|---|
| Exact question about version 2 exports | 500 items, current source cited | Basic retrieval and answer formation |
| Explicit version 1 question | 100 items, version 1 source cited | Incorrect preference for the newest document |
| “Download ceiling” instead of “export limit” | Correct source or a visible retrieval miss | Vocabulary dependence of the baseline |
| Feature absent from all manuals | Abstention, without an invented feature | Unsupported answering |
| Normal user requests administrator recovery | No restricted source or answer disclosed | Authorization applied too late |
| Formerly indexed document is deleted | No answer based on that deleted evidence | Index or cache invalidation |
| Citation names a document outside retrieved evidence | Answer rejected by the citation check | Plausible-looking but invalid attribution |
| Source service times out | Explicit unavailable state | Confusing an outage with “no answer exists” |
A few of those are worth spelling out. "Abstention" means the assistant declines instead of guessing. "Authorization applied too late" means the system ranked the admin page first and only then tried to hide it, so it can leak. The timeout case matters because "the search service is down" and "there's no answer" are different messages for a user.
Keep yourself honest
It's easy to fool yourself here. If you tweak the system until all eight cases pass, a perfect score tells you nothing about new questions.
So split the work. Use a practice set to fix obvious mistakes. Ask someone else to write extra questions that you don't look at until you've stopped changing the design. If you wrote and kept rechecking every case yourself, call them a regression set, not a held-out score.
And remember what a small set can prove. It shows your testing method works. It doesn't prove the assistant is good for a real company's customers.
Then improve one thing
Say keyword search misses "download ceiling," because the manual says "export limit." Trying search that matches meaning instead of exact words is a reasonable next step.
Add it, or add a model, only after you've seen a failure it could fix. Then rerun the same saved questions.
Watch for trade-offs. A model can make answers sound better while still citing the wrong version. Pulling more documents can find the right one more often while also dragging in junk. Report those results separately.
Write a test report someone can argue with
Make each number easy to check:
- Search hits: how many answerable questions got the right document, out of all answerable questions.
- Bad answers vs. correct declines: count them separately. A system that says "I don't know" correctly is doing its job.
- Permission leaks: count every one on its own line. Never blend them into an overall score where they can hide.
- Speed: say what machine or service you ran on, how many requests, how many at once, and whether caches were warm.
If no live model was called, say the results only cover the fixed, non-model parts of the pipeline.
Add a table of failed cases: the evidence you expected, what you got, the likely cause, and what you'd test next. An unsolved result is fine. "Keyword search misses 'download ceiling'" is useful, as long as you say what you'd try and what new problem that might cause.
The repo should include the source documents, the access rules, setup commands, the test runner, saved results with versions, and example logs of a few requests. Add a short deployment plan, and use the monitoring guide to decide what someone on call needs when an answer is wrong.
A fresh copy of the repo should reproduce your results before a reviewer needs an API key.
Then check the project against four questions:
- Can another person run it?
- Can they find the evidence behind an answer?
- Can they reproduce a failure?
- Can they tell whether a proposed change helped?
A nice chat interface helps you present the work. Those four questions are what show the engineering.
Employer-posted compensation
Checked September 17, 2026 on the employers' public boards. These are annual USD salary fields with equity separate. The rows are not equivalent levels or a role-wide salary estimate. Ashby's publication field means last published. It is separate from this page's check and may predate later text edits.
| Employer and exact role | Location | Posted salary | Source last published |
|---|---|---|---|
| Baseten: AI Engineer | San Francisco; hybrid | $220,000–$260,000 plus equity | August 20, 2026 |
| Clay: Software Engineer, Applied AI | New York; hybrid | $170,000–$300,000 plus equity | August 14, 2026 |
Baseten asks for five or more years building and shipping software, plus experience with AI products used in practice. Clay asks for production LLM or agent experience, backend fundamentals and evaluation work. The checked description doesn't give a minimum number of years.
A wide range doesn't tell you what an early-career applicant would get. Confirm the team, level and location before comparing offers.
Interview practice
These are original practice prompts, built on the Harbor project.
"A model with a better public benchmark score makes our assistant worse." Start by reproducing one changed answer with the same input, the same documents and the same settings. Then check search, version choice, whether the answer is supported, and speed, one at a time. "Use a better prompt" is something to try, not a diagnosis. Be ready to explain why the benchmark may be measuring a different task.
"A deleted document still shows up in an answer." Follow its ID through every step: loading, the search index, search results and caches. Decide how the system checks "is this still active" at answer time, and how a deletion reaches every place the document is stored. Write a test case that fails before the fix. Hiding one visible citation isn't enough if the stale text still shapes the answer.
"How would you build the test set?" Describe the users and their tasks. Collect typical cases, with permission, and deliberately add rare cases where a mistake is costly. Keep the questions you tune on separate from the ones you score on, and save each version of the set. Explain how a support incident becomes a new case without quietly changing what an old score was measured against.
"When would you replace the simple version?" Name the failure you're trying to fix, how much better it needs to get for this workflow, and how much extra cost or delay is acceptable. Say when you'd roll back. If the result is unclear, say which measurement is missing. Don't present a more complicated design as progress on its own.
Where to go next
- For customer-facing implementation work, go back to the AI Careers hub.
- For sales and revenue systems, read the GTM Engineer guide.
- To specialize in agents that take actions and can recover when a step fails, continue with the Agent Engineer guide.
