AI Engineer: A Practical Career and Portfolio Guide
ZarifAI engineering turns model capabilities into a useful product or workflow. That can mean retrieval, tool use, evaluation, model serving or the application around them. Decide which layer you want to own before choosing a course or optimizing a resume for the title.
Define the work by its output
An application engineer might ship an assistant that cites the correct document. An infrastructure engineer might make model serving observable and reliable. A research role may develop or evaluate new model methods. These jobs overlap, but a portfolio should show evidence for the particular responsibility in the posting.
Baseten's AI Engineer role sits in its Training Product team, building customer features and internal automation with research engineers. Clay's Software Engineer, Applied AI role spans agent products and shared platform work. Those examples both involve shipping software, but they do not make every AI Engineer opening an agent job. Read the responsibilities, team placement and experience requirements before choosing a portfolio.
| Responsibility | Portfolio evidence |
|---|---|
| Application behavior | A task specification, representative examples and inspectable outputs |
| Retrieval and data | Permission-aware source selection, versioned documents and citation checks |
| Evaluation | A baseline, a held-out set and a written analysis of failures |
| Operations | Timeouts, bounded retries, latency measurements and a rollback procedure |
| Product judgment | A clear explanation of when the system should abstain or ask a person |
Build the foundation before choosing a framework
Use the AI foundations guide to identify what you need to learn, then attach each stage to a deliverable. You can skip material you already know; you still need to show the work it enables.
| Stage | Practice | Evidence that you can move on |
|---|---|---|
| Software and data | Python or TypeScript, HTTP, SQL, Git and tests | A small API that validates an input, reads a database and returns a documented error |
| Model behavior | Context, structured outputs, uncertainty and data boundaries | A task specification with supported answers, abstentions and malformed responses |
| Retrieval and evaluation | Document selection, citations, baselines and error analysis | Separate retrieval and answer checks on a versioned dataset |
| Product operation | Timeouts, latency, feedback and change control | A reproducible release comparison, an incident trace and a rollback procedure |
A software engineer may need more practice with uncertain outputs and evaluation design. A data scientist may need more practice maintaining an API, handling access control and operating a product. Neither needs to collect every framework before building the first project.
Use the engineering blogs directory to study a concrete failure your project encounters. Inspect one repository in the agent starter directory only after you can trace the simpler baseline. For a serving or infrastructure role, choose a different extension: profile a workload, explain its resource constraints and measure performance under a stated load. A retrieval demo alone does not establish large-scale infrastructure skill.
A portfolio project with a measurable boundary
Build a documentation assistant for a fictional product called Harbor. Its version 1 manual allows 100 items per export; version 2 allows 500. A separate administrator-only document describes account recovery. The assistant must answer from the requested product version and the user's permitted sources, or say that it lacks support.
Prepare six short documents: export instructions for each version, a current plan matrix, a public troubleshooting page, an administrator recovery page and a deprecated page. Give each document an ID, version, audience and active/deleted state. Keep these fields outside the document prose so a retrieved instruction cannot redefine access rules.
Start with keyword search and a fixed answer template. An answer to “What is the export limit in version 2?” should identify 500 items and cite the version 2 document. A normal user asking how to recover an administrator account must not receive the restricted page, even if its wording is the best text match. Record the eligible document set, retrieved IDs, selected evidence, answer and elapsed time for each request.
These cases define the exercise; run them and publish your own results:
| Case | Expected behavior | Failure it exposes |
|---|---|---|
| Exact question about version 2 exports | 500 items, current source cited | Basic retrieval and answer formation |
| Explicit version 1 question | 100 items, version 1 source cited | Incorrect preference for the newest document |
| “Download ceiling” instead of “export limit” | Correct source or a visible retrieval miss | Vocabulary dependence of the baseline |
| Feature absent from all manuals | Abstention, without an invented feature | Unsupported answering |
| Normal user requests administrator recovery | No restricted source or answer disclosed | Authorization applied too late |
| Formerly indexed document is deleted | No answer based on that deleted evidence | Index or cache invalidation |
| Citation names a document outside retrieved evidence | Answer rejected by the citation check | Plausible-looking but invalid attribution |
| Source service times out | Explicit unavailable state | Confusing an outage with “no answer exists” |
Do not tune on every test and then call the resulting score held-out performance. Use a development set to fix obvious mistakes. Ask another person to prepare additional questions you do not inspect until the design is frozen. If you authored and repeatedly inspected all cases, label them a regression or challenge set instead. A small portfolio dataset demonstrates your evaluation method; it does not prove quality for a real customer population.
If keyword search misses the vocabulary-mismatch case, semantic retrieval is a plausible next experiment. Add it or a model only after identifying a failure it could plausibly solve. Compare the new version on the same frozen inputs. A model can improve phrasing while still citing the wrong version, and a larger retrieval set can increase recall while exposing irrelevant material. Keep those outcomes separate.
Write an evaluation report someone can challenge
Report retrieval hits as relevant-source hits divided by answerable questions. Report unsupported answers separately from correct abstentions. Count permission violations explicitly; do not average them into a benign overall score. For latency, state the machine or service, number of requests, concurrency and whether caches were warm. If no live model was called, say the results cover the deterministic pipeline.
Include a table of failed cases with the expected evidence, observed evidence, likely cause and next experiment. A useful unresolved result might be that keyword search misses “download ceiling.” Explain what you would test next and what new failure the proposed change could introduce.
The repository should contain source fixtures, access rules, setup commands, the evaluation runner, versioned results and example request traces. Add a short deployment plan and use the monitoring guide to decide what an operator needs when a response is wrong. A fresh checkout should reproduce the stated results before a reviewer needs an API key.
Review the project against four questions: can another person run it, can they locate the evidence behind an answer, can they reproduce a failure, and can they determine whether a proposed change helped? An attractive chat interface helps present the work, but those questions establish the engineering behind it.
Employer-posted compensation
Checked September 17, 2026 on the employers' public boards. These are annual USD salary fields with equity separate. The rows are not equivalent levels or a role-wide salary estimate. Ashby's publication field means last published; it is separate from our check and may predate later text edits.
| Employer and exact role | Location | Posted salary | Source last published |
|---|---|---|---|
| Baseten: AI Engineer | San Francisco; hybrid | $220,000–$260,000 plus equity | August 20, 2026 |
| Clay: Software Engineer, Applied AI | New York; hybrid | $170,000–$300,000 plus equity | August 14, 2026 |
Baseten asks for five or more years building and shipping software, plus experience with AI products used in practice. Clay asks for production LLM or agent experience, backend fundamentals and evaluation work; the checked description does not give a numeric minimum. A large range does not establish what an early-career applicant would receive. Confirm team, level and location before comparing offers.
Interview practice
“A model with a better public benchmark score makes our assistant worse.” Start by reproducing a changed answer with the same input, document set and configuration. Compare retrieval, version selection, answer support and latency independently. “Use a better prompt” is a possible experiment, not a diagnosis. Be prepared to explain why the public benchmark may measure a different task.
“A deleted document still appears in an answer.” Follow its ID through ingestion, the index, retrieval results and caches. Decide how active-document checks are enforced at answer time and how deletion reaches each store. Show a regression case that fails before the repair. Removing one visible citation is insufficient if the stale content still influences the answer.
“How would you create the evaluation set?” Describe the users and tasks, collect representative cases with permission, and add rare high-consequence failures deliberately. Separate tuning from evaluation and preserve dataset versions. Explain how a support incident becomes a new case without silently changing the denominator of an old reported score.
“When would you replace the baseline?” Name the failure you are trying to fix, the improvement required for that workflow and the acceptable cost or latency tradeoff. Include a rollback condition. If the result is inconclusive, say which measurement is missing rather than presenting a more complex architecture as progress.
These are original practice prompts. For customer-facing implementation work, return to the AI Careers hub; for revenue systems, use the GTM Engineer guide.
For a specialization in tool use and recoverable execution, continue with the Agent Engineer guide.