How to Become an AI Engineer: A Path With Proof
ZarifAI engineers build software on top of models someone else trained. The job is getting a model's output into a product that behaves predictably, and proving that it does. That means the fastest route in is not a course certificate. It is one project with an evaluation set a reviewer can rerun.
This guide is for people who already write code or work with data. It orders what to learn, points to a runnable lesson for each step, and specifies the project that ties them together. Postings quoted here were read on September 26, 2026.
What employers are asking for
The term took off after swyx's 2023 essay The Rise of the AI Engineer, which described a software engineer who builds with foundation models and APIs rather than training models. Chip Huyen's book AI Engineering (O'Reilly, 2025) draws the same line: it covers building applications on foundation models through prompting, context construction and fine-tuning, where traditional ML engineering centered on feature engineering and model training.
Current postings put that into requirement lines. Baseten's AI Engineer asks for "demonstrated experience building AI or LLM-powered products, agents, or agentic workflows that real users depend on" and "strong software engineering fundamentals and the ability to clear a real technical bar, not just prompt well." Its nice-to-have list includes "experience building evals, agent observability, or tooling for non-deterministic systems."
Read those three phrases together. Employers want a shipped product, ordinary engineering depth, and a way to measure something that does not give the same answer twice. A portfolio should show all three. The AI engineer vs ML engineer vs software engineer comparison sets out how the postings split the work.
What you already have, and the gap
| You come from | You already have | The usual gap |
|---|---|---|
| Backend or full-stack engineering | APIs, databases, tests, deployment, on-call habits | Designing for uncertain output, building an eval set, reading failures statistically instead of as bugs |
| Data science or analytics | Datasets, metrics, sampling, error analysis | Maintaining a service: request handling, auth, timeouts, versioned releases |
| ML engineering | Training, evaluation, experiment tracking | Product scope, latency and cost budgets, shipping behind a user interface |
A software engineer's instinct is to fix a wrong answer like a bug. That works for one case and hides the other fifty. A data scientist's instinct is to measure, which is exactly right, but the measurement has to live inside a service someone can call. Pick the row that fits and spend most of your time on its right-hand column.
The path, in order
Each step has a runnable lesson on this site. They use synthetic data and local fixtures, so none needs a paid API key to reproduce the recorded result.
- Call a model and handle what comes back. The AI APIs lesson sends a request, validates the JSON response and tests authentication failures, rate limits and malformed output. Everything later depends on treating a model call as an unreliable network call.
- Write a contract for the output. Prompt contracts with evaluation fixtures defines three prompts with explicit output shapes and twelve fixtures that should pass or fail. This is where "the answer looks right" becomes a check.
- Retrieve evidence before answering. Build a small RAG pipeline filters by tenant and active status before ranking, validates citations against what was retrieved, and keeps a deliberate synonym miss in its results.
- Score a dataset honestly. Evaluate a ticket agent keeps format, evidence and correctness as separate checks and refuses to drop a timeout from the denominator. Then compare evaluation tools on the same ten cases before adopting one.
- Let the model call a tool, within limits. The tool-calling lesson separates a model's proposal from an authorized action. The MCP lesson exposes the same kind of tool through a standard protocol and tests schema and protocol errors.
- Ship and watch it. Deploying agents to production covers hosting, state, cost controls and rollouts, and monitoring and debugging covers the traces an operator needs when an answer is wrong.
Steps 5 and 6 matter more if you are aiming at agent-heavy roles. Many AI engineer postings stop at retrieval and evaluation. The agent engineer guide and how to become an agent engineer cover the version of the job centered on tools and recoverable execution.
Frameworks are deliberately missing from this list. Learn one after you can explain what it replaces in the plain version. A reviewer who asks "why this framework" wants to hear the failure it solved.
The portfolio project
Build one project, finish it, and document it well. Choose either a retrieval app or a tool-using app. Both need the same evidence.
Option A: an evaluated RAG app
Pick a small corpus you can publish: a public product's documentation, a set of open government policies, or synthetic documents you write. Twenty to fifty documents is enough. Give each one an ID, a version, an audience and an active flag, kept outside the text.
The app answers a question from permitted, current documents and cites them, or says it cannot. The AI engineer career guide specifies a version of this with a fictional product called Harbor and eight test cases, including a stale document and a restricted page. Start there if you want a ready-made spec.
Option B: an evaluated tool-using app
Give a model two or three tools against a fake system you control: look up an order, propose a refund, open a ticket. The model proposes a call, your code validates it, and anything that changes a record waits for approval. The ticket workflow capstone shows the full shape, from signed delivery to stored approval.
What either project must include
| Artifact | What it shows a reviewer |
|---|---|
| A task specification | You know what "correct" means before measuring it, including when the app should abstain |
| An eval set of 30 to 100 cases, versioned | Development cases you tuned on, plus a held-back set someone else wrote or you froze before tuning |
| Separate scores | Retrieval hit rate, answer correctness, citation validity, abstention and permission violations, each with its denominator |
| A baseline and one change | A simple version first, then one improvement measured on the same frozen cases |
| A failure report | The cases that still fail, the likely cause and the next experiment |
| Traces | For a few requests, the input, what was retrieved or called, the output and the time taken |
| A README that reproduces the results | A fresh checkout runs the eval without an API key, using recorded outputs if needed |
The case count is a practical range, not a standard. It is enough to show a method. It does not estimate quality for real users, and the write-up should say so. Hamel Husain's Your AI Product Needs Evals (March 2024) makes the case for building these checks early and looking at the data constantly. It is worth reading before you write your first case.
A strong write-up is short and specific. A made-up example of the right register: "Keyword retrieval missed 4 of 12 paraphrased questions. Adding embeddings fixed 3 and introduced one wrong-version citation" tells a reviewer more than a dashboard screenshot. Report what you measured, on what, and what you did not test.
Show it where hiring managers look
Put the project in a public repository with the README first and the demo second. Link the failure report from your resume, not just the repository. In a screen, a hiring manager has a few minutes, and one table of cases they can read will do more than a live chat demo.
If your current job lets you, do a smaller version of the same thing at work: add an eval set to an existing model feature, or a trace to a flaky one. Postings ask for products "that real users depend on." A modest internal tool with real users counts for more than a polished side project with none. Describe it without sharing anything your employer would not.
Where to apply first
Titles vary. Applied AI Engineer at one company is a product engineer building features. At another, such as OpenAI's Applied AI Engineer, Startups, it is a customer-facing role closer to a solutions or forward deployed engineer. Read the responsibilities before the title, and match your project to the one you want.
Postings also set a real experience bar. Baseten asks for five or more years building and shipping software. If you are earlier than that, a software engineer role on a team that ships model features is a legitimate first step, and the project above still sets you apart.
Next steps
- Follow the AI engineer career path: reading order, employer-posted pay and practice prompts.
- Read the AI engineer interview guide before your first screen. Your project will come up in most rounds.
- If you are switching from a non-technical role, start with the career transition guide and the free foundations guide.