Forward Deployed Engineers for Enterprise AI: Why the Model Works

A claims team at an insurance company watches a demo. The AI reads a policy document and answers questions about it perfectly. Everyone's impressed.

Then a claims handler pastes in a real claim. It's a scanned form with a handwritten note, the policy has three amendments, and the answer depends on an exclusion buried deep in the document. The AI gets it wrong, confidently. The handler goes back to doing it by hand.

That story is made up. The gap it shows is the reason this job exists.

Enterprise AI rarely fails because nobody can call a model API. It fails because the output has to become part of a real decision, under real constraints. Someone has to connect the customer's data, define what "good" means for this team, set up permissions and monitoring, decide where a person checks the work, and carry the whole thing from prototype to production.

At AI companies, that someone is often a forward deployed engineer (FDE). They work where what the model can do meets how the customer actually operates, a spot a product team far away can't learn fast enough.

Why AI demos break inside real companies

Normal software does what its code says. An AI system mixes code with model behavior, data, prompts (the instructions given to the model), tools, document search, policies, and human review. The same setup can behave differently once the industry, the inputs, what users expect, or the rules change.

That opens six gaps between a demo and production.

Nobody has defined the problem. "Build a support agent" isn't something you can ship. Which conversations? What actions? How much freedom? Which mistake is unacceptable? What does better support even mean? The request has to become one bounded workflow and one decision.

Nobody has defined "good." Public benchmarks won't tell a claims team whether a model pulls the right exclusions from its documents. The customer and the FDE have to collect real cases, label them, write a grading rubric, set pass marks, and decide when a person reviews.

The data is scattered. Useful context lives in CRM records, documents, data warehouses, ticket systems, permission settings, and people's heads. The system needs secure, reliable connections to all of it, and a clear answer to which source wins when they disagree.

The rules depend on the job. A tool that drafts emails and a tool that approves payments can't share the same controls. Security, privacy, legal, compliance, and the business owner all need evidence they can sign off on.

People don't change how they work. Users don't adopt a model. They adopt a new way of working. The system has to fit how people are rewarded, who gets to decide what, how exceptions get handled, and how managers actually behave.

Feedback never reaches the builders. Failures in the field need to reach the product and research teams in a form they can use: example inputs, test results, logs of what the system did, error types, the effect on the workflow, and how often it happens. "The customer wants better accuracy" isn't something anyone can act on.

What the AI FDE owns

OpenAI's FDE postings describe a role that covers discovery, technical scoping, system design, building across the full stack, production rollout, adoption, and feedback from evals (tests of the model's output) that can change product and model roadmaps. Palantir's Forward Deployed AI Engineer listing stresses building LLM workflows at scale, owning AI strategy and implementation with customers, and feeding what's learned back into its AIP product. Scale AI's posting describes daily technical work with customers and building across the whole stack. All three are linked at the bottom.

Across those postings, the job comes down to eight things:

Turn an AI ambition into a workflow you can measure.
Measure how it works today, and build the tests.
Design the application and how it connects to the customer's systems.
Build the narrowest version that's actually useful in production.
Prove quality, security, reliability, speed, and cost.
Roll it out with users, with people able to step in.
Measure how behavior and operations change.
Use what you saw in the field to improve the product.

The rest of this page walks through that work in order, with the made-up claims team as the example.

Step 1: Find the decision, not the use case

Start with the work. Watch users, look at recent cases, and find where a decision or action is slow, costly, inconsistent, or impossible today.

For each workflow you're considering, write down:

What starts it, and what result it should produce
Who does it, who reviews it, and who owns the decision
Which systems feed it, and how sensitive the data is
How it performs today on time, cost, quality, and volume
The ways it fails, and what each failure costs
The checks and escalation that already exist
Where AI could help, recommend, or act
The business number that justifies the work

Then decide how much freedom the system gets. It can look things up, draft, sort, recommend, act after a person approves, or act on its own within set rules. Don't default to the most freedom. Pick the least freedom that still gets the value.

For the claims team, the decision is "does this exclusion apply to this claim?" The system recommends. A handler approves.

Step 2: Write down what good looks like, as tests

AI quality is a product requirement, so write it down as tests.

Build a test set from real customer cases: normal ones, hard edge cases, cases where policy is touchy, attempts to trick the system, and known failures. Domain experts decide what a good answer or action looks like.

Build the tests in layers.

Fixed checks. Is the output in the right format, with the required fields and citations? Did it stay inside its tool limits, policy rules, and data access?

Scored judgment. Relevance, completeness, sticking to the source, tone, reasoning, or anything else that needs judgment. If a model does the grading, check its grades against human reviewers.

Task results. Did the user close the case faster, make a better call, or avoid redoing work?

Safety and abuse. Prompt injection (text that tricks the model into ignoring its instructions), leaked sensitive data, forbidden actions, attempts to get around the rules, and unsafe tool use.

Running performance. Speed, uptime, cost, retries, and what happens when something fails.

Sort failures by type. A single accuracy number hides whether the system fails on rare, serious cases or just wobbles on low-risk wording.

Write the test plan before you touch prompts or pick an agent framework. Without a stable measure, every change is just a demo opinion.

For the claims team, the test set starts with past claims where an exclusion was disputed, because those are the expensive mistakes.

Step 3: Design the system around the model

The model is one part. A production AI system can also include:

Logins and role-based access
Connections to the customer's data, and search over it
Managed prompts, policies, and settings
A gateway that sends each call to the right model
Limits on which tools it can use and what they can touch
Stored state and audit history
Tests that rerun on every change
Human approval and escalation
Logging, tracing, cost tracking, and incident controls
A fallback for when the model or something it depends on fails

The FDE makes the trade-offs with the customer. A prototype might get by on one model and simple search. Production might need data kept in a certain region, a backup model vendor, fixed validation rules, approvals, or a smaller model to save cost and time.

Design for change, because models, prices, limits, and behavior keep shifting. Keep model access behind one interface, version the prompts and the tests, and keep policy decisions somewhere visible, outside the application code.

Step 4: Build with real constraints

Use real customer data as early as the rules allow. Made-up examples help during development, but they hide problems with data quality, naming, permissions, language, and edge cases.

The FDE keeps the same production habits as any other team:

Version control and code review
Automated tests, including the output tests
Careful handling of secrets and logins
Separate environments for testing and production
Deployments anyone can repeat
Monitoring and a way to roll back
A record of where data and dependencies come from
Thinking through how someone could attack the model and its tools

Fast doesn't mean disposable. Prototypes built for one customer have a habit of becoming systems that customer depends on.

Step 5: Prove it's safe and reliable

Controls should match the risk, not the excitement.

Agree on these up front:

Who may use it, and for what
What data it may read, keep, or create
What actions it may take
Where a person has to review
What evidence gets kept for audit
The test scores each release has to hit
Monitoring and incident response
Who approves a change to the model or the prompts
Vendor and regional limits
An off switch, and a way to do the work by hand

Bring the risk owners in early, during design. A security review that shows up late forces design changes after the team has already committed to the build.

Step 6: Roll out the new way of working

Start with a small group and a known fallback. Train users on what the system can and can't do, not just where the buttons are. Managers need to know when the new workflow applies and how exceptions get handled.

Then watch what people actually do:

Are users checking every output because they don't trust it?
Are they copying results into some unofficial system?
Do they know why the system refused to do something?
Do managers use the output when they make decisions?
Do exceptions get routed somewhere, or vanish into chat?
Does the human review step make things safer, or just slower?

Change the workflow when the evidence says so. The AI can be good while its place in the work is wrong.

Step 7: Bring field evidence back to the product

The FDE brings back structured evidence, not stories.

For the model team: the type of failure with example cases, how test scores changed across model or prompt versions, patterns in a domain or language, safety and refusal behavior, and speed and cost trade-offs.

For the product team: the missing control, API, connector, or screen, how many customers run into it, the current workaround and what it costs, the effect on adoption or results, and a suggested reusable fix.

For the deployment team: friction in security reviews, design patterns that keep repeating, a standard test suite, templates or reference builds, and gaps in the docs.

Someone inside the company then decides where each item goes: the core product, model work, a platform extension, deployment tooling, documentation, or a fix for this one customer. The goal isn't to grant every request. It's to make every future deployment faster and better.

For the claims team, scanned forms with handwriting turn out to be the weak spot. That goes to the product team with the failing examples attached, not as "accuracy could be better."

Where FDEs help most in AI

Type of AI deployment	Why an FDE helps	What gets reused
Knowledge and search	Data permissions, source quality, citations, and user trust are different at every customer	Connectors, access patterns, test sets
Agent workflows	Tools, how much freedom to give, recovering from failure, and policy all need designing per workflow	Tool framework, guardrails, traces, approval patterns
Customer service	Quality depends on policies, real cases, escalation, and whether the team adopts it	Failure categories, integrations, rollout playbook
Document processing	Formats, exceptions, controls, and what happens next all vary	Extraction formats, validation, human review patterns
Decision support	The bar for evidence, calibration, accountability, and audit is high	Test framework, explanation and audit controls
Developer platforms	Value depends on fitting the repository, security, CI, and how engineers work	Reference architecture and policy tooling

For managers: when an FDE is the wrong answer

Don't send FDEs to patch over:

A model or product that can't reach minimum quality on the task
Controls nearly every enterprise needs and the product doesn't have
A low-value use case with no owner and no baseline
Data the customer can't legally or practically hand over
A workflow that should stay manual
A customer expecting unlimited custom development
A standard integration the product or partners already handle

If FDEs keep building the same custom fix, that's a sign the platform needs to change.

An FDE may also come back with evidence that the deployment should stop. That's a good risk decision, not a failed engineer.

For candidates: what to learn

On top of general production engineering and customer discovery, AI FDEs need:

How LLM applications and agents are put together
How to design output tests and run fair experiments
Search, data pipelines, and permissions
What models can't do, and how context, speed, and cost trade off
Reading traces to see what the system actually did
AI security, including defenses against prompt injection
Designing workflows where a person checks the AI's work
Explaining risk to people who aren't engineers
Telling a model problem apart from a data, prompt, product, or workflow problem

The career guide walks through building those skills. The FDE interview guide has a worked AI case to practice on. The FDE playbook covers a whole engagement, from first request to handoff.

Frequently asked questions

Why can't a normal implementation team deploy enterprise AI?

It can, once the product and the way it gets deployed are repeatable. FDEs earn their place while the model's quality, the workflow, the design, the controls, or the product itself are still uncertain, and someone has to do production engineering and learn fast inside the customer's environment.

Should an AI FDE fine-tune models?

Only when the evidence points there. Most failures trace back to a fuzzy problem definition, search, tools, prompts, data, the interface, or the workflow, not the model itself. Diagnose the whole system first, and pick the simplest fix that meets the goal and the risk limits.