Agent Development Environments: Coding, RL and Runtimes
People mean three things by agent development environment: a coding agent, a place to train one, or a sandbox to run its code. Here's how to tell them apart.
People say "agent development environment" and mean one of three different things.
One is a tool that helps a person write software with an agent. One is a set of tasks and scores used to train or test a model. And one is a safe place for an agent to run code.
Putting all three in one ranked list hides the choice you actually have to make. So this guide splits them apart.
The three meanings at a glance
| Meaning | Primary user | What it provides | A useful success measure |
|---|---|---|---|
| Coding-agent product | Developer shipping a change | Repository context, editing, tools and a review interface | A correct, reviewable change that passes relevant checks |
| RL training or evaluation environment | Researcher improving or measuring an agent | Tasks, observations, actions, reset behavior and scoring | Reliable task outcomes on held-out examples |
| Hosted execution runtime | Engineer running agent workloads | Isolated compute, files, processes and lifecycle controls | Work completes within the required isolation, latency and resource limits |
One project can use all three. A developer might use a coding assistant to build a training task, and that task's practice runs might happen inside hosted sandboxes. They sit at different layers.
1. A coding-agent product helps you build software
A coding agent reads your repository, suggests edits, and may run commands to check its own work. The product around it decides which files it looks at, which commands it's allowed to run, and how you review what it changed. Claude Code's overview is one example of this kind of tool.
Judge it on a real task with clear edges. Fix a bug you can reproduce, add a small feature, or get a failing check to pass. Write down where things started and what "done" means.
Then read the diff, run the checks, and look at anything the tests don't cover. A confident explanation from the agent doesn't prove the change works.
Questions worth asking when you compare them:
- Can you see which tools the agent uses, and limit them?
- Does it follow the patterns your repository already uses?
- When a command fails, can it recover without throwing away unrelated work?
- Can you read the final patch and rerun its checks yourself?
A great coding demo doesn't mean the model was trained on anything like your setup. How the product works and how the model was trained are two separate claims.
2. An RL environment supplies tasks and feedback
In reinforcement learning, a model gets better by trying tasks and getting scored. The environment is the setup for that. It decides what the agent can see, what it's allowed to do, when a try is over, and how the result gets scored.
For a software task, that could mean: start from one specific commit of a repository, let the agent edit files and run tests, and score the final patch against a set of checks.
A few projects supply different pieces of this:
- Verifiers gives you building blocks for environments and evaluations.
- OpenEnv gives you a standard way to talk to environments that run code.
- SWE-Gym supplies software-engineering tasks and the checks that grade them.
These are parts of a training setup. None of them replaces a code editor.
The hard part is usually the task itself. Can you reset it to the same starting point every time? Does the score measure what you actually want? Could the agent get credit by editing a test, finding a leaked answer, or skipping the real requirement?
Keep training tasks and test tasks separate. Where you can, keep the scoring code somewhere the agent can't write to.
The RL environments directory covers five projects and what to ask before using each. You can also use an environment just to test a model, without training it at all.
3. A hosted runtime supplies a place to execute
A runtime is the machine, or sandbox, where the agent's commands actually run. It might give the agent processes, files, network access, snapshots and time limits. E2B's documentation and Modal's sandbox guide describe two examples.
A sandbox doesn't decide which task matters or whether an answer is right. And calling something a "sandbox" doesn't make every setup safe for every job.
Check your specific setup: how it's walled off from other work, what network it can reach, what data is mounted into it, how secrets get in, how it's torn down, and what sticks around afterward.
For coding work, test the unhappy paths. Install dependencies. Run a command that fails. Hit a timeout. Interrupt a session. Then see if you can get back the file you needed.
Time how long it takes to start and to run your actual workload before you pick a provider. Prices and limits change, so check the provider's current docs instead of trusting a comparison table with no sources.
Where agent frameworks fit
A framework coordinates the model calls, the tools, and what the agent remembers between steps. It might plug into a coding tool or a runtime without being either one. If the missing piece is how your app moves from step to step, start with the agent repos and starter examples.
When you describe your setup, name every layer: the model, the framework around it, the tools it's allowed, where it runs, and what tasks you test it on. That makes problems easier to pin down. A timeout is a runtime problem. A misleading score is an environment problem. A bad code change is a coding-tool problem. Each needs a different fix.
Choose the next experiment
- Want to ship a change? Compare coding tools on that change.
- Want a model to behave better? Define a task you can measure, then check how its environment is set up.
- Want tools to run reliably? Test a runtime against your isolation and operations needs.
The ADE glossary entry has the short definition. The Agents and AI Engineering hub shows where this choice fits in the wider build.