If you're training an AI agent, or evaluating vendors who say they'll help you, you'll run into RL environments within the first conversation. An RL environment is a controlled world where an agent acts and gets scored, and in 2026 frontier labs are spending more on them than on almost any other kind of training input. Read on to find out what an RL environment is, how it trains an agent, the kinds labs use, who builds them, what they cost, and the real-world data every one of them is built from.
What an RL environment is
An RL environment has three parts. A task space, which is the world the agent acts in (a replica of a piece of enterprise software, a codebase, a browser, an operating system, a game). A goal, usually written by a domain expert, that says what the agent is supposed to achieve in that world. And a verifier, which decides whether the agent succeeded and hands back a reward. RL stands for reinforcement learning, the training method where a model improves by acting, getting scored and adjusting, and never needs a worked example.
The verifier is where the engineering lives. A weak verifier gets gamed. The agent finds a way to collect the reward without doing the task, and the model learns to cheat. Reward hacking is the failure mode everyone building in this market is quietly fighting, and it's the first thing to ask any vendor about.
How an RL environment trains an agent
The loop is simple to describe. The agent is dropped into the environment with a goal. It takes actions (clicks, commands, code edits, moves). The verifier scores the result. The model behind the agent updates toward whatever earned reward, and the loop runs again, thousands or millions of times, across many environments. Over enough runs the agent learns to finish tasks nobody walked it through.
That's why environments became the center of agent training. Reading about how to use a CRM teaches a model what a CRM is. Practicing in a working replica of one, with a verifier that checks whether the deal got logged correctly, teaches it to use one.
The kinds of RL environments labs use
| Kind of environment | What the agent practices | What it's built from |
|---|---|---|
| Coding environments over real repositories | Fixing bugs, shipping features, passing tests | Private codebases with their history, tickets and reviews |
| Enterprise software replicas (CRMs, spreadsheets, ticketing systems) | Doing a job inside the tools companies run on | Real workflows: how a ticket, a deal or a project moves through the system |
| Browser and web environments | Navigating sites, filling forms, completing multi-step tasks online | Recordings of people using real websites |
| Desktop and operating system environments | Operating a computer end to end, across applications | Screen recordings and usage records of real software |
| Games and simulated worlds | Planning, navigation, acting under uncertainty | Gameplay footage and first-person video of the physical world |
| Verifiable reasoning tasks (math, logic, data analysis) | Producing an answer that can be checked automatically | Expert-written problems with known solutions |
Most vendors specialize in one or two rows. Most labs buy across several.
RL environments, RL gyms and benchmarks
Three terms get mixed up. An RL environment is one world with a goal and a verifier. An RL gym is a collection of environments behind a common interface, so a training run can move between them; the name comes from the open-source Gym toolkit that standardized the interface, and today it covers community hubs that host thousands of environments. A benchmark is an environment used only to measure a model, the same kind of world held out so the score means something. Vendors sell all three, and a buyer should know which one is on the quote.
Who builds RL environments
Three kinds of company sell them. The human data companies that added environments to their existing feedback and evaluation businesses (Scale AI, Surge AI, Mercor, Turing, Centific). Environment-native startups founded for this market, which tend to go deeper on one environment type and move faster (Mechanize on coding, Fleet AI on enterprise software replicas, HUD on wrapping real software as agent-callable tools, Veris AI and Plato on simulated enterprise and web worlds, Bespoke Labs on open tooling). And the open ecosystems and infrastructure underneath: Prime Intellect's hub of community-built environments with an open verifier library, General Reasoning's community hub, and sandbox providers like Modal and E2B that run the containers everyone else's environments execute in. Our guide to RL environment companies covers each vendor and how to buy in a market this young.
What RL environments cost
The frontier labs are the buyers and the budgets are large. In September 2025, The Information reported that leaders at one major lab had discussed spending more than a billion dollars on RL environments over the following year. Vendors at the top of the market describe contracts that run to seven figures a quarter.
Per unit, the reported figures run from 200 to 2,000 dollars for a single task with a goal and a verifier, up to around 20,000 dollars for an unusually complex software engineering task, around 20,000 dollars for a replica of a website or app interface, and around 300,000 dollars for a high-fidelity replica of a complex product. An environment sold exclusively goes for roughly four to five times the non-exclusive price. Our guide to how to sell RL environments to AI labs covers those figures with their sources and the three ways environments get sold.
What RL environments are built from
An environment teaches an agent to act, and the model inside the agent still has to understand the world it's acting in. That understanding comes from training data, and so does the environment itself. A replica of enterprise software is only as realistic as the real workflows it was modeled on. A coding environment over a repository needs the repository, with its history and its tickets. A browser environment needs recordings of people using real sites. A game or physical-world simulation needs gameplay footage and first-person video of people doing things.
That's the part of the stack the environment wave tends to obscure. Simulation and real-world data are complements. The environment provides practice and verification, and the data provides the ground truth the simulation is built from and the model is grounded in. Labs that skip the data get agents that ace the benchmark and fumble reality. Our guide to gameplay data for world models covers the simulated-world side, our guide to egocentric video data for AI covers the first-person side, and our glossary entry on workflow trajectories covers the unit of work that enterprise environments are modeled on.
What to check before you buy an environment
Three questions sort the market.
- What ships? One vendor's environment is a full software replica with expert-written goals and tested verifiers. Another's is a sandbox with a prompt. Ask for the exact contents.
- How is the verifier tested against reward hacking? A beautiful simulation with a gameable reward teaches your agent to cheat. Ask for the failure cases they've caught.
- What real-world data was it modeled on, and is that data licensed? An environment built from scraped software or unlicensed recordings carries the same rights risk as any other training input, and the questions buyers ask about rights-cleared training data apply here too.
Most environment work is still custom, so treat public offerings as starting points and scope carefully.
Where Troveo fits
Troveo doesn't build RL environments. Troveo licenses the real-world data that environments are built from and that agent models are grounded in (gameplay footage, task demonstrations, tool manipulation, first-person video, and the real workflows of operating companies), sourced from more than 7,000 rights holders and cleared for AI training with documentation per asset. If you're standing up an environment program, that data is what makes the simulation realistic and the model useful outside it. Browse the catalog in Troveo Lens or talk to us about what your agents need to learn from.