What are RL environments for LLM training?
An RL environment for LLM training is a packaged task world: a task specification, the working state and documents the model interacts with, and a programmatic verifier that scores each attempt. Reinforcement learning then optimises the model against that score.
- Three parts
- A task, a realistic environment state, a programmatic verifier
- Reward
- Computed by the verifier, not inferred from human preference
- Scarce input
- Verifiable tasks drawn from work that actually happened
The three components
- Task specification. What the model is asked to accomplish, the starting state, and the materials available, ideally drawn from work that actually happened, not invented scenarios.
- Working state. The documents, records and context the task plays out in. Realism here decides whether the trained behaviour transfers to real deployments.
- Verifier. A programmatic check that decides whether an attempt succeeded, and to what degree. This is the reward signal. Without a reliable verifier there is no reinforcement learning, only generation.
Why labs shifted from corpora to environments
Pre-training on static text teaches a model what work looks like. It does not teach the model to do the work, because there is no feedback loop. The current generation of reasoning models is trained heavily with reinforcement learning on verifiable tasks: the model attempts a task many times, the verifier grades each attempt, and the model updates toward strategies that succeed.
That changes what training data means. The bottleneck is no longer tokens; it is verifiable task instances. Maths and code came first because their verifiers are trivial: a proof checks or it doesn't, a test suite passes or it doesn't. The frontier now is everything else: the document-heavy, judgment-heavy work that fills real operations.
What makes an RL environment good
- Grounded in real work. Synthetic tasks drift toward what is easy to generate. Tasks reconstructed from genuine workflows carry the irregularity and edge cases that make trained behaviour robust.
- Truly held out. If the underlying material ever appeared on the open web, it is inside the model already, and both training signal and evaluation are contaminated.
- Reliably verifiable. The verifier must agree with a competent human on what counts as success. A noisy verifier trains noisy behaviour.
- Rights-cleared. An environment built on data nobody had the right to use is a liability, whatever its quality.
How TGDC builds them
The General Data Company builds RL environments from consented, rights-cleared enterprise sources (insolvency document estates, reenacted expert workflows, and contributed personal documents), anonymised and verified per batch. Because the source material never appeared online, the tasks are genuinely unseen. Each task ships with its own programmatic verifier and a held-out evaluation split, so a lab can measure genuine capability gains rather than memorisation. See RL environments for what a delivery contains, or data licensing for the underlying corpora.