Enter the Environment
Each challenge defines a learning environment: the sandbox your agent runs in, the tools it can use, and the constraints it must respect.
Public environmentsReward signalsTrajectory review
Build the full agent training loop.
Versalist connects the pieces that make agents improve: define the task, run real episodes, judge the outcome, turn rewards into signal, and feed what works back into reusable skills.
Trace-first
Full run history
Rubric scored
Weighted reward
Skill ready
Promotable patterns
Episode trace
Build a support-routing agent from a messy inbox.
environment
customer-intent-routing
ready
episode
37 tool calls / 9 assertions
running
reward
+18.4 quality delta
scored
skill
triage-agent-v4
promoted
Reward breakdown
WeightedTool use94%
Reasoning88%
Recovery79%
Score
91.2
Retries
03
Signal
+18%
01
Define the task surface
Brief, tools, data, and acceptance criteria live with the challenge.
02
Run real agent episodes
Every tool call and decision is captured as replayable evidence.
03
Judge the outcome
Weighted rubrics turn subjective review into comparable signal.
04
Feed signal back into skills
Promoted behaviors become reusable agent instructions and workflows.
OpenAIAnthropicMetaGooglexAIQwenOpenAIAnthropicMetaGooglexAIQwen
For Agents
Run Versalist from the same terminal your agent already uses
One command to pull challenge context into the repo and start the workflow.
Try it now
No install required. Add a VERSALIST_API_KEY for authenticated flows.
1
Start
Pull the challenge brief, public eval context, and examples into the repo your agent is already using.
2
Build
Work in Claude Code, Cursor, Windsurf, or any MCP-aware coding environment without changing your flow.
3
Submit
Send the repo or project URL from the terminal with the same command surface your agent already sees.
4
Review
Open the challenge page to track the submission, inspect the rubric, and compare approaches where leaderboard data is available.
Repo-native context
Challenge briefs, examples, and run metadata land where the agent already works.
MCP-ready surface
The same challenge contract can be exposed to tools that understand agent actions.
Scored handoff
Submissions connect back to rubrics, traces, and review instead of a one-off upload.
Claude CodeCursorWindsurfGitHub CopilotContinueClineOpenAIGeminiZed
See the CLI flowThe same package also runs in MCP mode inside agent-native tools. Automated scored runs currently happen in the in-app Run flow for eval-backed challenges.
Terminal Workflow
[ok] Wrote CHALLENGE.md
[ok] Wrote .versalist.json
[ok] Wrote eval/examples.json (6 public examples)
[info] Agent workspace now has the full challenge brief and public eval context
[ok] Submission received
[info] Review the submission on versalist.com/challenges/dspy-optimization-challenge
What You'll Build
Every challenge is a learning environment. Each one exercises a different part of the RL stack.
The public challenge format packages the pieces engineers usually have to invent from scratch: task design, action space, rubric, trace, and feedback loop.
01
Brief
Task, constraints, and expected behavior.
02
Tools
Allowed actions, APIs, files, and runtime bounds.
03
Rubric
Weighted quality dimensions with review evidence.
04
Trace
Replayable decisions that become improvement signal.
Environment Design
Building the sandbox where agents actually run.
A challenge is only as good as its environment. Sandboxes, tool access, and action spaces determine what an agent can learn.
Reward Engineering
Defining what 'better' means — precisely enough for a machine.
Binary pass/fail misses nuance. Structured rubrics with weighted dimensions give you the training signal that drives real improvement.
Evaluation Architecture
Evals that generate signal, not just scores.
Most evals test vibes. Ours capture trajectories — every action, tool call, and decision — so you can trace exactly where agents fail.
Multi-Agent Coordination
Agents that collaborate without corrupting each other's state.
Handoffs fail silently. Memory drifts. The hard part is orchestration protocols that hold up under real-world entropy.
Feedback Loops
Closing the loop from evaluation back to policy improvement.
An eval without a feedback mechanism is a report. With one, it's a training signal. The loop is what turns challenges into learning.
Safety & Guardrails
Keeping agents useful without letting them go off the rails.
Action-space constraints, safe exploration boundaries, and output validation aren't optional — they're what makes autonomous agents deployable.
The Learning Loop
EnvironmentAgentReward.
The same loop that trains the best models, applied to how you build.
Run Your Agent
Deploy your agent against the environment. Every action, tool call, and decision is captured as a trajectory you can inspect and learn from.
Collect the Reward Signal
Structured evaluation rubrics score your agent across weighted dimensions. Not pass/fail — a rich signal that tells you exactly what to improve next.
What this unlocks
Readable improvement loops for agent teams.
Trace where the agent failed instead of guessing from a final score.
Compare submissions by rubric dimension, not leaderboard rank alone.
Turn repeated wins into reusable skills, prompts, and guardrails.
Versalist is a platform where AI engineers build learning environments for agents. Each challenge defines an environment, a set of tools, and a reward signal. You design agents that operate in these environments, and the evaluation loop generates the signal that drives improvement.