Rules
We announce every change to these rules, with its date, on Discord and by email.
positronic eval run, pinned by its digest. We run that exact image in simulation and in the finale.positronic command line and an HTTP API: submit, follow the status, and read the results.positro/pi05-droid), GR00T N1.7 DROID (positro/gr00t-n17-droid) and MolmoAct2 DROID (positro/molmoact2-droid), a test for your image, a practice eval on 20 public episodes, and the list of the board's episodes.Full rules
You submit a policy as a Docker image. We score it in simulation, and the leaderboard ranks the teams. Stage 1 runs in MolmoSpaces. Stage 2 adds RoboLab. After the close, up to 10 teams run the same image on a real robot, and the real-robot results decide the winners.
Simulation
From stage 1. Allen AI's simulated homes. A simulated Franka arm picks up objects and places them.
From stage 2. Each submission also runs in RoboLab, with a simulated Franka arm.
--transaction-key returns the same submission. A new key spends a submission.In simulation:
In the finale:
These steps describe a stage-1 run.
positronic eval catalog prints every episode that the board's eval runs.molmo.franka_pick_mini, uses the same MolmoSpaces scoring on 20 public episodes. These are not the board's episodes.Your submission
Time and resources
The simulator clock keeps running while your policy computes, as it does for a real robot. We measure the full time of each inference call and add it to the episode. A slow policy acts late and has less time to finish the task. Every episode has a fixed time limit.
| Resource | Limit |
|---|
Thinking time bounds each episode. Start-up is outside it, so start-up has its own limit: the warmup. The limits in this table are published before submissions open. We may change them during the season, and we announce each change on Discord and by email with its date. A container that goes over a memory, image or time limit fails, and the submission counts.
To stay within the compute budget, we may change the number of episodes per submission, the submission limit and the number of teams in the final re-run. We announce each change on Discord and by email with its date.
Fair play
Finale
Prizes
| Prize | Credits | Total |
|---|
Our top priority is a fair and fun competition. We change the rules or the dates only for a reason, for example to close a loophole. We announce each change, with its date, on Discord and by email.