Graders & RL

Grade rollouts you can actually train on.

An open Grader Store with video-grounded, reward-hack-resistant success detection, reward shaping, and a one-click RL epoch — every result paired with real-world eval.

This is the real moat — and it's harder than reconstruction. LLM-as-physical-grader is promising but trivially reward-hacked. Artha invests disproportionately here: multi-signal graders (geometric + LLM-judge + physics-consistency), adversarial hack-detection, and human-in-the-loop calibration. vs. everyone: reconstruction is becoming commodity; reliable graders are not.

Grader Store · task: load_dishwasher

📐

geometric_success

all 6 items inside rack bounding volume, upright

primary

🧠

vlm_judge

Cosmos-Reason: "is this correctly & safely loaded?"

cross-check

⚖️

physics_consistency

no clipping / interpenetration / impossible forces

anti-hack

🚫

breakage_penalty

no glass shattered, no collision > force threshold

shaping

reward = 1.0·geometric_success + 0.3·vlm_judge.agree − 0.5·(1 − physics_consistency) # kills sim-exploit policies − 0.2·breakage_penalty

Hack detector

⚠ Reward hack flagged
Batch #5510: 12% of "successful" rollouts placed items by clipping through the rack wall (physics_consistency = 0.0). Excluded from training; grader auto-tightened.

Without the physics-consistency term, these would have trained a policy that exploits the simulator — and fails 100% on real hardware. This is the failure mode that kills naïve sim-data startups.

RL loop — train, then verify on real hardware

Sim success per RL epoch · π0.5 fine-tune (RECAP-style)

42%

54%

61%

66%

71%

~74%

epoch 1epoch 6 (proj.)

Real-world eval

65%

real success (RoboArena-style, n=40)

sim success71%

sim-to-real gap−6%

A −6% gap (not −40%) is the whole value proposition. Achieved via physics-faithful worlds + conservative-physics DR + real co-training data flagged in the Robot Store.

Export — what the customer buys

artha dataset.export #5512 \ --format lerobot \ --include sim,real_eval,grader_traces \ --filter physics_consistency=1.0

Artifact	Count
RL-ready trajectories	182,400
Real-eval episodes	40
Grader traces (auditable)	all
Reward-hacks excluded	21,888

The sellable unit: a LeRobot-format dataset + auditable grader traces + a real-eval validation report. Neutral to whose model consumes it — a humanoid, industrial, or brain company buys data & eval, not a competing robot. The "Scale AI / Surge of robot RL."

Mock only — illustrative data. See Artha.md §3 (Pillar 4), §6 differentiation, §7 risks.