This repository fine-tunes and evaluates NVIDIA's GR00T-N1.7 vision-language-action model on the SO-101 tabletop manipulation benchmark using an Isaac Lab digital twin, all without requiring a physical robot.
15
No data
0
0
No data
2026-07-12
It provides a complete, simulation-only pipeline for fine-tuning a state-of-the-art VLA model (GR00T-N1.7) on a structured manipulation benchmark, with automatic scoring and detailed setup instructions, enabling reproducible research without hardware.
Fine-tune and evaluate GR00T-N1.7 on SO-101 Bench in an Isaac Lab digital twin — no robot required.
This is the GR00T-N1.7 variant of 5hadytru/so101_bench:
a simulated twin of a language-conditioned SO-101 tabletop manipulation benchmark
(bin / next-to / between / move tasks over 56 household objects), retargeted from
GR00T-N1.6 to N1.7 (Cosmos-Reason2 / Qwen3-VL backbone). See the upstream repo
for the benchmark design and the paper's real-robot findings.
Two separate Python environments are required — their transformers pins conflict:
| Environment | Used for | Key stack |
|---|---|---|
Isaac Lab (~/IsaacLab/isaaclab.sh) | Simulator + evaluator | Isaac Sim 5.1, Isaac Lab 5.1 |
N1.7 LeRobot venv (grootn1.7/lerobot/.venv) | Fine-tuning + policy serving | LeRobot groot_n1_7, transformers 5.x |
1. Install the benchmark extension into the Isaac Lab env (must point at this clone — an editable install from another clone will shadow it with stale modules):
~/IsaacLab/isaaclab.sh -p -m pip install -e source/so101_bench
2. Download the USD assets (~430 MB of room/bin/arm/object meshes; gitignored):
huggingface-cli download 5hadytru/so101_bench_assets so101_bench_usd_assets.tar.gz \
--repo-type dataset --local-dir /tmp/so101_assets
tar -xzf /tmp/so101_assets/so101_bench_usd_assets.tar.gz \
-C source/so101_bench/so101_bench/assets/
3. Set up the N1.7 LeRobot env (fine-tune + serving):
cd ../grootn1.7/lerobot && pip install -e '.[groot,training]'
Prerequisite: accept the gated
nvidia/Cosmos-Reason2-2B
backbone repo with your Hugging Face account, or policy loading dies with a 403.
4. Smoke-test the sim:
~/IsaacLab/isaaclab.sh -p scripts/list_envs.py
A containerized alternative (sim image + policy-server image, wired via compose)
lives under docker/.
Entry point: finetune/finetune_groot_n1.7.sh
(LeRobot lerobot-train, GrootPolicy). Run it inside the N1.7 LeRobot venv — a
preflight guard refuses to run on an N1.5 (Eagle-backbone) LeRobot build.
``
SO-101 Bench: GR00T-N1.7 Edition — Sim-Based VLA Evaluation Without the Robot — leads with the value proposition.
Similar projects matched by category, topics, and programming language.
Harness Engineering is a methodology for improving coding agent outputs by carefully crafting the environment around them—providing curated context, tools, and executable constraints that encode an organization’s nonfunctional requirements and cumulative lessons.
A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.
mimic captures traffic from any iOS or web app and automatically generates a Python client library that lets you call the app's API like a regular library.