Skip to content

Commit d583316

Browse files
pkrackpkrack
committed
Add text content
1 parent 645020b commit d583316

File tree

1 file changed

+74
-32
lines changed

1 file changed

+74
-32
lines changed

‎index.html‎

Lines changed: 74 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -3,21 +3,21 @@
33

44
<head>
55
<meta charset="utf-8">
6-
<!-- Meta tags for social media banners, these should be filled in appropriatly as they are your "business card" -->
6+
<!-- Meta tags for social media banners, these should be filled in appropriately as they are your "business card" -->
77
<!-- Replace the content tag with appropriate information -->
88
<meta name="description" content="A Lean Ecosystem for Robot Learning at Scale">
99
<meta property="og:title" content="Robot Control Stack: A Lean Ecosystem for Robot Learning at Scale" />
1010
<meta property="og:description" content="A Lean Ecosystem for Robot Learning at Scale" />
1111
<meta property="og:url" content="https://robotcontrolstack.github.io/" />
12-
<!-- Path to banner image, should be in the path listed below. Optimal dimenssions are 1200X630-->
12+
<!-- Path to banner image, should be in the path listed below. Optimal dimensions are 1200X630-->
1313
<meta property="og:image" content="static/images/rcs_eye_candy_final.jpg" />
1414
<meta property="og:image:width" content="1500" />
1515
<meta property="og:image:height" content="1500" />
1616

1717

1818
<!-- <meta name="twitter:title" content="TWITTER BANNER TITLE META TAG">
1919
<meta name="twitter:description" content="TWITTER BANNER DESCRIPTION META TAG"> -->
20-
<!-- Path to banner image, should be in the path listed below. Optimal dimenssions are 1200X600-->
20+
<!-- Path to banner image, should be in the path listed below. Optimal dimensions are 1200X600-->
2121
<!-- <meta name="twitter:image" content="static/images/your_twitter_banner_image.png">
2222
<meta name="twitter:card" content="summary_large_image"> -->
2323
<!-- Keywords for your paper to be indexed by-->
@@ -202,7 +202,7 @@ <h2 class="title is-3">Abstract</h2>
202202
principles.
203203
Second, we evaluate its usability and performance along the development cycle of VLA and RL policies.
204204
Our experiments also provide an extensive evaluation of Octo, OpenVLA, and Pi Zero on multiple robots
205-
and shed light on how simulation data can improve real-world policy performance.
205+
and shed light on the benefits of simulated data for robotic foundation models.
206206
</p>
207207
</div>
208208
</div>
@@ -224,6 +224,23 @@ <h2 class="title is-3 has-text-centered">RCS in 3 Minutes</h2>
224224
<source src="static/videos/rcs_3min_overview.mp4" type="video/mp4">
225225
</video>
226226
</div>
227+
<br>
228+
<p>
229+
Traditional robotics is built around hardware, with many interacting parts and specialized AI modules.
230+
With machine learning taking the lead, this relationship flips around: <em> robots are components of a machine learning pipeline</em>.
231+
</p>
232+
233+
<br>
234+
<p>
235+
Many libraries embrace this and adopt a Python- and ML-first approach, but they often lack robust robotics features and hardware support.
236+
Robust policies require careful debugging in both simulation and hardware, which relies on classical robotics tools.
237+
</p>
238+
239+
<br>
240+
<p>
241+
RCS bridges this gap by combining an ML-first design with the essential robotics tools.
242+
It gives you the means to debug interfaces, validate tasks, and test directly on hardware&mdash;while remaining a lightweight pip-installable package with minimal dependencies.
243+
</p>
227244
</div>
228245
</div>
229246
</div>
@@ -236,11 +253,28 @@ <h2 class="title is-3 has-text-centered">RCS in 3 Minutes</h2>
236253
<div class="container is-max-desktop">
237254
<div class="columns is-centered has-text-centered">
238255
<div class="column">
239-
<div class="hero-body">
240-
<h2 class="title is-3">Architecture</h2>
256+
<div class="hero-body has-text-justified">
257+
<h2 class="title is-3 has-text-centered">Architecture</h2>
258+
<p>
259+
<b>C++/Python API</b>&nbsp;&nbsp;
260+
We provide device APIs in C++ with automatically generated Python bindings, ensuring mirrored functionality in both languages.
261+
A new device can be integrated into RCS in either C++ or in Python, ensuring broad hardware compatibility.
262+
</p>
263+
<br>
264+
<p>
265+
<b>Composable scenes</b>&nbsp;&nbsp;
266+
Higher-level abstractions are built on top of our own device APIs.
267+
They leverage Gymnasium wrappers to enable modular scene creation through composition.
268+
</p>
269+
<br>
270+
<p>
271+
<b>Layered architecture</b>&nbsp;&nbsp;
272+
Because we build upon a minimal low-level device API, you can quickly get up and running with new hardware: implement our interface, benefit from all the wrappers and apps higher up in the stack.
273+
</p>
274+
<br>
241275
<img src="static/images/rcs_architecture_small.svg" alt="RCS Architecture." width="100%">
242276
<div class="content has-text-justified">
243-
Applications (teleoperation, RL, VLA) interface with the environment (sim or real) through a unified
277+
Fig. 1: Applications (teleoperation, RL, VLA) interface with the environment (sim or real) through a unified
244278
Gymnasium API. Sensors, actuators, and observers wrap the environment, mutating action/observation
245279
spaces.
246280
</div>
@@ -258,7 +292,12 @@ <h2 class="title is-3">Architecture</h2>
258292
<!-- Setups and twins -->
259293
<section class="section">
260294
<div class="container is-max-desktop">
261-
<h2 class="title is-3 has-text-centered">Robots Setups with Digital Twins</h2>
295+
<h2 class="title is-3 has-text-centered">Robot Setups with Digital Twins</h2>
296+
<p>
297+
We evaluate the usability of RCS's hardware oriented features by integrating multiple setups with different robots, grippers, cameras and touch sensors.
298+
In total, four robots, four end-effectors, two cameras and a tactile sensor are implemented, both in simulation and on physical hardware.
299+
</p>
300+
<br>
262301

263302
<style>
264303
/* minimal CSS */
@@ -322,14 +361,21 @@ <h2 class="title is-3 has-text-centered">Robots Setups with Digital Twins</h2>
322361
<!-- TODO: Applications -->
323362
<section class="section hero is-small">
324363
<div class="container is-max-desktop">
325-
<div class="columns is-centered has-text-centered">
364+
<div class="columns is-centered has-text-justified">
326365
<div class="column">
327366
<div class="hero-body">
328-
<h2 class="title is-3">Applications</h2>
367+
<h2 class="title is-3 has-text-centered">Applications</h2>
368+
<br>
369+
<p>
370+
All implemented robots can be teleoperated with multiple devices and can be used to record data.
371+
We also verify that RCS integrates cleanly into ML pipelines, both in the imitation learning and reinforcement learning settings.
372+
We deploy multiple VLAs, and solve a simple simulated pick-up task with PPO, using proprioceptive and RGB states as observations.
373+
</p>
374+
<br>
329375
<br>
330376

331377
<hr>
332-
<h3 class="title is-4">Teleoperation & Data Collection</h3>
378+
<h3 class="title is-4 has-text-centered">Teleoperation & Data Collection</h3>
333379

334380
<!-- First row: 3 videos -->
335381
<div class="columns">
@@ -370,7 +416,7 @@ <h4 class="title is-5">Scripted Data Collection</h4>
370416
</div>
371417
<br>
372418
<hr>
373-
<h3 class="title is-4">Reinforcement Learning</h3>
419+
<h3 class="title is-4 has-text-centered">Reinforcement Learning</h3>
374420
<!-- 2 videos -->
375421
<div class="columns is-centered">
376422
<div class="column is-5">
@@ -387,7 +433,7 @@ <h3 class="title is-4">Reinforcement Learning</h3>
387433

388434
<br>
389435
<hr>
390-
<h3 class="title is-4">VLA Inferencing</h3>
436+
<h3 class="title is-4 has-text-centered">VLA Inference</h3>
391437
<!-- 2 videos -->
392438
<div class="columns is-centered">
393439
<div class="column is-5">
@@ -416,8 +462,11 @@ <h4 class="title is-5">Simulation</h4>
416462
<div class="container is-max-desktop">
417463
<div class="columns is-centered has-text-centered">
418464
<div class="column">
419-
<div class="hero-body">
420-
<h2 class="title is-3">Results</h2>
465+
<div class="hero-body has-text-justified">
466+
<h2 class="title is-3 has-text-centered">Results</h2>
467+
<p>We demonstrate how RCS supports VLA research by investigating VLA generalization across multiple embodiments and assessing the benefit of simulated data for robotic foundation models.
468+
</p>
469+
<br>
421470
<!-- Left column: Image -->
422471
<div class="columns is-vcentered">
423472
<div class="column is-half">
@@ -454,26 +503,19 @@ <h2 class="title is-3">Results</h2>
454503
</div>
455504

456505
<div class="content has-text-justified">
457-
Fig. 2: Success rate plots of different VLA comparisons.
458-
<i>Left:</i>
459-
The Pi Zero model fine-tuned on four datasets from different setups.
460-
Each fine-tuning dataset contains of less then 150 episodes and each model is evaluated on 50 rollouts.
461-
<i>Center:</i>
462-
Different models fine-tuned on 143 episodes on our FR3 setup (real) with a down-sampled frequency of 5Hz
463-
and evaluated on the real-world setup and the replicated simulated scene on 30 real-world and 100
464-
simulated rollouts.
465-
<i>Bottom:</i>
466-
Different data mixes of synthetic and real data evaluated on the real-world setup and the simulated
467-
scene on 30 real-world and 100 simulated rollouts.
468-
The number denotes the amount of episodes from the respective domain used in the training mix.
506+
Fig. 2: We fine-tune Pi Zero on four datasets from different setups.
507+
Each dataset contains fewer than 150 episodes.
508+
The fine-tuned models are deployed on the corresponding setups.
509+
The robots that are more prominent in the base model's data mix achieve better results.
469510
</div>
470511
<img src="static/images/results/sim_real_eval.svg"
471512
alt="Success rate plot over training checkpoints." width="100%">
472513
<div class="content has-text-justified">
473-
Fig. 3: Evaluation success rates measured for each checkpoint throughout the training process in the
474-
real and replicated simulated domain. Each checkpoint is evaluated on 20 real and 100 simulated
475-
rollouts. Left: Trained on 143 episodes on our FR3 dataset. Right: Trained on a mix of 143 episodes from
476-
our FR3 dataset and 500 episodes from the scripted dataset of the replicated simulated domain.
514+
Fig. 3: We investigate the impact of simulated data on VLA performance.
515+
Our setup is replicated in simulation and used to generate 500 trajectories using a scripted policy, which is then used to complement our manually collected dataset of 143 trajectories.
516+
The plots show the success rate of the policy, both in the simulated scene and on the hardware, as training progresses.
517+
Success rates in simulation correlate with success rates on the physical robot—consistent with a good evaluation metric.
518+
Adding simulated data to the training mix improves performance in both settings.
477519
</div>
478520
</div>
479521
</div>
@@ -531,4 +573,4 @@ <h2 class="title">BibTeX</h2>
531573

532574
</body>
533575

534-
</html>
576+
</html>

0 commit comments

Comments
 (0)