R-, r-, reply- Articles, what are they? Case Studies of Reproducibility and Benchmarking in Robotic Research

October 14th 2024 PM

Scope

The primary aim of this workshop is to provide practical examples, case studies and guidance towards a comprehensive understanding of reproducibility in robotics, through expert inspiration and mentoring, presentations of concrete examples of R- and r-articles and interactive discussion. By spotlighting the significance of consistent and reproducible outcomes in real-world robotic applications, we aim to push the boundaries of knowledge, encourage robust discussions, and pave the way for collaborative research that will define the next era of robotic advancement. Beside general aspects about reproducibility and benchmarking in robotic research, the workshop will particularly focus on the challenges and possible solutions related to autonomous mobile systems, using R- and r-articles as key tools for advancing reproducibility.
The workshop intertwines with a competition that is being organized within IROS 2024, focusing on a race of autonomous vehicles. This competition will be benchmarked and made reproducible at three different levels: pre-field simulation, simulation-to-real transfer and real-world experiments. This workshop can provide valuable inputs and methodologies to the post-processing data analyzing and benchmarking of obtained performance, while in turn the competition can feed new challenges and aspects to be discussed within the community in this and also future events.