R-, r-, reply- Articles, what are they? Case Studies of Reproducibility and Benchmarking in Robotic Research

Exploring Our Focus


The era of robotics is upon us, with autonomous vehicles emerging as pivotal aspect in current robotics and a crucial testbed for research methodology. Robotics has made tremendous progress. However, the real transfer into everyday life is often less quick than expected due to lacking procedures and standards to benchmark and assure safe behaviors of robots. This workshop faces such topics, focusing on the critical aspect of reproducibility in general robotics and, specifically, in the critical area of research on autonomous locomotion as specific case study of interest. The introduction and adoption of R- and r-articles represent a significant advancement in this effort. R-articles, with their emphasis on the reproducibility of results, provide a solid foundation for credible peer evaluations and objective assessments of system behaviors. By requiring the provision of all information necessary to replicate experiments, R-articles ensure that research can be independently verified, thus enhancing the reliability and credibility of the findings. The subsequent creation of r-articles, which detail attempts to reproduce the experiments, further solidifies this process, allowing for the confirmation or refutation of original results. This approach has the potential to revolutionize research in robotics by establishing a culture of transparency and replicability, which are essential for the field's advancement. The appreciation of the significance of these topics is widespread within the community, especially as society progresses towards a future where robots play an integral role in diverse sectors, from industry to healthcare, research, civilian and everyday life. By emphasizing state-of-the-art techniques and fostering discussions on challenges, the workshop is poised to stand as a landmark of originality and diversity for IROS 2024.

Delving into Key Topics

This workshop is designed to encompass a wide range of topics, with special focus, but not limited, to the following areas:
• Benchmarking methods for robotics research
• R-, r- and reply article in Robotics
• Best Practices Autonomous mobile robots
• Software Tools and Platforms
• Benchmarking methods for robotics research
• Statistical significance of results
• Open-Source and Novel Platforms for Reproducible Research Robotic Task Standardization
• Peer Evaluation and Metrics
• Real-World Case Studies
• Collaborative Robotics & Advancements