Scope
In Robotics research the replicability and reproducibility of results and their objective evaluation and comparison is very difficult to put into practice. Controlling for environmental conditions is hard, defining comparable metrics and identifying goal similarity across various domains is poorly understood, and techniques for performance comparison are in an early stage. The importance of calibration issues is still not enough acknowledged in the community. Even determining the information required to enable replication of results has been the subject of extensive discussion. Even worse, there is still no solid theoretical foundation for experimental replicability in robotics. This situation impairs both research progress and technology transfer. However, significant progress has been made in these respects in recent years. IEEE R&A Magazine is now accepting reproducible papers, the so called R-articles, see here: http://www.ieee-ras.org/publications/ram/information-for-authors .
EuRobotics Topic Group on Benchmarking and Competitions