Why Rawseeds?
The absence of standard benchmarks is a widely acknowledged problem in the robotics field, and is doubly harmful to it: firstly, because it prevents recognition of scientific and technical progress, thus discouraging research and development; and secondly, because it prevents new actors (and particularly companies) from entering the robotic sector, as heavy investments are needed to compensate for that absence. In particular, there is a distinct lack of quantitative benchmarking instruments to reliably assess and compare the performance of different systems.
Actually, one of the most substantial limitations to the development of mobile autonomous robotics is the sheer arduousness and cost of performing repeatable and reliable tests of systems and algorithms. In addition, it is difficult to quantitatively assess the performance of a system in ways which are meaningful to people outside the group who designed the system; and it is often impossible to compare the results obtained with different solutions or by different research teams. This has a stifling effect on the whole robotics field, especially where industrial research policies are concerned: no one is happy to operate in a field where marketable applications abound, but heavy investments are needed to simply check if an idea is a good one, before any engineering or industrial effort can even begin.
The problem described above has two causes: first, the experiments needed to test an algorithm or robotic system are extremely difficult and costly to set up; second, the unavailability of standard methodologies and applications to quantitatively evaluate the performance of such algorithms and systems makes the experiments nearly useless for groups different from the one which performed them in the first place.
Rawseeds strives to tackle and eliminate both causes of the problem.
There are many reasons why Rawseeds is useful and many categories of user who can benefit from Rawseeds.
- For researchers, Rawseeds will speed up considerably the production of innovative, high-quality results (i.e., algorithms, software and complete robotic applications). The use of Rawseeds’ Benchmarking Toolkit will let research groups evaluate each other’s work and compare it with their own. The best solutions will be easily and rapidly singled out; highly innovative but “far out” ideas will no more be neglected for the sheer cost of testing out them. Finally, even the smallest research groups will have the opportunity to see their work publicized and acknowledged by the community, by publishing their results on the Rawseeds website as Benchmark Solutions.
- For companies with an interest into advanced robotics but not yet involved, Rawseeds will represent the force that will drive them in. Knowing that their ideas and solutions can be tested with a small expense of time and money, the main reason for them to wait will disappear. This is especially true for SMEs, i.e. Small and Medium Enterprises, both because they are especially sensible to cost issues and because they will have access to an easy mean to compare the performance of their (prospective) products with those of competitors already entered into robotics or with state-of-the-art research.
- For companies that are already involved into robotics, Rawseeds will let them make their achievements known to all the robotics community, without compromising their intellectual property. The availability of Rawseeds’ Benchmarking Toolkit as a development tool will also open the way to the design of new, more advanced products.
The presence within the Toolkit of a corpus of already working algorithms (in the form of Benchmark Solutions) means that no one will ever need to “start from scratch”.
Still having questions? Please take a look to our Frequently Asked Questions page (and possibly submit your own question if you don’t find it there).
Of course, if you need it you can contact us.