About

Call for participation

General principles

Participation

Tracks

Test collections

History

2004

2005

About the ROMIP Seminar

Russian information retrieval evaluation initiative was launched in 2002 with purpose to increase communication and support community of researchers (from both academia and industry) in the area of text retrieval for Russian language collections by providing infrastructure necessary for evaluation of information retrieval methodologies.

In many respects ROMIP seminars are similar to other world information retrieval events such as TREC, CLEF, NTCIR, etc. Initiation of the new one was motivated by several reasons:

absence of publicly available Russian test collections
relatively low interest for the creation of Russian language tracks/collections within the framework of the existing evaluation initiatives
low rate of participation of Russian research groups in the existing evaluation initiatives.

Similar to TREC ROMIP has cycle nature and is overseen by a program committee consisting of representatives from academia and industry. Given collection and tasks participants run their own system on the data and submit results to the organizing committee. Collected results are independently judged and the cycle ends with a workshop for sharing experience and discussing future plans.

ROMIP is a non-commercial project and is supported by a community of Russian researchers and developers working in the area of information retrieval. The organizational costs are recouped from participation fees. Evaluation results are intended only for research purposes and cannot be used for marketing or commercial purposes.

The methods and techniques we use for evaluation are developed by the program committee in collaboration with participants and are open for suggestions and discussion. The proceedings of the seminar are freely available on our web site. To get access to the test collections you can contact the program committee to complete the necessary agreements with the copyright holders.

Acknowledgements

It would not be possible to run the seminar without the cooperation of many different organizations and companies. We would like to acknowledge the following organizations and companies for supporting the seminar and providing test collections:

Special thanks to the organizers of the Russian Conference on Digital Library (RCDL), who helped us to carry out the annual workshops over the years and to the Russian Foundation for Basic Research for supporting the seminar (grant 04-07-90280).

We would also like to thank the assessors for their hard work in evaluation of results produced by participating systems.