Application for participation
RIRES: Russian Information Retrieval Evaluation Seminar

 Call for participation 
 General principles 
 Test collections 
 Relevance tables 


Participation in the seminar

The seminar is open for new participants. The initiative is open for participation for all those interested, the commercial application developers as well as researchers in information retrieval.

A participant is free to decide on the set of tracks in which he/she would like participate and can directly influence the rules of behaviour of these tracks during their definition. We also encourage participants to suggest new versions of tracks for general discussion.

The participant is provided with:

  • A collection of data and tasks for the chosen tracks;
  • Detailed information about rules of conduction of the seminar and the approved methodology;
  • Anonymous evaluation of the results of the system being tested;
  • Information about all collected expert evaluations for the track(s) in which a participant is taking part in;
  • Collection of additional tools to facilitate import and evaluation procedures.

A participant is expected:

  • to present the result of the passage(s) of the system for its independent evaluation in predefined deadlines in the required format;
  • to present a paper with description of general principles of the employed approach and the obtained results and the report (presentation) at the seminar meeting;
  • to participate in compensation of expenses for the distribution of test corpora and evaluation of the results (through direct financial contributions or provision of resources for evaluation).
Solutions of modified tasks based on the collection of employed data are also encouraged.

Application for participation

The application for participation in the ROMIP seminar must contain the following information:

  • short presentation of the system (1-2 passages)
    (title, weblink, short description of the principles of the method used)
  • contact information
    (organization, contact e-mail)
  • list of tracks in which you are interested to participate
Applications are accepted at romip[AT]

Заходи: Лобня окна participation.