Legal Classification Track

Overview

For this track the standard procedure is used.

Test Collection

Task Description for Participating Systems

Each participant is granted access to the training set, and a set of documents from the legal documents collection. The task is to assign topic(s) from the training set to each document from the collection. Valid number of topics per document is from 0 to 5.

The training set is a subset of the categories based on the Kodeks catalog.

All the documents from the collection must be classified by participants, so for each of these documents must be specified a list of topics (which should be sorted in descending order of confidence).

Evaluation Methodology

• A random subset of the categories is selected. The full Kodeks catalog (which was verified by experts manually) is used for the evaluation of the selected subset.
• official metrics:
  
  • precision
  • recall

Data Formats

• collection
• training set
• results