About
Implementation of the standard evaluation metrics as described in Dan Marcu's book.
Related publications
- Joty, Shafiq, Giuseppe Carenini, and Raymond T. Ng. CODRA: A Novel Discriminative Framework for Rhetorical Analysis. [PDF] [BibTeX]
@article{jotycodra, title={CODRA: A Novel Discriminative Framework for Rhetorical Analysis}, author={Joty, Shafiq and Carenini, Giuseppe and Ng, Raymond T} }
Download
- Evaluation Metrics for Discourse Parsing. Latest release.
Usage
- Extract
Set.tar.gz. - Run the perl script:
Perl ParsingAccuracyMeasuresDocLevelForSystems.pl path_to_sys_dir path_to_gold_dir res.outThe main perl script takes three arguments:- Path to the directory with the system annotations:
The filenames should end with *. doc_dis, but you can change the code according to your need. Here is where you need to change:
my @canFiles = grep /\w+\.doc_dis/, readdir(DIR); - Path to the Directory with the gold annotations: It assumes that the file names are the same as the system outputs (e.g., *.doc_dis).
- The name of the output file: In the output file, it shows the results for the individual documents as well as the summary.
- Path to the directory with the system annotations:
The filenames should end with *. doc_dis, but you can change the code according to your need. Here is where you need to change: