Results


 

Relatedness Subtask

(Official ranking: Pearson correlation)

 

TEAM ID PEARSON SPEARMAN MSE
ECNU_run1 0.82795 0.76892 0.32504
StanfordNLP_run5 0.82723 0.75594 0.32300
The_Meaning_Factory_run1 0.82680 0.77219 0.32237
UNAL-NLP_run1 0.80432 0.74582 0.35933
Illinois-LH_run1 0.79925 0.75378 0.36915
CECL_ALL_run1 0.78044 0.73166 0.39819
SemantiKLUE_run1 0.78019 0.73598 0.40347
RTM-DCU_run1 0.76391 0.68769 0.42906
UTexas_run1 0.71455 0.67444 0.49900
UoW_run1 0.71116 0.67870 0.51137
FBK-TR_run3 0.70892 0.64430 0.59135
BUAP_run1 0.69698 0.64524 0.52774
UANLPCourse_run2 0.69327 0.60269 0.54225
UQeResearch_run1 0.64185 0.62565 0.82252
ASAP_run1 0.62780 0.59709 0.66208
Yamraj_run1 0.53471 0.53561 2.66520
asjai_run5 0.47952 0.46128 1.10372


 

 

Entailment Subtask

 

TEAM ID ACCURACY
Illinois-LH_run1 84.575
ECNU_run1 83.641
UNAL-NLP_run1 83.053
SemantiKLUE_run1 82.322
The_Meaning_Factory_run1 81.591
CECL_ALL_run1 79.988
BUAP_run1 79.663
UoW_run1 78.526
UEdinburgh_run1 77.106
UIO-Lien_run1 77.004
FBK-TR_run3 75.401
StanfordNLP_run5 74.488
UTexas_run1 73.229
Yamraj_run1 70.753
asjai_run5 69.758
haLF_run2 69.413
RTM-DCU_run1 67.201
UANLPCourse_run2 48.731

 

Contact Info

Organizers

  • Marco Marelli, University of Trento, Italy
  • Stefano Menini, Fondazione Bruno Kessler, Italy
  • Marco Baroni, University of Trento, Italy
  • Luisa Bentivogli, Fondazione Bruno Kessler, Italy
  • Raffaella Bernardi, University of Trento, Italy
  • Roberto Zamparelli, University of Trento, Italy

email : marco.marelli@unitn.it

Other Info

Announcements

  • We have released the primary runs submitted by participants. You can find them at the Data and Tools page.
  • We have released the general results summarizing the performances of the primary runs of all participating systems. You can found them at the 'Results' page
  • We have released the gold scores of the test set. They can be downloaded from the 'Data and tools' page
  • The test data have been released: you can find them in the 'Data and tools' page
  • We uploaded a new version of the evaluation script: you can find it in the 'Data and tools' page
  • We released a script for computing baselines: you can find it in the 'Data and tools' page
  • The train data and the evaluation script have been released: you can find them in the 'Data and tools' page
  • You can now register to the task at the 'SemEval 2014 website' .
  • Please note that we released a new version of the trial data on December 6th, after Johan Bos pointed out that the earlier release contained some repeated items (thanks Johan!). Nothing fundamental should hinge on the difference.
  • Trial data released: you can find them in the 'Data and tools' page
Copyright 2024 - SemEval-2014 Task 1. All Right Reserved

Designed by BhMad Studio  |  Powered by GetSimple