Data and Tools

English TRAIN+DEV data v3.2

  • Data for all English subtasks v3.2 is here
  • It includes a TRAIN/DEV split with reliable double-checked DEV
    • Subtask A (6,398 questions + 40,288 comments) + unannotated (189,941 questions + 1,894,456 comments)
    • Subtask B (317 original + 3,169 related questions)
    • Subtask C (317 original questions + 3,169 related questions + 31,690 comments)

Arabic TRAIN+DEV data v1.3

  • The Arabic TRAIN+DEV data v 1.3 can be found here
  • It includes a TRAIN/DEV split with reliable double-checked DEV (1,281 original questions, and 37,795 potentially related question-answer pairs) + unannotated (163,383 question--answer pairs)

Scorer v2.2 and random baselines (English and Arabic)

 

*** TEST INPUT DATA ***

  • Can be found here
  • Format checker for the test output is here
  • The GOLD labels and results are here

 

Contact Info

Organizers


  • Preslav Nakov, Qatar Computing Research Institute, HBKU
  • Lluís Màrquez, Qatar Computing Research Institute, HBKU
  • Alessandro Moschitti, Qatar Computing Research Institute, HBKU
  • Walid Magdy, Qatar Computing Research Institute, HBKU
  • James Glass, CSAIL-MIT
  • Bilal Randeree, Qatar Living

email : semeval-cqa@googlegroups.com

Other Info

Announcements


  • Task description paper is now released!
  • EVALUATION results are now released!
  • Test format checker has been released!
  • Test data has been released!
  • Arabic TRAIN+DEV data v1.3 released!
  • English TRAIN+DEV data v3.2 released!
  • Scorer and baselines v2.2 released!
  • Register to participate here