International Workshop on Semantic Evaluation 2018

SemEval has evolved from the SensEval word sense disambiguation evaluation series. The SemEval wikipedia entry and the ACL SemEval Wiki provide a more detailed historical overview. SemEval-2018 will be the 12th workshop on semantic evaluation and will be collocated with the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2018). SemEval will be held in New Orleans, LA, USA, June 5-6 2018.

 

Important Dates

 

Task Proposals (Final call for Task Proposals - posted April 12, 2017):

  • Fri 21 Apr 2017: Task proposals due
  • Mon 01 May 2017: Task proposal reviewers assigned
  • Mon 22 May 2017: Task proposal reviews due
  • Mon 05 Jun 2017: Task proposal notifications

Setup for the Competition:

  • Mon 14 21 Aug 2017: CodaLab competition website ready and made public. Should include basic task description and mailing group information for the task. Trial data ready. Evaluation script ready for participants to download and run on the trial data.
  • Mon 18 25 Sep 2017: Training data ready. Development data ready. CodaLab competition website updated to include an evaluation script uploaded as part of the competition so that participants can upload submissions on the development set and the script immediately checks the submission for format and computes the results on the development set. This is also the date by which a benchmark system should be made available to participants. Also, the organizers should run the submission created with the benchmark system on CodaLab, so that participants can see its results on the LeaderBoard.

Competition and Beyond:

  • Mon 08 Jan 2018: Evaluation start*
  • Mon 29 Jan 2018: Evaluation end*
  • Mon 05 Feb 2018: Results posted
  • Mon 26 Feb 2018: System description paper submissions due by 23:59 GMT -12:00
  • Mon 05 Mar 2018: Task description paper submissions due by 23:59 GMT -12:00
  • Mon 19 Mar 2018: Paper reviews due (for both systems and tasks)
  • Mon 02 Apr 2018: Author notifications
  • Mon 16 Apr 2018: Camera ready submissions due

 

Discussion Group

 

Please join our discussion group at semeval3@googlegroups.com to receive announcements and participate in discussions. For details, questions, and discussion on a particular task, visit the task website. You will also find there a link for the task mailing list.

 

 

Anti-Harassment policy

 

SemEval highly values the open exchange of ideas, the freedom of thought and expression, and respectful scientific debate. We support and uphold the NAACL Anti-Harassment policy. Participants are encouraged to send any concerns or questions to the NAACL Board members, Priscilla Rasmussen and/or the workshop organizers.

Contact Info

Organizers

  • Marianna Apidianaki, LIMSI, CNRS, University Paris-Saclay & University of Pennsylvania
  • Saif M. Mohammad, National Research Council Canada
  • Jonathan May, ISI, University of Southern California
  • Ekaterina Shutova, University of Cambridge
  • Steven Bethard, University of Alabama at Birmingham
  • Marine Carpuat, University of Maryland

Email

semeval-organizers@googlegroups.com Note that this is the mailing list for SemEval organizers. For questions on a particular task, post them at the *task* mailing list or contact the task organizers directly. You can find the task mailing list from the task webpage.

Other Info

Announcements

  • Nov. 20: SemEval-2018 will be co-located with NAACL HLT 2018 in New Orleans. The workshop will be on June 5-6, 2018.