International Workshop on Semantic Evaluation 2020

 

SemEval has evolved from the SensEval word sense disambiguation evaluation series. The SemEval wikipedia entry and the ACL SemEval Wiki provide a more detailed historical overview. SemEval-2020 will be the 14th workshop on semantic evaluation. 

 

SemEval-2020 will be held September 13-14, 2020 in Barcelona, Spain, collocated with The 28th International Conference on Computational Lingustics (COLING-2020).

Important Dates

 

Setup for the Competition:

31 July 2019: CodaLab competition website ready and made public. Should include basic task description and mailing group information for the task. Trial data ready. Evaluation script ready for participants to download and run on the trial data.
4 September 2019: Training data ready. Development data ready. CodaLab competition website updated to include an evaluation script uploaded as part of the competition. Participants should be able to upload submissions on the development set, and the script will immediately check the submission for format and compute results on the development set. This is also the date by which a benchmark system should be made available to participants. Also, the organizers should run the submission created with the benchmark system on CodaLab, so that participants can see its results on the LeaderBoard.

 

Competition and Beyond:

10 Jan 2020: Evaluation start*
31 Jan 2020: Evaluation end*
05 Feb 2020: Results posted
23 Feb 2020: System and Task description paper submissions due
29 March 2020: Author notifications
6 Apr 2020: Camera ready submissions due
Summer 2020:  SemEval 2020

 

* 10 Jan to 31 Jan 2020 is the period during which the task organizers must schedule the evaluation periods for their individual tasks. Usually, evaluation periods for individual tasks are 7 to 14 days, but there is no hard and fast rule about this. Contact the task organizers for the tasks you are interested in for the exact time frame when they will conduct their evaluations. They should tell you the date by which they will release the test data, and the date by which participant submissions are to be uploaded. Note that some tasks may involve more than one sub-task, each having a separate evaluation time frame.

 

Questions?

 

For details, questions, and discussion on a particular task, visit the task website. You will also find there a link for the task mailing list. If you have general questions, you can email the organizers at the email address shown in the sidebar.

 

Anti-Harassment policy

 

SemEval highly values the open exchange of ideas, freedom of thought and expression, and respectful scientific debate. We support and uphold the NAACL Anti-Harassment policy. Participants are encouraged to send any concerns or questions to the NAACL Board members, Priscilla Rasmussen and/or the workshop organizers.

Contact Info

Organizers

Email

Other Info

Announcements

  • 2019/10/25: SemEval-2020 will be held September 13–14, 2020 in Barcelona, Spain, colocated with COLING 2020.