International Workshop on Semantic Evaluation 2020
☞ NEW: SemEval-2020 best task and paper awards!
☞ NEW: SemEval-2020 program for Dec. 12-13! Registration is through COLING.
☞ For future editions including SemEval-2021, see the SemEval homepage.
SemEval has evolved from the SensEval word sense disambiguation evaluation series. The SemEval wikipedia entry and the ACL SemEval Wiki provide a more detailed historical overview. SemEval-2020 will be the 14th workshop on semantic evaluation.
SemEval-2020 will be held December 12-13, 2020 in Barcelona, Spain, collocated with The 28th International Conference on Computational Lingustics (COLING-2020).
Important Dates
Task proposals for SemEval-2021 are closed. For further information, see the SemEval-2021 website (including the list of tasks).
Setup for the Competition:
31 July 2019: CodaLab competition website ready and made public. Should include basic task description and mailing group information for the task. Trial data ready. Evaluation script ready for participants to download and run on the trial data.
4 September 2019: Training data ready. Development data ready. CodaLab competition website updated to include an evaluation script uploaded as part of the competition. Participants should be able to upload submissions on the development set, and the script will immediately check the submission for format and compute results on the development set. This is also the date by which a benchmark system should be made available to participants. Also, the organizers should run the submission created with the benchmark system on CodaLab, so that participants can see its results on the LeaderBoard.
Competition and Beyond:
19 February 2020: Evaluation start*
11 March 2020: Evaluation end*
18 March 2020: Results posted
15 May 2020: System description paper submissions due (11:59pm, UTC-12)
22 May 2020: Task description paper submissions due (11:59pm, UTC-12)
24 Jun 2020: Author notifications
24 Jul 2020: Camera-ready submissions due for system description papers (11:59pm, UTC-12)
31 Jul 2020: Camera-ready submissions due for task description papers (11:59pm, UTC-12)
12-13 December 2020: SemEval 2020
* 19 February to 11 March 2020 is the period during which the task organizers must schedule the evaluation periods for their individual tasks. Usually, evaluation periods for individual tasks are 7 to 14 days, but there is no hard and fast rule about this. Contact the task organizers for the tasks you are interested in for the exact time frame when they will conduct their evaluations. They should tell you the date by which they will release the test data, and the date by which participant submissions are to be uploaded. Note that some tasks may involve more than one sub-task, each having a separate evaluation time frame.
For details, questions, and discussion on a particular task, visit the task website. You will also find there a link for the task mailing list. If you have general questions, you can email the organizers at the email address shown in the sidebar.
Anti-Harassment policy
SemEval highly values the open exchange of ideas, freedom of thought and expression, and respectful scientific debate. We support and uphold the NAACL Anti-Harassment policy. Participants are encouraged to send any concerns or questions to the NAACL Board members, Priscilla Rasmussen and/or the workshop organizers.