SemEval-2020

Q: I have questions pertaining to SemEval and its tasks. How do I find answers?
A: For questions specific to a task: browse the task webpage, join the task mailing list, look if the question has already been responded to in the mailing list. If you still do not find the answer to your question, post it on the task mailing list.
For general SemEval questions: browse the SemEval webpages, including this FAQ page. If you still do not find the answer to your question, email semeval-organizers@googlegroups.com.

Q: How many runs can be sent for each task?
A: Task organizers often specify how many submissions by each team will be evaluated. Email your task organizers if these details are not already stated on the task webpage.

Q: Can I take part in a Semeval Task as an individual?
A: Yes.

Q: Can I make submissions to multiple SemEval tasks?
A: Yes, the same person or team can make submissions to multiple SemEval tasks. However, individual task organizers might place restrictions on how many teams the same person can be involved in for their task. For example, they might stipulate that one cannot make submissions as part of more than one team for their task. That is, they may state that a person cannot be in two separate teams that both make submissions to the same task. Task organizers often specify how many submissions by each team will be evaluated, and the above stipulation is a way to avoid abusing that by simply creating multiple teams. Email your task organizers if these details are not already clearly stated on the task webpage.

Q: What is the evaluation period for the task I am interested in?
A: 10 Jan to 31 Jan 2020 is the period during which the task organizers must schedule the evaluation periods for their individual tasks. Usually, evaluation periods for individual tasks is 7 to 14 days, but there is no hard and fast rule about this. Contact the task organizers for the tasks you are interested in for the exact time frame when they will conduct their evaluations. They should tell you the date by which they will release the test data, and the date by which participant submissions are to be uploaded. Note that some tasks may involve more than one sub-tasks, each having a separate evaluation time frame.

Q: Do I have to submit a paper to the SemEval workshop? Do I have to register for the SemEval workshop?
A: While we strongly encourage all participants to attend the workshop, and to submit papers, it is not required to either register or attend the workshop for your system paper to be included in the proceedings.

Q: Can I submit system output after the competition deadline?
A: You could still request the task organizers to evaluate your system output, however, agreeing to do so is up to them. If the deadline has passed, the best option is to use the evaluation script and gold answers available on the task website to evaluate your system output yourself.

Q: Is participation in the tasks and workshop free?
A: Submitting a system output to SemEval tasks is free.  You can attend the workshop by registering for the workshop, and it comes with a registration fee.