Frequently Asked Questions

 

Q: I have questions pertaining to SemEval and its tasks. How do I find answers?
A: For questions specific to a task: browse the task webpage, join the task mailing list, look if the question has already been responded to in the mailing list. If you still do not find the answer to your question, post it on the task mailing list.
For general SemEval questions: browse the SemEval webpages, including this FAQ page. If you still do not find the answer to your question, email semeval-organizers@googlegroups.com
 

 

Q: I want to participate, what are the first steps?
A: Register for the task you want to participate in through the SemEval-2017 registration form. See task website for task details. Download any data or scripts made available on the task website. Join the task mailing list.

 

Q: I registered but I did not get an acknowledgment email. Has my registration been recorded?
A: Once you register, expect to see a page that says "Your response has been recorded". The system does *not* send a confirmation email.

 

Q: What is the deadline for registering a team for a task?
A: You can register any time before the task evaluation period ends. However, it is preferable to register early. Your registration helps us plan for the tasks. 

 

Q: How many runs can be sent for each task?
A: Task organizers often specify how many submissions by each team will be evaluated. Email your task organizers if these details are not already stated on the task webpage.

 

Q: Can I take part in a Semeval Task as an individual?
A: Yes.

 

Q: Can I make submissions to multiple SemEval tasks?
A: Yes, the same person or team can make submissions to multiple SemEval tasks. However, individual task organizers might place restrictions on how many teams the same person can be involved in for their task. For example, they might stipulate that one cannot make submissions as part of more than one team for their task. That is, they may state that a person cannot be in two separate teams that both make submissions to the same task. Task organizers often specify how many submissions by each team will be evaluated, and the above stipulation is a way to avoid abusing that by simply creating multiple teams. Email specific task organizers if these details are not already clearly stated on the task webpage.

 

Q: What is the evaluation period for the task I am interested in?
A: 9 Jan to 30 Jan 2017 is the period during which the task organizers must schedule the evaluation periods for their individual tasks. Usually, evaluation periods for individual tasks is 7 to 14 days, but there is no hard and fast rule about this. Contact the task organizers for the tasks you are interested in for the exact time frame when they will conduct their evaluations. They should tell you the date by which they will release the test data, and the date by which participant submissions are to be uploaded. Note that some tasks may involve more than one sub-tasks, each having a separate evaluation time frame.

 

Q: Do I have to submit a paper to the SemEval workshop? Do I have to register for the SemEval workshop?
A: While we strongly encourage all participants to attend the workshop, it is not required to either register or attend the workshop for your system paper to be included in the proceedings.

 

Q: Will the participant submissions be made available for download?
A: Yes.

 

Q: Can I submit system output after the competition deadline?
A: You could still request the task organizers to evaluate your system output, however, agreeing to do so is up to them. If the deadline has passed, the best option is to use the evaluation script and gold answers available on the task website to evaluate your system output yourself.

 

Q: Is participation in the tasks and workshop free?
A: Submitting a system output to SemEval tasks is free.  You can attend the workshop by registering for the workshop, and it comes with a registration fee.

Contact Info

Organizers

Email

semeval-organizers@googlegroups.com Note that this is the mailing list for SemEval organizers. For questions on a particular task, post them at the *task* mailing list. You can find the task mailing list from the task webpage.

Other Info

Announcements