Data and Tools


Download the evaluation scripts

TaskA Eval Script TaskB Eval Script



Download the data



Trial and Training Data

The training/trial data consits of a single directory with several files. Each file corresponds to a single hashtag, and is named appropriately. For example, for the hashtag #DogSongs, the file is called Dog_Songs. We add the underscore between hashtag tokens for easier parsing of the hahstags. We believe a better semantic understanding of the hashtag will contribute to a better performance in the task.

The tweets are labeled 0, 1, or 2. 0 corresponds to a tweet not in the top 10 (most of the tweets in a file). 1 corresponds to a tweet in the top 10, but not the winning tweet (usually, 9 tweets per hashtag). 2 corresponds to the winning tweet (one tweet per hashtag).


Annotating Trial/Training Data

To do this task well there are potentially several subtasks that need to be accomplished. For example, most of the tweets in #DogSongs are dog-related puns to existing songs. To understand why a pun is funny, one would need to know the song it references. Therefore, we allow participants to provide manual annotations for the trial/training data, such as annotating the proper nouns referenced in a tweet. Annotations of any type cannot be done on Evaluation Data.


Evaluation data

For evaluation, tweets with different labels will be paired, and the goal will be to determine which tweet is the funnier. We ask that participants do not use the knowledge of label distributions directly when creating their systems.

Contact Info

Discussion Group
Hashtag Wars SemEval

Other Info


  • 2/6/2017 [new]
    The results have been posted!
  • 1/9/2017
    Evaluation data has been released!
  • 12/6/2016
    CodaLab competitions are ready!
  • 10/19/2016
    Evaluation scripts for both subtasks have been released!
  • 9/5/2016
    Train data has been released!
  • 8/1/2016
    Trial data has been released!
  • For participation in any of this year's tasks, please register by completing this form