SemEval-2017 Task 8

Submissions should be submitted on CodaLab.

• RumourEval task A
• RumourEval task B (open)
• RumourEval task B (closed)

This download contains the data, split into training and development data:

• semeval2017-task8-dataset.tar.bz2

Test data

• scoring scripts: scorer.zip
• development data key: dev_key.zip
• test tweets: rumoureval2017-test.tar.bz2
• gold standard test data: subtaskA.json subtaskB.json

Two news stories are included, which form the input for both subtasks. You can use the scorer over the development to validate your system. Final submissions are to be made on the CodaLab site.

The dataset used for training and development testing is from this PLoS article:

• Zubiaga A, Liakata M, Procter R, Wong Sak Hoi G, Tolmie P (2016) Analysing How People Orient to and Spread Rumours in Social Media by Looking at Conversational Threads. PLoS ONE 11(3): e0150989. doi:10.1371/journal.pone.0150989

The original data can be found here:

• PHEME rumour scheme dataset: journalism use case, version 2

For the open variant of task B, the following wikipedia dump may be used:

• 20160901: https://dumps.wikimedia.org/enwiki/20160901/