Data and Tools

System annotations must be valid according to the TimeML XSD.

TimeML XSD: This XML schema is taken as reference to decide if a TimeML annotation is valid (Download).

TimeML validator: Tool that validates a TimeML annotation using the official timeml.xsd. (Download).


Dev Data

  • QA Kit with data and tools for testing over sample data (Download) (QA version is outdated)

Train Data

  • TimeML annotated data to train systems: TBAQ-cleaned (Download) and TE3-Platinum (Download). TempEval-3 data and tools are re-used because the task for participating systems is the same as the original Task ABC (Link)
  • QA Kit with data and tools for testing over training data (Download, new Mar 9th, 2015)

Test Data (Domains: news, wiki, informal blogs)

  • Unannotated documents to be annotated by systems: download
  •  QA Kit for testing the test data: download (new Mar 9th, 2015)


QA system source

Note this is only useful to understand the evaluation, but the system will be used only by organizers to evaluate. The participants task remains just automatically annotating plain text documents following TimeML.

In addition to the old TempEval-3 tools and data that will be completely reused (same format), in this QA evaluation we also have a QA system capable of answering temporal questions given a TimeML annotated document.

Download: binary (new March 9th, 2015)

Source code:

To know more details about this system see

More documentation and examples will be posted soon.

Please contact for further information.

Contact Info


  • Hector Llorens
  • Nate Chambers
  • Naushad UzZaman
  • Nasrin Mostafazadeh
  • James Allen
  • James Pustejovsky

email :

Other Info


  • 2015-03-09 UPDATED results of QA-TempEval
  • 2015-01-15 Results of QA-TempEval
  • Fill in SemEval registration form to participate
  • 2014-11-04 evaluation period starts on December 15
  • 2014-12-01 crowd-sourced test creation: OPEN (from Dec 5 to Jan 10)
  • 2014-11-04 A new domain will be included: informal blogs
  • 2014-08-26 PDF detailed task description
  • 2014-06-23 Train QA data released (data&tools)
  • 2014-06-14 Dev QA data released (data&tools)
  • 2014-06-02 TimeML QA system released (data&tools)
  • 2014-05-30 TimeML data available (TempEval-3 format) (data&tools)