Data and Tools
System annotations must be valid according to the TimeML XSD.
TimeML XSD: This XML schema is taken as reference to decide if a TimeML annotation is valid (Download).
TimeML validator: Tool that validates a TimeML annotation using the official timeml.xsd. (Download).
Dev Data
- QA Kit with data and tools for testing over sample data (Download) (QA version is outdated)
Train Data
- TimeML annotated data to train systems: TBAQ-cleaned (Download) and TE3-Platinum (Download). TempEval-3 data and tools are re-used because the task for participating systems is the same as the original Task ABC (Link)
- QA Kit with data and tools for testing over training data (Download, new Mar 9th, 2015)
Test Data (Domains: news, wiki, informal blogs)
- Unannotated documents to be annotated by systems: download
- QA Kit for testing the test data: download (new Mar 9th, 2015)
--------------------------------------
QA system source
Note this is only useful to understand the evaluation, but the system will be used only by organizers to evaluate. The participants task remains just automatically annotating plain text documents following TimeML.
In addition to the old TempEval-3 tools and data that will be completely reused (same format), in this QA evaluation we also have a QA system capable of answering temporal questions given a TimeML annotated document.
Download: binary (new March 9th, 2015)
Source code: https://github.com/hllorens/timeml-qa
To know more details about this system see http://doi.ieeecomputersociety.org/10.1109/ICSC.2012.34
More documentation and examples will be posted soon.
Please contact hector.llorens.martinez@gmail.com for further information.