End-User Development using Natural Language
Updates for the Evaluation Period
1- We've updated the action KB file[1] and the action-command training mapping file[2] to correct some small inconsistencies. Please, update your algorithm using the new files.
2- The evaluation period has started. The input file is available in our website[3] and the codalab competition[4] is ready to receive the submissions.
Updates
1- The training dataset has been updated to correct some errors. It is available in the data page.
2- The Codalab competition was released in https://competitions.codalab.org/competitions/15842
Overview
This task proposes a challenge to support the interaction of end-users with applications, micro-services and software APIs using natural language. It aims at supporting the evaluation and evolution of the discussions surrounding the application of natural language processing techniques within the context of end-user natural language programming.
Recent advances in Natural Language Processing bring the opportunity of improving the interaction between users and software artefacts, supporting end-users programming mediated by natural language. This ability to match user’s action intents and information needs to formal actions within a set of Application Programming Interfaces (APIs), using the semantics of natural language as the mediation layer between both, can significantly impact the accessibility of software development.
This SemEval task aims at developing the state-of-the-art discussions and techniques concerning the semantic interpretation of natural language commands and user action intents, bridging the semantic gap between users and software artefacts.
Mail List
In case of questions, please join our Google Group: https://groups.google.com/forum/#!forum/semeval-task11