HBKU - QCRI
AraBench: Benchmarking Dialectal Arabic-English Machine Translation

Abstract:
 
Low-resource machine translation suffers from the scarcity of training data and the unavailability of standard evaluation sets. While a number of research efforts target the former, the unavailability of evaluation benchmarks remain a major hindrance in tracking the progress in low-resource machine translation. In this paper, we introduce AraBench, an evaluation suite for dialectal Arabic to English machine translation. Compared to Modern Standard Arabic, Arabic dialects are challenging due to their spoken nature, non-standard orthography, and a large variation in dialectness. To this end, we pool together already available Dialectal Arabic-English resources and additionally build novel test sets. AraBench offers 4 coarse, 15 fine-grained and 25 city-level dialect categories, belonging to diverse genres, such as media, chat, religion and travel with varying level of dialectness. We report strong baselines using several training settings: fine-tuning, back-translation and data augmentation. The evaluation suite opens a wide range of research frontiers to push efforts in low-resource machine translation, particularly Arabic dialect translation. The evaluation suite and the dialectal system are publicly available for research purposes.
 
Related publications

  • H. Sajjad, A. Abdelali, N. Durrani and F. Dalvi, “AraBench: Benchmarking Dialectal Arabic-English Machine Translation,” in COLING, 2020, pp. 123-456. doi:0000-0000
    [BibTeX] 
    @INPROCEEDINGS{hsajjad2020arabench,
author={Sajjad, Hassan and Abdelali, Ahmed and Durrani, Nadir and Dalvi, Fahim},
booktitle={GOLING},
title={AraBench: Benchmarking Dialectal Arabic-English Machine Translation},
year={2020},
month={Dec},
pages={123-456},
doi={doi:00000-0000}
}
Download
 
 
License
 
The Modern Standard Arabic Pronunciation Dictionary (Copyright 2015, QCRI a member of Qatar Foundation. All Rights Reserved) is licensed under the Apache License, Version 2.0 (the "License"); you may not use it except in compliance with the License. You may obtain a copy of the License here.Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.