ArabicSense: A Benchmark for Evaluating Commonsense Reasoning in Arabic with Large Language Models
Proceedings of the 4th Workshop on Arabic Corpus Linguistics (WACL-4), at COLING 2025, Abu Dhabi, UAE
Presented on , 7 citations on Google Scholar
In short
The paper introduces ArabicSense, a benchmark for commonsense reasoning in Arabic with three tasks: spotting statements that do not make sense, choosing the reason why, and generating an explanation. Fine-tuning on the dataset helped: Gemma and Mistral-7B reached F1 scores of 95.5% and 94.8% on the second task, and LLaMA-3 led the generation task with a BERTScore F1 of 77.3%.
Cite
@inproceedings{lamsiyah-etal-2025-arabicsense,
title = "{A}rabic{S}ense: A Benchmark for Evaluating Commonsense Reasoning in {A}rabic with Large Language Models",
author = "Lamsiyah, Salima and
Zeinalipour, Kamyar and
El amrany, Samir and
Brust, Matthias and
Maggini, Marco and
Bouvry, Pascal and
Schommer, Christoph",
editor = "Ezzini, Saad and
Alami, Hamza and
Berrada, Ismail and
Benlahbib, Abdessamad and
El Mahdaouy, Abdelkader and
Lamsiyah, Salima and
Derrouz, Hatim and
Haddad Haddad, Amal and
Jarrar, Mustafa and
El-Haj, Mo and
Mitkov, Ruslan and
Rayson, Paul",
booktitle = "Proceedings of the 4th Workshop on {A}rabic Corpus Linguistics ({WACL}-4)",
month = jan,
year = "2025",
address = "Abu Dhabi, UAE",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.wacl-1.1/",
pages = "1--11"
}