FarSense: A Comprehensive Commonsense Benchmark and Evaluation Framework for the Farsi Language
Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL 2025), Mumbai, India
Presented on
In short
The paper presents FarSense, a Farsi commonsense benchmark with six tasks (true/false, multiple choice, explanation, cause-effect, counterfactual reasoning and knowledge completion) built from about 4,210 Farsi Wikipedia passages and validated by native speakers. It evaluates five major LLMs and shows that fine-tuning six compact models (1B to 24B) on FarSense brings substantial gains.
Cite
@inproceedings{zeinalipour-etal-2025-farsense,
title = "{F}ar{S}ense: A Comprehensive Commonsense Benchmark and Evaluation Framework for the {F}arsi Language",
author = "Zeinalipour, Kamyar and
Jamshidi, Neda and
Hejazi, Seyedehbahareh and
Maggini, Marco and
Bianchini, Monica and
Paoletti, Simone and
Gori, Marco",
editor = "Inui, Kentaro and
Sakti, Sakriani and
Wang, Haofen and
Wong, Derek F. and
Bhattacharyya, Pushpak and
Banerjee, Biplab and
Ekbal, Asif and
Chakraborty, Tanmoy and
Singh, Dhirendra Pratap",
booktitle = "Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the {A}sia-Pacific Chapter of the {A}ssociation for {C}omputational {L}inguistics",
month = dec,
year = "2025",
address = "Mumbai, India",
publisher = "The Asian Federation of Natural Language Processing and The Association for Computational Linguistics",
url = "https://aclanthology.org/2025.ijcnlp-long.187/",
doi = "10.18653/v1/2025.ijcnlp-long.187",
pages = "3529--3599",
ISBN = "979-8-89176-298-5"
}