TalkFa: A Unified Benchmark for Farsi Dialogue Generation and Understanding
arXiv preprint arXiv:2609.01810
Preprint posted on
In short
The paper releases TalkFa, a Farsi dialogue benchmark with three human-reviewed datasets: 4.2K Wikipedia-grounded dialogues, 6.6K dialogues labeled for dialogue acts and emotions, and 2.1K theatrical dialogues with sentiment labels. Experiments with six LLaMA and Mistral models show that LoRA fine-tuning recovers over 90% of its gains with only 25 to 50% of the training data, and that automatic metrics and an LLM judge overrate dialogue quality compared with human evaluation.
Cite
@misc{jamshidi2026talkfaunifiedbenchmarkfarsi,
title={TalkFa: A Unified Benchmark for Farsi Dialogue Generation and Understanding},
author={Neda Jamshidi and Kamyar Zeinalipour and Fahimeh Akbari and Monica Bianchini and Marco Maggini and Marco Gori},
year={2026},
eprint={2609.01810},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2609.01810},
}