GENERATING LEXICAL GAME RESOURCES FOR THE AZERBAIJANI LANGUAGE FROM THE WIKIDATA KNOWLEDGE GRAPH

УДК 004.8

Abstract

Taboo is a word game in which a player describes a target word without naming any of five forbidden related words. Building such cards by hand does not scale, and for a low-resource language such as Azerbaijani there is too little corpus and embedding signal to generate them reliably from distributional statistics alone. The objective of this work is to generate high-quality Taboo game cards for Azerbaijani automatically from a knowledge graph, without human card authoring and without machine translation. To this end the paper presents KG-Taboo, a pipeline that inverts the usual embedding-driven approach: the Wikidata knowledge graph generates the cards through explicit, typed semantic relations, while embeddings are demoted to a secondary ranking role. Candidate forbidden words are harvested by relation type (hypernym, co-hyponym, part or property, category, association, description keyword), normalized into clean game words, filtered to remove near-copies of the target, and the final five are selected with Maximal Marginal Relevance under a relation-diversity constraint. The resulting synthetic dataset, a category-balanced set of 3,444 cards over six categories drawn from a 5,305-card pool, is produced with no human card authoring and no machine translation, and passes every intrinsic quality gate. Every forbidden word is traceable to the Wikidata relation and property that produced it: co-hyponymy and hypernymy together supply about two thirds of the clues, with the remainder drawn from categories, descriptions, concrete properties and associations. The results show that a knowledge graph can serve as a substitute for scarce corpora when generating linguistic game content for a low-resource language.

Аннотация

«Табу» это словесная игра, в которой игрок описывает загаданное слово, не называя пяти запрещённых связанных слов. Создание таких карточек вручную плохо масштабируется, а для малоресурсного азербайджанского языка слишком мало корпусов и эмбеддингов, чтобы надёжно генерировать их только на основе дистрибутивной статистики. Цель работы автоматически генерировать качественные карточки игры «Табу» для азербайджанского языка на основе графа знаний, без ручного создания карточек и без машинного перевода. Для этого в данном исследовании представлен KG-Taboo конвейер, который переворачивает привычный подход на основе эмбеддингов: карточки порождает граф знаний Wikidata через явные типизированные семантические отношения, тогда как эмбеддинги выполняют лишь вспомогательную роль ранжирования. Кандидаты в запрещённые слова собираются по типу отношения (гипероним, согипоним, часть или свойство, категория, ассоциация, ключевое слово описания), нормализуются, очищаются от близких к загаданному слову, а пять финальных выбираются методом Maximal Marginal Relevance с ограничением на разнообразие отношений. Полученный синтетический набор сбалансированное по категориям множество из 3444 карточек по шести категориям, отобранных из пула в 5305 карточек, создан без ручного авторства и без машинного перевода и проходит все внутренние проверки качества. Каждое запрещённое слово прослеживается до отношения и свойства Wikidata, которое его породило: согипонимия и гиперонимия вместе дают около двух третей подсказок, а остальное берётся из категорий, описаний, конкретных свойств и ассоциаций. Результаты показывают, что граф знаний может служить заменой дефицитных корпусов при генерации языкового игрового контента для малоресурсного языка.

Keywords: generation, lexical game resources, Azerbaijani language, Wikidata knowledge graph.

Ключевые слова: генерация, лексические игровые ресурсы, азербайджанский язык, граф знаний Wikidata.

Introduction

Taboo is a description game: a player must get teammates to guess a target word without saying any of five forbidden (“taboo”) words printed on the card. A good card therefore needs a target plus five words that are genuinely related to it yet distinct enough that the player cannot simply utter a near-synonym. Authoring such cards by hand is slow and does not scale. A human-authored Azerbaijani set, “Yasaq”, exists, but hand authoring is exactly the bottleneck this work removes. Word games have also served as testbeds in natural language processing: Taboo and similar games are used to evaluate language models [13], [22], and the related game Codenames has been approached with language graphs and word embeddings [21] the same structured-plus-distributional toolkit this work builds on.

The Azerbaijani language presents several challenges for this task due to its linguistic complexity and limited computational resources [11], [12]:

• Azerbaijani is a morphologically rich and agglutinative language, producing many word variations and complex forms.

• There is a shortage of annotated corpora and high-quality datasets for training and evaluating models.

• Pretrained models and language-specific tools (tokenizers, lemmatizers) are lacking or underdeveloped.

• Its low-resource status limits transfer learning from models trained on high-resource languages.

• Purely corpus-driven or embedding-driven card generation is fragile when the underlying text data are thin.

An earlier line of work generates taboo candidates from embeddings and then cleans them with a filter. We argue this is the wrong way around for a low-resource language: where corpus statistics are thin, structured human knowledge is comparatively rich. KG-Taboo therefore inverts the pipeline. The Wikidata knowledge graph [1] generates the cards through explicit semantic relations, in the spirit of generating language from structured graph data [3], [19], and embeddings are kept only for secondary ranking and diversity. This turns the scarcest resource (corpora) into the optional one and the most abundant resource for the language (curated graph facts) into the generator.

KG-Taboo continues a line of work on natural language processing for serious games in Azerbaijani. Earlier studies applied deep learning to procedural content integration for learning games [14] and to multilingual game settings [15], analysed serious-game text with Word2Vec embeddings and a Naïve Bayes classifier [16], and used ontology-driven analysis to make such classification explainable [17]. KG-Taboo extends this ontology and knowledge-graph theme from analysing game content to generating it, and produces a dataset whose downstream value can be measured with the same embedding and classification pipelines used in that prior work.

The objective of this study is threefold: (i) to design a reproducible pipeline that generates Taboo game cards for Azerbaijani directly from the Wikidata knowledge graph using explicit, typed semantic relations rather than scarce corpora; (ii) to produce and release a category-balanced, fully provenanced dataset in which every forbidden word is traceable to the Wikidata relation and property that generated it; and (iii) to show, through intrinsic quality metrics and a provenance analysis, that a knowledge graph can serve as a substitute for text corpora when generating lexical game content for a low-resource language.

Materials and methods

For each target word, candidate forbidden words are harvested from Wikidata and grouped by the type of semantic relation, mirroring the lexical relations of hypernymy and co-hyponymy familiar from lexical databases [2]; the relation type also determines the card’s difficulty. Hypernyms come from a parent class or parent taxon (properties P279, P31, P171); co-hyponyms are siblings under the same parent; part or property words come from has-part, material, colour and use (P527, P186, P462, P366); category and topic words come from part-of and genre (P361, P136); associated words come from other linked entities; and description keywords are nouns drawn from the Wikidata description and the Azerbaijani Wikipedia summary. Hypernym and co-hyponym relations are treated as easy, part/property and category as medium, and association and description keywords as hard.

A multi-stage refinement then runs. Labels are normalized into clean game words (disambiguators are stripped, casing is corrected with a locale-aware lowercaser that handles the Azerbaijani dotted/dotless I pairs). A filter removes noise, non-Azerbaijani tokens, and any candidate too similar to the target — sharing a word stem, high character-trigram overlap, or high normalized Levenshtein similarity [5] — so a player cannot win with a near-copy. From the surviving candidates, the final five are selected with Maximal Marginal Relevance [4], which balances relevance against redundancy so the five words are both on-topic and diverse, with a constraint that at least three distinct relation types appear when possible.

Formally, the filter rejects a candidate w for a target t if stem(w) = stem(t), or if the character-trigram Jaccard similarity J(w, t) ≥ 0.65, or if the normalized Levenshtein similarity L(w, t) ≥ 0.85. From the remaining pool C, words are added to the selected set S one at a time by the Maximal Marginal Relevance rule:

w* = arg max  [ λ · rel(w, t) − (1 − λ) · max  sim(w, v) ],  w ∈ C \ S,  v ∈ S

where rel(w, t) is the relevance of a candidate to the target (the relation prior of Table 2, optionally combined with embedding cosine similarity), sim(w, v) is the similarity between two candidates, and λ ∈ [0, 1] trades relevance against diversity (we use λ = 0.75). Selection is seeded, so the five-word set is deterministic and reproducible.

Wikidata is the only required data source; the pipeline produces complete cards without machine translation. To find everyday vocabulary rather than obscure species, the Wikidata sitelink count (how many language Wikipedias cover an entity) is used as a language-agnostic popularity proxy applied as a cheap server-side filter. The pipeline runs as four resumable stages — select targets, harvest candidates, build cards, report statistics and every external call is cached, so re-runs never re-hit the network. Card difficulty is assigned per word from its relation and per card by majority vote, and seeded selection makes the output deterministic.

Results and discussions

The released dataset is a full pool of 5,305 cards and a category-balanced subset of 3,444 cards spanning six categories (animal, food, human, item, location, and specialty, the last a set of professional specialisations), each card holding a unique target and exactly five forbidden words. Table 1 reports the composition and the intrinsic quality metrics of the balanced set. All quality gates pass: 0% of cards share a stem between target and a forbidden word, no card contains a duplicate word, no token fails the Azerbaijani-alphabet check, and no word comes from machine translation. The mean normalized Levenshtein similarity between a target and its forbidden words is only 0.16, and between forbidden words within a card 0.13, confirming that the words are orthographically distinct from the target and from one another.

Table 1. Composition and quality metrics of the KG-Taboo dataset (full pool and balanced set)

Because every forbidden word is recorded with the Wikidata relation and property that produced it, the dataset is fully traceable to the knowledge graph. Table 2 reports how the 17,220 forbidden-word slots of the balanced set were generated. Co-hyponymy (sibling entities under a shared parent) is the single largest source at 38.9%, followed by hypernymy (the instance-of and subclass-of relations P31, P279 and the parent-taxon relation P171) at 26.7%. Together these two taxonomic relations supply about two thirds of all clues, which is why the cards skew easy: a parent class or a sibling is a strong, recognizable hint. The remaining clues come from description keywords drawn from the Wikidata gloss and Azerbaijani Wikipedia summary (14.3%), grandparent categories and the part-of relation (11.5%), concrete properties such as material P186, has-part P527, use P366 and colour P462 (4.7%), and a tail of association properties (4.0%). No clue comes from machine translation; the knowledge graph alone is sufficient to fill every card.

Table 2. Provenance of the generated forbidden words by relation type and Wikidata property (balanced set, 17,220 word slots)

On average a generated card draws on 3.18 distinct relation types, and 98.8% of cards use at least three, so most cards combine a hypernym, one or two siblings and a property rather than relying on a single relation. Table 3 shows six representative cards, one per category, with the relation behind each clue. The card for pələng (tiger), for example, pairs the hypernym panter with the siblings amerika şiri (American lion) and bəbir (leopard) and the two colour properties ağ (white) and narıncı (orange) a mix that is on-topic yet hard to give away with a single word.

Table 3. Example cards generated from Wikidata, with the relation behind each forbidden word

Three limitations follow from the generation process. First, category balance is bounded by Wikidata coverage: location is the most populous source (1,998 cards) while food is the smallest of the main categories; expanding the animal category with two-token labels and taxon-sibling and description clues raised it from a few hundred to 985 cards, showing that thin categories can be grown but only by broadening the source classes. Second, difficulty skews easy because the richest, most reliable relations (hypernymy and co-hyponymy) are themselves the easy clues; the harder cards concentrate in the taxon-derived animal category. Third, description keywords (14.3% of clues) occasionally yield weak, inflected or function-like words such as adlanır (“is called”), and the human and specialty categories overlap because both describe professions. As a sanity check we confirmed that simple co-occurrence and embedding models (CountVectorizer and TfidfVectorizer [6], Word2Vec [7], FastText [8] with pretrained Azerbaijani vectors [9], and BERT [10]) recover sensible neighbourhoods from the generated cards, consistent with our earlier embedding analysis of serious-game data [16], indicating that the clues carry genuine semantic structure rather than noise.

Two further limitations bound the scope of the present study. First, the evaluation is intrinsic: the cards have not yet been assessed by native speakers in actual game play, and the dataset has not yet been compared head-to-head with the human-authored “Yasaq” set, so the claim that a knowledge graph can substitute for corpora is supported by provenance and quality metrics but still awaits empirical confirmation on a downstream task. Second, the approach assumes reasonably developed Wikidata coverage for the target language; its transfer to languages with sparser knowledge-graph infrastructure, and its practical and economic value for game production, remain open questions. These points define the agenda for future work, in line with recent efforts to represent low-resource languages through knowledge graphs and to generate text from graph data for under-resourced languages [18], [19], [20].

Conclusion

This work shows that Taboo game cards for a low-resource language can be generated directly from the Wikidata knowledge graph, without any human card authoring and without machine translation. By harvesting forbidden words through explicit typed relations hypernymy, co-hyponymy, part and property, category, association and description keywords — normalizing and filtering them, and selecting five with Maximal Marginal Relevance, the pipeline produces a reproducible, fully provenanced dataset of 3,444 category-balanced Azerbaijani cards over six categories (from a 5,305-card pool) that passes every intrinsic quality gate. The provenance analysis shows that two taxonomic relations, co-hyponymy and hypernymy, carry about two thirds of the clues, which both explains the generation’s strength and its easy-difficulty skew. More broadly, the work is a case study in treating knowledge graphs as a substitute for data where corpora are scarce, extending the author’s ontology- and embedding-based analysis of serious games [16], [17] from analysing content to generating it. Future work will balance difficulty across categories, reduce description-keyword noise, and compare this generated set against the human-authored Yasaq set in a downstream training experiment using the same embedding and classification pipelines.

References:

1. D. Vrandečić and M. Krötzsch, “Wikidata: a free collaborative knowledgebase,” Communications of the ACM, vol. 57, no. 10, pp. 78–85, 2014, doi: 10.1145/2629489.
2. G. A. Miller, “WordNet: a lexical database for English,” Communications of the ACM, vol. 38, no. 11, pp. 39–41, 1995, doi: 10.1145/219717.219748.
3. C. Gardent, A. Shimorina, S. Narayan, and L. Perez-Beltrachini, “The WebNLG Challenge: Generating Text from RDF Data,” in Proceedings of the 10th International Conference on Natural Language Generation (INLG), 2017, pp. 124–133, doi: 10.18653/v1/W17-3518.
4. J. Carbonell and J. Goldstein, “The use of MMR, diversity-based reranking for reordering documents and producing summaries,” in Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 1998, pp. 335–336, doi: 10.1145/290941.291025.
5. V. I. Levenshtein, “Binary codes capable of correcting deletions, insertions, and reversals,” Soviet Physics Doklady, vol. 10, no. 8, pp. 707–710, 1966.
6. F. Pedregosa et al., “Scikit-learn: Machine Learning in Python,” Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011.
7. T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient estimation of word representations in vector space,” in Workshop Track Proceedings of ICLR, 2013. [Online]. Available: http://arxiv.org/abs/1301.3781.
8. P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov, “Enriching word vectors with subword information,” Transactions of the Association for Computational Linguistics, vol. 5, pp. 135–146, 2017, doi: 10.1162/tacl_a_00051.
9. E. Grave, P. Bojanowski, P. Gupta, A. Joulin, and T. Mikolov, “Learning word vectors for 157 languages,” in Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC), 2018. [Online]. Available: https://aclanthology.org/L18-1550/
10. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of NAACL-HLT, 2019, pp. 4171–4186, doi: 10.18653/v1/N19-1423.
11. S. Y. Feng, V. Gangal, J. Wei, S. Chandar, S. Vosoughi, T. Mitamura, and E. Hovy, “A survey of data augmentation approaches for NLP,” in Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, 2021, pp. 968–988, doi: 10.18653/v1/2021.findings-acl.84.
12. J. Isbarov, K. Huseynova, E. Mammadov, M. Hajili, and D. Ataman, “Open foundation models for Azerbaijani language,” in Proceedings of the First Workshop on Natural Language Processing for Turkic Languages (SIGTURK), 2024. [Online]. Available: http://arxiv.org/abs/2407.02337
13. K. Chalamalasetti, J. Götze, S. Hakimov, B. Madureira, P. Sadler, and D. Schlangen, “clembench: Using game play to evaluate chat-optimized language models as conversational agents,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023, doi: 10.18653/v1/2023.emnlp-main.689.
14. A. Mammadli, “Application of Deep Learning for Procedural Content Integration for Learning Serious Games,” Proceedings of Azerbaijan High Technical Educational Institutions, vol. 48, no. 06, pp. 455–469, Jan. 2025.
15. A. Mammadli, “Advancing Serious Games with Multilingual Deep Learning,” 6th International Boğaziçi Scientific Research Congress, pp. 1100–1106, Jan. 2025.
16. A. Mammadli, “Unlocking Educational Insights: Integrating Word2Vec Embeddings and Naive Bayes Classifier for Serious Game Data Analysis and Enhancement,” Azerbaijan Journal of High Performance Computing, vol. 6, no. 2, pp. 191–198, Dec. 2023, doi: 10.32010/26166127.2023.6.2.191.198.
17. A. Mammadli, E. Ismayilov, and C. Zanni-Merk, “Explainability of text classification through ontology-driven analysis in Serious Games,” Procedia Computer Science, vol. 246, pp. 2128–2137, 2024, doi: 10.1016/j.procs.2024.09.626.
18. S. Pan, L. Luo, Y. Wang, C. Chen, J. Wang, and X. Wu, “Unifying Large Language Models and Knowledge Graphs: A Roadmap,” IEEE Transactions on Knowledge and Data Engineering, vol. 36, no. 7, pp. 3580–3599, 2024, doi: 10.1109/TKDE.2024.3352100.
19. L. Cripwell, A. Belz, C. Gardent, A. Gatt, C. Borg, M. Borg, J. Judge, M. Lorandi, A. Nikiforovskaya, W. Soto-Martinez, and C. Thomson, “The 2023 WebNLG Shared Task on Low Resource Languages: Overview and Evaluation Results,” in Proceedings of the Workshop on Multimodal, Multilingual Natural Language Generation and Multilingual WebNLG Challenge (MM-NLG 2023), 2023, pp. 55–72. [Online]. Available: https://aclanthology.org/2023.mmnlg-1.6/
20. M. Lorandi and A. Belz, “High-quality Data-to-Text Generation for Severely Under-Resourced Languages with Out-of-the-box Large Language Models,” in Findings of the Association for Computational Linguistics: EACL 2024, 2024. [Online]. Available: http://arxiv.org/abs/2402.12267
21. D. Koyyalagunta, A. Sun, R. L. Draelos, and C. Rudin, “Playing Codenames with Language Graphs and Word Embeddings,” Journal of Artificial Intelligence Research, vol. 71, pp. 319–346, 2021, doi: 10.1613/jair.1.12665.
22. M. Stephenson, M. Sidji, and B. Ronval, “Codenames as a Benchmark for Large Language Models,” 2024. [Online]. Available: http://arxiv.org/abs/2412.11373