Development and Experimental Evaluation of Models for Infrastructure-Domain Classification and Related-Complaint Retrieval in Citizen Complaints

Daniyar Rakhimzhanov, Aigul Adamova, Aigerim Mansurova, Aiganym Mansurova, Dana Satvaldina

2026

Full text Open access: Gold Metadata (CSL-JSON)
DOI 10.3390/info17100995
Data confidence 70 medium
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

Citizen complaint platforms generate large volumes of multilingual textual data that can support urban service management, yet conventional text-based representations typically analyze each complaint independently and do not explicitly capture relationships with the surrounding urban context. This study investigates whether relational information can complement semantic text representations for the thematic analysis of citizen complaints. A multi-view heterogeneous graph was constructed from municipal requests collected through the iKOMEK109 service in Astana, Kazakhstan, integrating complaint texts with annotated urban entities, spatial and contextual entity relations, and semantic complaint–complaint relations. Entity, Spatial, and Semantic graph views were evaluated using R-GCN, HGT, and an attention-based multi-view R-GCN configuration, with BGE-M3 embeddings and Logistic Regression serving as the text-only baseline. The representations were evaluated for infrastructure-domain classification and related-complaint retrieval. The bidirectional Multi-View R-GCN achieved a mean Macro F1-score of 0.7615 ± 0.0221 across three model-initialization seeds on the reference partition. However, a non-graph baseline combining complaint-text and directly linked entity embeddings achieved a Macro F1-score of 0.7810 on the same partition, compared with 0.7220 for the text-only baseline and 0.7381 for the R-GCN in the fixed-seed comparison, indicating that part of the observed benefit arises from additional entity information rather than graph propagation alone. Repeated-split analysis across five stratified data partitions further showed that the relative performance differences were sensitive to data partitioning. Under fine-grained human relevance assessment, BGE-M3 and the Hybrid representation achieved comparable retrieval performance. The corrected two-hop Hybrid representation obtained MRR@10 = 0.7596 compared with 0.7628 for BGE-M3, while producing small numerical improvements in Precision and NDCG. Query-level paired bootstrap confidence intervals included zero for all evaluated metrics, and paired permutation tests found no statistically significant differences, indicating that the graph-derived signal did not provide a statistically reliable retrieval advantage over the text representation.