QUALITY-AWARE POSE–HAND KEYPOINT EXTRACTION PIPELINE FOR SKELETON-BASED SIGN LANGUAGE RECOGNITION

D. Akbarov, Talgat Sembayev, A. Kydyrbekova

2026

Full text Open access: Hybrid Metadata (CSL-JSON)
DOI 10.32014/2026.2518-1726.426
Data confidence 70 medium
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

Skeleton-based sign language recognition (SLR) uses pose and hand keypoints as a compact, background-invariant representation, but in real signing videos the keypoint stream produced by off-the-shelf estimators is often unreliable. Hand trajectories frequently contain missing detections, left/right identity swaps, duplicate outputs, temporal jitter, and occlusion-related drift. These artifacts are not merely preprocessing noise: they directly degrade the quality of downstream SLR training. This paper presents a quality-aware pose–hand keypoint extraction pipeline that treats keypoint generation as an auditable, reliability-aware stage rather than a hidden preprocessing step. The proposed method combines: (i) pose-guided left/right hand assignment with spatial gating, (ii) pose-conditioned second-pass region-of-interest (ROI) refinement for low-confidence or missing hands, (iii) lightweight short-gap temporal bridging (tracking/hold), (iv) explicit occlusion-consistent labeling, (v) plausibility checks to reject physically implausible configurations, and (vi) anchor-preserving postprocessing for short-gap filling and jitter reduction. Beyond cleaned keypoints, the pipeline exports per-hand reliability semantics (provenance and state) and a dataset-ready quality report based on proxy metrics, including coverage, confidence, recovery fractions, intervention rates, and temporal stability indicators. A fixed composite quality score is also defined to support reproducible ranking and filtering without manually annotated hand landmarks. Dataset-scale evaluation on Slovo (n = 20,400 clips) shows that the proposed full configuration substantially improves extraction reliability over a pass1-only baseline: the composite quality score increases from 0.684 to 0.794, any-hand coverage from 0.889 to 0.969, and both-hands coverage from 0.454 to 0.646, while MissingRate decreases from 0.329 to 0.193. Recovery analysis further shows that most robustness gains come from ROI-based second-pass recovery, with tracking acting mainly as a sparse fallback mechanism. The pipeline is designed for reproducible keypoint export, dataset curation, and reliability-aware SLR training.