Abstract
Manual Requirement Engineering (RE) in Agile software development creates a significant bottleneck. The reliance on natural language user stories at scale results in high-volume backlogs prone to ambiguity, duplication, and incompleteness, leading to costly, downstream development defects. This research aims to design, develop, and empirically validate a novel, hybrid Natural Language Processing (NLP) framework, termed the Agile Requirement Quality (ARQ) framework, to automate the detection of these common requirement defects. The goal is to reduce cognitive load and improve defect detection velocity during backlog refinement. A mixed-methods Design Science Research (DSR) methodology was employed. We developed the ARQ artifact (a hybrid BERT and heuristic model) and validated it both in-vitro against a 5,000-story “gold standard” annotated corpus (Fleiss’ Kappa 0.86) and in-situ through a quasi-experiment with professional Agile teams. The findings demonstrate high efficacy. In-vitro validation achieved high accuracy (overall 95.2%, with F1-scores of 0.87 for ambiguity and 0.94 for duplication). The in-situ experiment was conclusive: the ARQ-assisted team achieved a 73% increase in defect detection and an 87.5% reduction in “defect leakage” compared to the control team, registering high usability (88.5 SUS). This study provides robust empirical evidence that NLP-driven automation is a viable, high-impact strategy for mitigating risk in Agile RE. The framework functions as a practical “augmented intelligence” tool, significantly reducing defect leakage and improving quality assurance velocity.