Privacy-Preserving Speaker Verification in End-to-End Encrypted Chats: A Feasibility Study

Mamyr Altaibek, Altanbek Zulkhazhav, Banu Yergesh, Gulmira Bekmanova, Assel Omarbekova, Altynbek Sharipbay

IEEE Access, 2025

Full text Open access: Gold Metadata (CSL-JSON)
DOI 10.1109/access.2025.3648046
Data confidence 85 high
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

We present a deployment-oriented, privacy-preserving speaker-verification pipeline purpose-built for end-to-end encrypted chats. All nonlinear acoustic modeling and embedding extraction run on the user device, while the cloud performs only fixed-point cosine matching and a single threshold comparison via secure multiparty computation, so that the verifier learns only a one-bit accept/reject decision. Integer templates are rendered cancelable with per-tenant salts, enabling revocation and cross-realm unlinkability throughout the template lifecycle. To align evaluation with deployment arithmetic, we introduce a decision-only calibration procedure that operates in the same fixed-point regime as inference; it uses isotonic regression with binomial-uncertainty intervals to select operating points without exposing scores. Empirically, the system yields accuracy comparable to its plaintext baseline at matched operating points on standard VoxCeleb protocols, maintains stable operating characteristics under a unified tie policy across floating-point, fixed-point, and multiparty computation instantiations, and exhibits latency and communication footprints compatible with interactive use. The design further generalizes across conditions (e.g., duration and noise) and to a low-resource language. We release trial lists, per-threshold tables, and code to support independent audits. Overall, this work should be read as a feasibility study rather than a ready-to-deploy product: our goal is to surface the trade-offs and limitations of privacy-preserving speaker verification in realistic encrypted messaging environments, not to claim production-grade performance. Within this feasibility scope, our contribution is an engineering integration under end-to-end encryption and MPC constraints: (i) a practical split-inference architecture that keeps all non-linear acoustics and embeddings on device; (ii) a fixed-point, decision-only calibration pipeline that is numerically aligned with the secure deployment regime; and (iii) a revocable, unlinkable template mechanism instantiated via per-tenant salts and orthonormal transforms, consistent with ISO/IEC 24745. In contrast to HE-based schemes that homomorphically evaluate large parts of the embedding network or PLDA-based MPC designs that assume server-side access to embeddings, our verifier outsources only cosine-family matching and a single threshold comparison under MPC, never exposing raw audio, float embeddings, or plaintext templates. Together, these elements offer a pragmatic path to piloting privacy-preserving speaker verification in controlled encrypted channels without relaxing end-to-end encryption guarantees.