Fixing ill-formed UTF-16 strings with SIMD instructions [r-libre/4197]

Clausecker, Robert et Lemire, Daniel ORCID: https://orcid.org/0000-0003-3306-6922 (sous presse). Fixing ill-formed UTF-16 strings with SIMD instructions. Software: Practice and Experience. 10.1002/spe.70105

Fichier(s) associé(s) à ce document :

PDF - fixutf16_2026-5.pdf
Contenu du fichier : Manuscrit soumis (avant évaluation)
Licence : Creative Commons CC BY.

Catégorie de document :	Articles de revues
Évaluation par un comité de lecture :	Oui
Étape de publication :	Accepté (sous presse)
Résumé :	UTF-16 is a widely used Unicode encoding representing characters with one or two 16-bit code units. The format relies on surrogate pairs to encode characters beyond the Basic Multilingual Plane, requiring a high surrogate followed by a low surrogate. Ill-formed UTF-16 strings---where surrogates are mismatched---can arise from data corruption or improper encoding, posing security and reliability risks. Consequently, programming languages such as JavaScript include functions to fix ill-formed UTF-16 strings by replacing mismatched surrogates with the Unicode replacement character (U+FFFD). We propose using Single Instruction, Multiple Data (SIMD) instructions to handle multiple code units in parallel, enabling faster and more efficient execution. Our software is part of the Google JavaScript engine (V8) and thus part of several major Web browsers.
Déposant:	Lemire, Daniel
Responsable :	Daniel Lemire
Dépôt :	18 aout 2026 15:58
Dernière modification :	18 aout 2026 15:58

RÉVISER