When str.lower() is a security vulnerability in Python – Seth Larson

Aug 26, 2026 03:49 AM - 2 hours ago 1

Some net standards only support ASCII characters, but the world uses much much than the Latin alphabet. Thus, a mapping from Unicode to ASCII for usage successful domain names is required.

NamePrep was portion of that solution, defined successful RFC 3491 arsenic a floor plan of StringPrep, and is crucially a constituent of Internationalizing Domain Names successful Applications (IDNA), besides known arsenic “IDNA 2003”. The StringPrep algorithm is defined in RFC 3454. IDNA 2003 has been obsoleted by IDNA 2008 defined successful RFC 5890, 5891, 5892, and 5893.

Python supports IDNA 2003 done the idna codec (str.encode('idna')) and IDNA 2008 is supported by the idna package connected the Python package Index. Python's implementation of StringPrep is implemented successful the stringprep module in the modular library. In general, you should beryllium utilizing the idna package (IDNA 2008) and not .encode("idna") (IDNA 2003), but sometimes you do request the older behavior.

StringPrep defines the “case folding” measurement (case folding is astir “how to lowercase/uppercase a codepoint”) successful Section 3.2, enabling case-insensitive comparisons of strings, by mapping each characters done mapping tables B.2 and B.3. B.2 is efficaciously str.lower(), lowercasing each characters according to Unicode rules and B.3 contains the exceptions. The Python codification implementing this (and assuming B.3 array is captured correctly) is the pursuing codification below:

def map_table_b3(code): r = b3_exceptions.get(ord(code)) if r is not None: return r return code.lower()

And that mightiness look fine... and the title astir apt gave it distant already. The str.lower() telephone successful this usability is simply a vulnerability!

Why? Because str uses immoderate Unicode information that the peculiar Python interpreter is shipped with, you tin fig retired what Unicode type your Python expert uses by accessing unicodedata.unidata_version:

>>> import unicodedata >>> unicodedata.unidata_version '17.0.0'

There's besides a database of Unicode 3.2.0 information disposable connected each version of Python (unicodedata.ucd_3_2_0) specifically for the StringPrep and IDNA algorithms:

$ grep -I "ucd_3_2_0" -R Lib/ Lib/stringprep.py:from unicodedata import ucd_3_2_0 arsenic unicodedata Lib/encodings/idna.py:from unicodedata import ucd_3_2_0 arsenic unicodedata

This is important! StringPrep depends connected this circumstantial type of Unicode to run consistently, the B.2 and B.3 tables successful RFC 3454 are essentially Unicode 3.2.0 case-folding rules encoded into a table. So we request to usage Unicode 3.2.0 case-folding rules, not newer Unicode case-folding rules. This is why calling str.lower() represents a quality successful the implementation and the specification, and truthful a vulnerability:

# RFC 3454 compliant worth ('Ꭰ' is U+13A0) >>> "ᎠᎠ".encode("idna") 'xn--58da' # Value if utilizing Unicode 17.0.0 case-folding >>> "ᎠᎠ".encode("idna") 'xn--kz9aa'

The hole was to create caller exceptions truthful that str.lower() would behave as if it was utilizing Unicode 3.2.0 for only peculiar function. So, we go done each Unicode codepoint and grounds erstwhile the behaviour of str.lower() is different erstwhile comparing the Unicode type shipped pinch Python and Unicode 3.2.0. And that's all, now IDNA 2003 is accordant pinch the specification.

Thanks to Bitshift for reporting the vulnerability, Stan Ulbrych for co-developing the remediation, and Marc-Andre Lemburg and Petr Viktorin for reviewing the remediation. See CVE-2026-17084 for much details.

My activity arsenic the Security Developer-in-Residence astatine the Python Software Foundation is sponsored by Alpha-Omega. Thanks to Alpha-Omega for supporting information successful the Python ecosystem.

Wow, you made it to the end!