Staging
v0.5.1
https://github.com/python/cpython
Revision e662c398d87f136497f8ec672e83657ae3a599e0 authored by Victor Stinner on 01 November 2020, 22:07:23 UTC, committed by GitHub on 01 November 2020, 22:07:23 UTC
If the nl_langinfo(CODESET) function returns an empty string, Python
now uses UTF-8 as the filesystem encoding.

In May 2010 (commit b744ba1d14c5487576c95d0311e357b707600b47), I
modified Python to log a warning and use UTF-8 as the filesystem
encoding (instead of None) if nl_langinfo(CODESET) returns an empty
string.

In August 2020 (commit 94908bbc1503df830d1d615e7b57744ae1b41079), I
modified Python startup to fail with a fatal error and a specific
error message if nl_langinfo(CODESET) returns an empty string. The
intent was to prevent guessing the encoding and also investigate user
configuration where this case happens.

In 10 years (2010 to 2020), I saw zero user report about the error
message related to nl_langinfo(CODESET) returning an empty string.

Today, UTF-8 became the defacto standard and it's safe to make the
assumption that the user expects UTF-8. For example,
nl_langinfo(CODESET) can return an empty string on macOS if the
LC_CTYPE locale is not supported, and UTF-8 is the default encoding
on macOS.

While this change is likely to not affect anyone in practice, it
should make UTF-8 lover happy ;-)

Rewrite also the documentation explaining how Python selects the
filesystem encoding and error handler.
1 parent 82458b6
History
Tip revision: e662c398d87f136497f8ec672e83657ae3a599e0 authored by Victor Stinner on 01 November 2020, 22:07:23 UTC
bpo-42236: Use UTF-8 encoding if nl_langinfo(CODESET) fails (GH-23086)
Tip revision: e662c39
File Mode Size
.azure-pipelines
.github
Doc
Grammar
Include
Lib
Mac
Misc
Modules
Objects
PC
PCbuild
Parser
Programs
Python
Tools
m4
.gitattributes -rw-r--r-- 1.9 KB
.gitignore -rw-r--r-- 1.8 KB
.travis.yml -rw-r--r-- 8.4 KB
CODE_OF_CONDUCT.md -rw-r--r-- 630 bytes
LICENSE -rw-r--r-- 13.6 KB
Makefile.pre.in -rw-r--r-- 67.8 KB
README.rst -rw-r--r-- 10.0 KB
aclocal.m4 -rw-r--r-- 15.2 KB
config.guess -rwxr-xr-x 43.1 KB
config.sub -rwxr-xr-x 35.4 KB
configure -rwxr-xr-x 501.3 KB
configure.ac -rw-r--r-- 169.3 KB
install-sh -rwxr-xr-x 15.0 KB
netlify.toml -rw-r--r-- 82 bytes
pyconfig.h.in -rw-r--r-- 44.6 KB
setup.py -rw-r--r-- 109.0 KB

README.rst

back to top