Linux系统更新后第三方Python文件出现Python UnicodeDecodeError

Question

我的 Ubuntu 机器最近进行系统更新后开始遇到问题。很多第三方Python文件突然抛出这样的UnicodeDecodeErrors：

ERROR : startup/gui/menus.py : Error loading Arnold module - "'utf-8' codec can't decode byte 0xa0 in position 0: invalid start byte".
 Traceback (most recent call last):
  File "/home/julius/bin/gaffer-1.3.2.0-linux/startup/gui/menus.py", line 123, in <module>
    import GafferArnoldUI
  File "/home/julius/bin/gaffer-1.3.2.0-linux/lib/python3.7/site-packages/shiboken2/files.dir/shibokensupport/feature.py", line 139, in _import
    return original_import(name, *args, **kwargs)
  File "/home/julius/bin/gaffer-1.3.2.0-linux/arnold/7.2/python/GafferArnoldUI/__init__.py", line 41, in <module>
    from . import ArnoldShaderUI
  File "/home/julius/bin/gaffer-1.3.2.0-linux/lib/python3.7/site-packages/shiboken2/files.dir/shibokensupport/feature.py", line 139, in _import
    return original_import(name, *args, **kwargs)
  File "/home/julius/bin/gaffer-1.3.2.0-linux/arnold/7.2/python/GafferArnoldUI/ArnoldShaderUI.py", line 400, in <module>
    __translateNodeMetadata( arnold.AiNodeEntryIteratorGetNext( nodeIt ) )
  File "/home/julius/bin/gaffer-1.3.2.0-linux/arnold/7.2/python/GafferArnoldUI/ArnoldShaderUI.py", line 255, in __translateNodeMetadata
    presetValues = __enumPresetValues( param )
  File "/home/julius/bin/gaffer-1.3.2.0-linux/arnold/7.2/python/GafferArnoldUI/ArnoldShaderUI.py", line 132, in __enumPresetValues
    preset = arnold.AiEnumGetString( enum, len( presets ) )
  File "/opt/Autodesk/arnold/7.2.3.2/python/arnold/ai_enum.py", line 22, in AiEnumGetString
    return AtPythonStringToStr(_AiEnumGetString(enum_type, index))
  File "/opt/Autodesk/arnold/7.2.3.2/python/arnold/ai_types.py", line 36, in AtPythonStringToStr
    return str(astring)
  File "/opt/Autodesk/arnold/7.2.3.2/python/arnold/ai_types.py", line 28, in __str__
    return self.decode()
  File "/opt/Autodesk/arnold/7.2.3.2/python/arnold/ai_types.py", line 33, in decode
    return self.value.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 0: invalid start byte

以下是停止工作的第三方应用程序的示例： https://github.com/gafferHQ/gaffer https://arnoldrenderer.com/ 这些应用程序包含许多 python 文件，因此编辑源文件对我来说并不是一个真正的选择，特别是因为它们在系统更新之前工作得很好。

我最初的感觉是我系统上的区域设置混乱了。我尝试将它们重置为出厂默认设置并重新启动，但无济于事：

> locale
LANG=en_US.UTF-8
LANGUAGE=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

我的Python版本指向这个：

> ll $(which python)
lrwxrwxrwx 1 root root 24 Sep  9 15:23 /usr/bin/python -> /etc/alternatives/python

> ll /etc/alternatives/python
lrwxrwxrwx 1 root root 19 Sep  9 15:23 /etc/alternatives/python -> /usr/bin/python3.10

不幸的是，我有点被困在这里，因为我不知道要注意什么。任何指示将不胜感激。

Answer 1

Verify Python Version:
Ensure that the Python version aligns with your project requirements and dependencies. Run:


python --version

Consider using a virtual environment to encapsulate project dependencies.

2. Confirm Locale Settings:
Check the system locale settings:


locale

Ensure that the selected locale supports the desired character encoding. Adjust settings if necessary using:


sudo dpkg-reconfigure locales

3. Inspect File Encoding:
Examine the encoding of problematic Python files:


file your_file.py

Ensure that the detected encoding is compatible with your Python version.

4. Text Editor Encoding:
Open the files in a text editor that allows specifying encoding (e.g., UTF-8). Ensure files are saved with the appropriate encoding.

5. Encoding Declaration:
Include the following encoding declaration at the top of Python files:

python

# -*- coding: utf-8 -*-

6. Dependency Management:
Update project dependencies, especially if they interact with file handling or encodings:


pip install --upgrade your_dependency

7. Virtual Environment:
Consider utilizing a virtual environment to isolate project dependencies and configurations. Create and activate a virtual environment:


python -m venv venv
source venv/bin/activate

8. Review System Updates:
Examine release notes or documentation for recent Linux updates. Identify changes that might impact Python or file encodings.

9. Debugging:
Introduce debug statements in problematic areas:


try:
    # Your code causing the error
except UnicodeDecodeError as e:
    print(f"UnicodeDecodeError: {e}")
    print(f"File path: {your_file_path}")
    print(f"Problematic line: {problematic_line}")
Utilize this information for more granular diagnostics.