使用特定python库时的UnicodeDecodeError(性别检测器)

问题描述 投票:0回答:1

我需要进行性别猜测以进行一些分析,经过一些研究后我在github上找到了这个Python库:malev/gender-detector

遵循说明并做一些调整(例如readme指示import gender_detector as gd但我需要做from gender_detector import gender_detector as gd

然后发生这种情况,lib有4个数据集,'us','uk','ar','uy',但只有在使用'us'或'uk'时才有效

见下面的例子:

from gender_detector import gender_detector as gd
detector = gd.GenderDetector('us')
detector2 = gd.GenderDetector('ar')

detector.guess('Marcos')
Out[25]: 'male'

detector2.guess('Marcos')
Traceback (most recent call last):

File "", line 1, in 
detector2.guess('Marcos')

File "/home/cpneto/anaconda3/lib/python3.6/site-packages/gender_detector/gender_detector.py", line 25, in guess
initial_position = self.index(name[0])

File "/home/cpneto/anaconda3/lib/python3.6/site-packages/gender_detector/index.py", line 19, in call
self._generate_index()

File "/home/cpneto/anaconda3/lib/python3.6/site-packages/gender_detector/index.py", line 25, in _generate_index
total = file.readline() # Omit headers line

File "/home/cpneto/anaconda3/lib/python3.6/codecs.py", line 321, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf1 in position 1078: invalid continuation byte

我相信这是因为py2与py3的兼容性,但我不确定,并且没有任何关于如何解决这个问题的线索。

有什么建议?

python python-unicode
1个回答
0
投票

该库假设您的ar文件是UTF-8编码,但它不是(因此byte 0xf1 in position 1078错误)。您需要将文件转换为UTF-8或找到一些方法将实际编码传递给库。

© www.soinside.com 2019 - 2024. All rights reserved.