这是我的code.it从excel文件(rev列)读取评论并列出列表。
xp是这样的
["['intrepid', 'bumbling', 'duo', 'deliver', 'good', 'one'],['better', 'offering', 'considerable', 'cv', 'freshly', 'qualified', 'private', 'investigator', 'thrust', 'murder', 'investigation', 'invisible'],[ 'man', 'alone', 'tell', 'fun', 'flow', 'decent', 'clip', 'need', 'say', 'sequence', 'comedy', 'gold', 'like', 'scene', 'restaurant', 'excellent', 'costello', 'pretending', 'work', 'ball', 'gym', 'final', 'reel']"]
但是当使用list for model时,它给出了错误“TypeError:'float'对象不可迭代”。我不知道我的问题在哪里。谢谢。
xp=[]
import gensim
import logging
import pandas as pd
file = r'FileNamelast.xlsx'
df = pd.read_excel(file,sheet_name='FileNamex')
pages = [i for i in range(0,1000)]
for page in pages:
text =df.loc[page,["rev"]]
xp.append(text[0])
model = gensim.models.Word2Vec (xp, size=150, window=10, min_count=2,
workers=10)
model.train(xp,total_examples=len(xp),epochs=10)
这就是我得到的.TypeError:'float'对象不可迭代
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-32-aa34c0e432bf> in <module>()
14
15
---> 16 model = gensim.models.Word2Vec (xp, size=150, window=10, min_count=2, workers=10)
17 model.train(xp,total_examples=len(xp),epochs=10)
C:\ProgramData\Anaconda3\lib\site-packages\gensim\models\word2vec.py in __init__(self, sentences, corpus_file, size, alpha, window, min_count, max_vocab_size, sample, seed, workers, min_alpha, sg, hs, negative, ns_exponent, cbow_mean, hashfxn, iter, null_word, trim_rule, sorted_vocab, batch_words, compute_loss, callbacks, max_final_vocab)
765 callbacks=callbacks, batch_words=batch_words, trim_rule=trim_rule, sg=sg, alpha=alpha, window=window,
766 seed=seed, hs=hs, negative=negative, cbow_mean=cbow_mean, min_alpha=min_alpha, compute_loss=compute_loss,
--> 767 fast_version=FAST_VERSION)
768
769 def _do_train_epoch(self, corpus_file, thread_id, offset, cython_vocab, thread_private_mem, cur_epoch,
C:\ProgramData\Anaconda3\lib\site-packages\gensim\models\base_any2vec.py in __init__(self, sentences, corpus_file, workers, vector_size, epochs, callbacks, batch_words, trim_rule, sg, alpha, window, seed, hs, negative, ns_exponent, cbow_mean, min_alpha, compute_loss, fast_version, **kwargs)
757 raise TypeError("You can't pass a generator as the sentences argument. Try an iterator.")
758
--> 759 self.build_vocab(sentences=sentences, corpus_file=corpus_file, trim_rule=trim_rule)
760 self.train(
761 sentences=sentences, corpus_file=corpus_file, total_examples=self.corpus_count,
C:\ProgramData\Anaconda3\lib\site-packages\gensim\models\base_any2vec.py in build_vocab(self, sentences, corpus_file, update, progress_per, keep_raw_vocab, trim_rule, **kwargs)
934 """
935 total_words, corpus_count = self.vocabulary.scan_vocab(
--> 936 sentences=sentences, corpus_file=corpus_file, progress_per=progress_per, trim_rule=trim_rule)
937 self.corpus_count = corpus_count
938 self.corpus_total_words = total_words
C:\ProgramData\Anaconda3\lib\site-packages\gensim\models\word2vec.py in scan_vocab(self, sentences, corpus_file, progress_per, workers, trim_rule)
1569 sentences = LineSentence(corpus_file)
1570
-> 1571 total_words, corpus_count = self._scan_vocab(sentences, progress_per, trim_rule)
1572
1573 logger.info(
C:\ProgramData\Anaconda3\lib\site-packages\gensim\models\word2vec.py in _scan_vocab(self, sentences, progress_per, trim_rule)
1552 sentence_no, total_words, len(vocab)
1553 )
-> 1554 for word in sentence:
1555 vocab[word] += 1
1556 total_words += len(sentence)
TypeError: 'float' object is not iterable
sentences
的Word2Vec
语料库参数应该是一个可迭代的单词列表序列。
您报告的xp
值实际上是一个包含一个长字符串的列表:
[
"['intrepid', 'bumbling', 'duo', 'deliver', 'good', 'one'],['better', 'offering', 'considerable', 'cv', 'freshly', 'qualified', 'private', 'investigator', 'thrust', 'murder', 'investigation', 'invisible'],[ 'man', 'alone', 'tell', 'fun', 'flow', 'decent', 'clip', 'need', 'say', 'sequence', 'comedy', 'gold', 'like', 'scene', 'restaurant', 'excellent', 'costello', 'pretending', 'work', 'ball', 'gym', 'final', 'reel']"
]
我不知道这会给你报告的错误,但这肯定是错的,所以应该修复。您应该在实例化xp
之前打印Word2Vec
,以确保您知道它包含的内容。
一个真正的列表,每个项目都是一个字符串标记列表,可以工作。因此,如果xp
是以下是正确的:
[
['intrepid', 'bumbling', 'duo', 'deliver', 'good', 'one'],
['better', 'offering', 'considerable', 'cv', 'freshly', 'qualified', 'private', 'investigator', 'thrust', 'murder', 'investigation', 'invisible'],
[ 'man', 'alone', 'tell', 'fun', 'flow', 'decent', 'clip', 'need', 'say', 'sequence', 'comedy', 'gold', 'like', 'scene', 'restaurant', 'excellent', 'costello', 'pretending', 'work', 'ball', 'gym', 'final', 'reel']
]
但请注意:
Word2Vec
对玩具大小的数据集表现不佳。因此,虽然这个微小的设置可能有助于检查基本的语法/格式问题,但是在您使用数十万个单词进行训练之前,不要期望得到真实的结果。train()
。该模型将自动执行所有步骤。 (另一方面,如果您不提供语料库,则必须同时调用build_vocab()
和train()
。)如果在INFO级别启用日志记录,幕后发生的所有步骤将更加清晰。