SPACY 自定义 NER 在加载训练模型后不预测任何实体

问题描述 投票:0回答:0

正如您在下面看到的是我的代码,经过模型训练后我加载了保存的模型但是如果我将测试数据提供给保存的模型,模型没有预测任何实体,我在输出中得到空白实体列表请帮助我。

==============================关于spaCy的信息================ ==============

spaCy 版本 3.5.1
位置 C:\Users
平台 Windows-10-10.0.19044-SP0
Python 版本 3.11.2
管道 en_core_web_sm (3.5.0)

############################################## #############

While training getting below loss values:
Iteration 0: Losses={'ner': 10061.062388668392}
Iteration 1: Losses={'ner': 6720.3977553011755}
Iteration 2: Losses={'ner': 6057.3629679099995}
Iteration 3: Losses={'ner': 5867.247623759959}
Iteration 4: Losses={'ner': 5778.171307897844}
Iteration 5: Losses={'ner': 5320.552820883172}
Iteration 6: Losses={'ner': 5262.6456313436065}
Iteration 7: Losses={'ner': 5134.586389105264}
Iteration 8: Losses={'ner': 5110.263808832533}
Iteration 9: Losses={'ner': 5039.2966978069635}
Iteration 10: Losses={'ner': 4966.384003252385}
Iteration 11: Losses={'ner': 4856.636463489224}
Iteration 12: Losses={'ner': 4779.732222652631}
Iteration 13: Losses={'ner': 4755.5519985695455}
Iteration 14: Losses={'ner': 4678.259963573367}
Iteration 15: Losses={'ner': 4597.943225886264}
Iteration 16: Losses={'ner': 4695.007055523597}
Iteration 17: Losses={'ner': 4594.813370778378}
Iteration 18: Losses={'ner': 4476.687746926943}
Iteration 19: Losses={'ner': 4495.782225101342}
Iteration 20: Losses={'ner': 4440.587775479399}
Iteration 21: Losses={'ner': 4443.889482193466}

  

 model = None
    output_dir=Path("./")
    n_iter=100


if model is not None:
    nlp = spacy.load(model)  
    print("Loaded model '%s'" % model)
else:
    nlp = spacy.blank('en')  
    print("Created blank 'en' model")

if 'ner' not in nlp.pipe_names:
    ner = nlp.create_pipe('ner')
    nlp.add_pipe('ner', last=True)
else:
    ner = nlp.get_pipe('ner')


    

from spacy.training import Example
from spacy.util import minibatch,compounding

# Convert (text, annotation) tuples to Example objects
examples = []
for text, annotations in output_list:
    example = Example.from_dict(nlp.make_doc(text), annotations)
    examples.append(example)

# Train the NER model
other_pipes = [pipe for pipe in nlp.pipe_names if pipe != 'ner']
with nlp.disable_pipes(*other_pipes):
    optimizer = nlp.begin_training()
    for itn in range(n_iter):
        losses = {}
        random.shuffle(examples)
        batches = minibatch(examples, size=compounding(4.0, 32.0, 1.001))
        for batch in batches:
            nlp.update(batch, sgd=optimizer, drop=0.5, losses=losses)
        print(f"Iteration {itn}: Losses={losses}")


nlp.to_disk("model--best")


nlp2 = spacy.load("model--best")
for text, _ in VAL_DATA:
    doc = nlp2(text)
    print('Entities', [(ent.text, ent.label_) for ent in doc.ents])
#     print('Tokens', [(t.text, t.ent_type_, t.ent_iob) for t in doc])
python deep-learning nlp spacy named-entity-recognition
© www.soinside.com 2019 - 2024. All rights reserved.