使用转换器 class BertForQuestionAnswering 进行抽取式问答

Question

我将 BERT 模型与 transformers class 库 BertForQuestionAnswering 一起用于提取 QA 任务。提取式问答是针对给定上下文文本回答问题并输出答案在上下文中匹配位置的开始和结束索引的任务。 My code 如下：

model = BertForQuestionAnswering.from_pretrained('bert-base-uncased',
    cache_dir=os.getenv("cache_dir", "../../models"))
question = "What is the capital of Italy?"
text = "The capital of Italy is Rome."
inputs = tokenizer.encode_plus(question, text, return_tensors='pt')
start, end = model(**inputs)
start_max = torch.argmax(F.softmax(start, dim = -1))
end_max = torch.argmax(F.softmax(end, dim = -1)) + 1 ## add one ##because of python list indexing
answer = tokenizer.decode(inputs["input_ids"][0][start_max : end_max])
print(answer)

我收到这个错误

start_max = torch.argmax(F.softmax(start, dim = -1))
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/functional.py", line 1583, in softmax
    ret = input.softmax(dim)
AttributeError: 'str' object has no attribute 'softmax'

这个方法我也试过，略有不同

encoding = tokenizer.encode_plus(text=question,text_pair=text, add_special=True)
inputs = encoding['input_ids']  #Token embeddings
sentence_embedding = encoding['token_type_ids']  #Segment embeddings
tokens = tokenizer.convert_ids_to_tokens(inputs) #input tokens
start_scores, end_scores = model(input_ids=torch.tensor([inputs]), token_type_ids=torch.tensor([sentence_embedding]))
start_index = torch.argmax(start_scores)
end_index = torch.argmax(end_scores)
answer = ' '.join(tokens[start_index:end_index+1])

但错误可能是相同的：

    start_index = torch.argmax(start_scores)
TypeError: argmax(): argument 'input' (position 1) must be Tensor, not str

我假设由于输出的解包为

start, end = model(**inputs)

如果是这样，如何正确解压该模型的输出？

Answer 1

由于版本更新，模型returns是一个字典，而不是开始、结束的元组。您可以添加以下参数： return_dict=False

使用转换器 class BertForQuestionAnswering 进行抽取式问答

Using transformers class BertForQuestionAnswering for Extractive Question Answering

python

bert-language-model

huggingface-transformers