如何在 Bert 分类中获得预测准确度分数

Question

我正在为我的 Chatbot 项目使用 Bert 分类器。我对传入的文本消息执行必要的分词器操作。然后我将其插入模型并进行预测。我怎样才能得到这个估计的准确性？

for text in test_texts:
    encoded_dict = tokenizer.encode_plus(
        text,
        add_special_tokens=True,
        max_length=max_len,
        pad_to_max_length=True,
        return_attention_mask=True,
        return_tensors='pt',
    )

    input_ids.append(encoded_dict['input_ids'])
    attention_masks.append(encoded_dict['attention_mask'])

input_ids = torch.cat(input_ids, dim=0)
attention_masks = torch.cat(attention_masks, dim=0)
print("input_ids ",input_ids)
print("attention_masks ",attention_masks)

batch_size = 32

prediction_data = TensorDataset(input_ids, attention_masks)
prediction_sampler = SequentialSampler(prediction_data)
prediction_dataloader = DataLoader(prediction_data, sampler=prediction_sampler, batch_size=batch_size)

print("prediction_data ",prediction_data)
print("prediction_sampler ",prediction_sampler)
print("prediction_dataloader ",prediction_dataloader)
model.eval()
predictions, true_labels = [], []

for batch in prediction_dataloader:
    batch = tuple(t.to(device) for t in batch)
    b_input_ids, b_input_mask = batch
    print("b input ids",b_input_ids)
    with torch.no_grad():
        outputs = model(b_input_ids, token_type_ids=None,
                        attention_mask=b_input_mask.to(device))

    logits = outputs[0]
    logits = logits.detach().cpu().numpy()
    label_ids = b_input_mask.to('cpu').numpy()

    predictions.append(logits)
    true_labels.append(label_ids)
    print("logits ",logits)

    print("label_ids ",label_ids)
    print("true_labels ",true_labels)

print('Prediction completed')

prediction_set = []

for i in range(len(true_labels)):
    pred_labels_i = np.argmax(predictions[i], axis=1).flatten()
    prediction_set.append(pred_labels_i)

prediction= [item for sublist in prediction_set for item in sublist]


print("prediction:", prediction[0])

我正在寻找百分比值。将根据此百分比值的结果响应或通过。

Answer 1

可以使用一些库直接计算准确度。

例如，你可以使用sklearn:

from sklearn.metrics import accuracy_score
print("Accuracy:", accuracy_score(true_labels, predictions)) # Value between 0 and 1

print("Accuracy Percentage {} %:".format(100*accuracy_score(true_labels, predictions))) # Value between 0 and 100
``

如何在 Bert 分类中获得预测准确度分数

How is get predict accuracy score in Bert Classification

python

machine-learning

pytorch

bert-language-model