Bert Trainer 实例中的提前停止
Early stopping in Bert Trainer instances
我正在为多类分类任务微调 BERT 模型。我的问题是我不知道如何为那些 Trainer 实例添加“提前停止”。有什么想法吗?
在正确使用 EarlyStoppingCallback()
之前,您需要进行一些修改
from transformers import EarlyStoppingCallback
...
...
# Defining the TrainingArguments() arguments
args = TrainingArguments(
f"training_with_callbacks",
evaluation_strategy ='steps',
eval_steps = 50, # Evaluation and Save happens every 50 steps
save_total_limit = 5, # Only last 5 models are saved. Older ones are deleted.
learning_rate=2e-5,
per_device_train_batch_size=batch_size,
per_device_eval_batch_size=batch_size,
num_train_epochs=5,
weight_decay=0.01,
push_to_hub=False,
metric_for_best_model = 'f1',
load_best_model_at_end=True)
您需要:
- 使用
load_best_model_at_end = True
(EarlyStoppingCallback()
要求为True
)。
evaluation_strategy
= 'steps'
而不是 'epoch'
.
eval_steps
= 50(评估 N 步后的指标)。
metric_for_best_model = 'f1'
,
在你的 Trainer()
:
trainer = Trainer(
model,
args,
...
compute_metrics=compute_metrics,
callbacks = [EarlyStoppingCallback(early_stopping_patience=3)]
)
当然,当你使用compute_metrics
时,例如它可以是这样的函数:
def compute_metrics(p):
pred, labels = p
pred = np.argmax(pred, axis=1)
accuracy = accuracy_score(y_true=labels, y_pred=pred)
recall = recall_score(y_true=labels, y_pred=pred)
precision = precision_score(y_true=labels, y_pred=pred)
f1 = f1_score(y_true=labels, y_pred=pred)
return {"accuracy": accuracy, "precision": precision, "recall": recall, "f1": f1}
compute_metrics()
的 return 应该是字典,您可以在函数内访问 want/compute 和 return.
的任何指标
我正在为多类分类任务微调 BERT 模型。我的问题是我不知道如何为那些 Trainer 实例添加“提前停止”。有什么想法吗?
在正确使用 EarlyStoppingCallback()
from transformers import EarlyStoppingCallback
...
...
# Defining the TrainingArguments() arguments
args = TrainingArguments(
f"training_with_callbacks",
evaluation_strategy ='steps',
eval_steps = 50, # Evaluation and Save happens every 50 steps
save_total_limit = 5, # Only last 5 models are saved. Older ones are deleted.
learning_rate=2e-5,
per_device_train_batch_size=batch_size,
per_device_eval_batch_size=batch_size,
num_train_epochs=5,
weight_decay=0.01,
push_to_hub=False,
metric_for_best_model = 'f1',
load_best_model_at_end=True)
您需要:
- 使用
load_best_model_at_end = True
(EarlyStoppingCallback()
要求为True
)。 evaluation_strategy
='steps'
而不是'epoch'
.eval_steps
= 50(评估 N 步后的指标)。metric_for_best_model = 'f1'
,
在你的 Trainer()
:
trainer = Trainer(
model,
args,
...
compute_metrics=compute_metrics,
callbacks = [EarlyStoppingCallback(early_stopping_patience=3)]
)
当然,当你使用compute_metrics
时,例如它可以是这样的函数:
def compute_metrics(p):
pred, labels = p
pred = np.argmax(pred, axis=1)
accuracy = accuracy_score(y_true=labels, y_pred=pred)
recall = recall_score(y_true=labels, y_pred=pred)
precision = precision_score(y_true=labels, y_pred=pred)
f1 = f1_score(y_true=labels, y_pred=pred)
return {"accuracy": accuracy, "precision": precision, "recall": recall, "f1": f1}
compute_metrics()
的 return 应该是字典,您可以在函数内访问 want/compute 和 return.