微调期间的 Bert sentence-transformers stops/quits

Question

我正在按照 BERT 说明进行微调 here

这是我的代码：

from sentence_transformers import SentenceTransformer, SentencesDataset, InputExample, losses, evaluation
from torch.utils.data import DataLoader

# load model
embedder = SentenceTransformer('bert-large-nli-mean-tokens')
print("embedder loaded...")

# define your train dataset, the dataloader, and the train loss
train_dataset = SentencesDataset(x_sample["input"].tolist(), embedder)
train_dataloader = DataLoader(train_dataset, shuffle=False, batch_size=16)
train_loss = losses.CosineSimilarityLoss(embedder)

sentences1 = ['This list contains the first column', 'With your sentences', 'You want your model to evaluate on']
sentences2 = ['Sentences contains the other column', 'The evaluator matches sentences1[i] with sentences2[i]', 'Compute the cosine similarity and compares it to scores[i]']
scores = [0.3, 0.6, 0.2]
evaluator = evaluation.EmbeddingSimilarityEvaluator(sentences1, sentences2, scores)

# tune the model
embedder.fit(train_objectives=[(train_dataloader, train_loss)], 
    epochs=1, 
    warmup_steps=100, 
    evaluator=evaluator, 
    evaluation_steps=1)

在 4% 时，训练停止，程序存在，没有警告或错误。没有输出。

我不知道如何排除故障 - 任何帮助都会很棒。

编辑：将标题从失败更改为 stops/quits 因为我不知道它是否失败

这是我在终端上看到的：纪元：0%| 死亡消耗：0%|

“杀死”一词与迭代一词重叠...也许是内存问题？仅供参考：我是运行它来自 vscode 的终端，在 windows

中的 ubuntu vm 上使用 wsl

在 github 上发现问题： https://github.com/ElderResearch/gpu_docker/issues/38

Answer 1

我的解决方案是将 batch 和 worker 设置为 1，但速度很慢

train_dataloader = DataLoader(train_dataset, shuffle=False, batch_size=1, num_workers=1)

微调期间的 Bert sentence-transformers stops/quits

Bert sentence-transformers stops/quits during fine tuning

python

machine-learning

sentence-similarity

bert-language-model