使用 HuggingFace 和 Tensorflow 与 AutoModel 进行迁移学习不起作用

Question

我尝试使用 HuggingFace 预训练的 BERT 模型进行迁移学习。我想用它来使用 tensorflow API。我不明白为什么最后一行会产生错误

from transformers import AutoTokenizer, AutoModel

model_name = "distilbert-base-uncased"
text = "this is a test"
tokenizer = AutoTokenizer.from_pretrained(model_name)    
text_tensor = tokenizer.encode(text, return_tensors="tf")

model = AutoModel.from_pretrained(model_name).to("cuda")
output = model(text_tensor) # ERROR!!, but why?

Answer 1

您正在混合使用 Tensorflow 和 Pytorch。

使用 TFAutoModel 而不是默认值 (Pytorch) AutoModel

from transformers import AutoTokenizer, TFAutoModel

model_name = "distilbert-base-uncased"
text = "this is a test"
tokenizer = AutoTokenizer.from_pretrained(model_name)    
text_tensor = tokenizer.encode(text, return_tensors="tf")

model = TFAutoModel.from_pretrained(model_name).to("cuda")
output = model(text_tensor)