RuntimeError: The expanded size of the tensor (585) must match the existing size (514) at non-singleton dimension 1
RuntimeError: The expanded size of the tensor (585) must match the existing size (514) at non-singleton dimension 1
我想使用 huggingface 预测数千个句子的情绪。
from transformers import pipeline
model_path = "cardiffnlp/twitter-xlm-roberta-base-sentiment"
pipe = pipeline("sentiment-analysis", model=model_path, tokenizer=model_path)
from datasets import load_dataset
data_files = {
"train": "/content/data_customer.csv"
}
dataset = load_dataset("csv", data_files=data_files)
dataset = dataset.map(lambda examples: dict(pipe(examples['text'])))
但我收到以下错误。
RuntimeError: The expanded size of the tensor (585) must match the existing size (514) at non-singleton dimension 1. Target sizes: [1, 585]. Tensor sizes: [1, 514]
此 post 提出了解决问题的方法，但没有说明如何在管道中解决问题。
The size of tensor a (707) must match the size of tensor b (512) at non-singleton dimension 1
只需在初始化管道时添加分词器参数。
pipe = pipeline("sentiment-analysis", model=model_path, tokenizer=model_path, max_length=512, truncation=True)
我想使用 huggingface 预测数千个句子的情绪。
from transformers import pipeline
model_path = "cardiffnlp/twitter-xlm-roberta-base-sentiment"
pipe = pipeline("sentiment-analysis", model=model_path, tokenizer=model_path)
from datasets import load_dataset
data_files = {
"train": "/content/data_customer.csv"
}
dataset = load_dataset("csv", data_files=data_files)
dataset = dataset.map(lambda examples: dict(pipe(examples['text'])))
但我收到以下错误。
RuntimeError: The expanded size of the tensor (585) must match the existing size (514) at non-singleton dimension 1. Target sizes: [1, 585]. Tensor sizes: [1, 514]
此 post 提出了解决问题的方法，但没有说明如何在管道中解决问题。 The size of tensor a (707) must match the size of tensor b (512) at non-singleton dimension 1
只需在初始化管道时添加分词器参数。
pipe = pipeline("sentiment-analysis", model=model_path, tokenizer=model_path, max_length=512, truncation=True)