TypeError: an integer is required (got type NoneType)

Question

目标：修改此 Notebook 以使用 distilbert-base-uncased 模型

第 1.3 节出现错误。

内核：conda_pytorch_p36。我做了 Restart & 运行 All，并刷新了工作目录中的文件视图。

---

第 1.3 节：

# define the tokenizer
tokenizer = AutoTokenizer.from_pretrained(
        configs.output_dir, do_lower_case=configs.do_lower_case)

回溯：

Evaluating PyTorch full precision accuracy and performance:
/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/data/processors/glue.py:67: FutureWarning: This function will be removed from the library soon, preprocessing should be handled with the  Datasets library. You can have a look at this example script for pointers: https://github.com/huggingface/transformers/blob/master/examples/pytorch/text-classification/run_glue.py
  warnings.warn(DEPRECATION_WARNING.format("function"), FutureWarning)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-31-1f864e3046eb> in <module>
    144 # Evaluate the original FP32 BERT model
    145 print('Evaluating PyTorch full precision accuracy and performance:')
--> 146 time_model_evaluation(model, configs, tokenizer)
    147 
    148 # Evaluate the INT8 BERT model after the dynamic quantization

<ipython-input-31-1f864e3046eb> in time_model_evaluation(model, configs, tokenizer)
    132 def time_model_evaluation(model, configs, tokenizer):
    133     eval_start_time = time.time()
--> 134     result = evaluate(configs, model, tokenizer, prefix="")
    135     eval_end_time = time.time()
    136     eval_duration_time = eval_end_time - eval_start_time

<ipython-input-31-1f864e3046eb> in evaluate(args, model, tokenizer, prefix)
     22     results = {}
     23     for eval_task, eval_output_dir in zip(eval_task_names, eval_outputs_dirs):
---> 24         eval_dataset = load_and_cache_examples(args, eval_task, tokenizer, evaluate=True)
     25 
     26         if not os.path.exists(eval_output_dir) and args.local_rank in [-1, 0]:

<ipython-input-31-1f864e3046eb> in load_and_cache_examples(args, task, tokenizer, evaluate)
    121     all_input_ids = torch.tensor([f.input_ids for f in features], dtype=torch.long)
    122     all_attention_mask = torch.tensor([f.attention_mask for f in features], dtype=torch.long)
--> 123     all_token_type_ids = torch.tensor([f.token_type_ids for f in features], dtype=torch.long)
    124     if output_mode == "classification":
    125         all_labels = torch.tensor([f.label for f in features], dtype=torch.long)

TypeError: an integer is required (got type NoneType)

如果还有什么我可以添加到 post 的，请告诉我。

Answer 1

一位开发人员在此Git Issue解释了这一困境。

Notebook 使用 BERT 进行实验，它使用 token_type_ids.

DistilBERT 不使用 token_type_ids 进行训练。

因此，这将需要重新开发笔记本；专门为此模型移除/调整所有提及 token_type_ids 的内容。