OSError: You seem to have cloned a repository without having git-lfs installed. Please install git-lfs and run git lfs install followed by git lfs pul
OSError: You seem to have cloned a repository without having git-lfs installed. Please install git-lfs and run git lfs install followed by git lfs pul
我在 AWS SageMaker 上使用 Jupyter Labs。
内核：conda_pytorch_p36 并重新启动并 运行 全部。
我git cloned这个repo.
尝试安装 git-lfs:
!curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.rpm.sh | sudo bash
!sudo yum install git-lfs -y
!git lfs install
运行宁 fit lfs fetch 或 git lfs pull 后不更改回溯。
from transformers import AutoTokenizer, AutoModelForSequenceClassification
model = AutoModelForSequenceClassification.from_pretrained('albert-base-v2-MRPC')
回溯：
---------------------------------------------------------------------------
UnpicklingError Traceback (most recent call last)
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
1363 try:
-> 1364 state_dict = torch.load(resolved_archive_file, map_location="cpu")
1365 except Exception as e:
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/serialization.py in load(f, map_location, pickle_module, **pickle_load_args)
592 return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
--> 593 return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
594
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/serialization.py in _legacy_load(f, map_location, pickle_module, **pickle_load_args)
761
--> 762 magic_number = pickle_module.load(f, **pickle_load_args)
763 if magic_number != MAGIC_NUMBER:
UnpicklingError: invalid load key, 'v'.
During handling of the above exception, another exception occurred:
OSError Traceback (most recent call last)
<ipython-input-15-34a92ef6f41b> in <module>
2
3 # load model
----> 4 model = AutoModelForSequenceClassification.from_pretrained(configs.output_dir) # "textattack/albert-base-v2-MRPC"
5 #model = AlbertForSequenceClassification.from_pretrained(configs.output_dir)
6 model.to(configs.device)
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/models/auto/auto_factory.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
439 elif type(config) in cls._model_mapping.keys():
440 model_class = _get_model_class(config, cls._model_mapping)
--> 441 return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
442 raise ValueError(
443 f"Unrecognized configuration class {config.__class__} for this kind of AutoModel: {cls.__name__}.\n"
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
1368 if f.read().startswith("version"):
1369 raise OSError(
-> 1370 "You seem to have cloned a repository without having git-lfs installed. Please install "
1371 "git-lfs and run `git lfs install` followed by `git lfs pull` in the folder "
1372 "you cloned."
OSError: You seem to have cloned a repository without having git-lfs installed. Please install git-lfs and run `git lfs install` followed by `git lfs pull` in the folder you cloned.
---------------------------------------------------------------------------
UnpicklingError Traceback (most recent call last)
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
1363 try:
-> 1364 state_dict = torch.load(resolved_archive_file, map_location="cpu")
1365 except Exception as e:
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/serialization.py in load(f, map_location, pickle_module, **pickle_load_args)
592 return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
--> 593 return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
594
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/serialization.py in _legacy_load(f, map_location, pickle_module, **pickle_load_args)
761
--> 762 magic_number = pickle_module.load(f, **pickle_load_args)
763 if magic_number != MAGIC_NUMBER:
UnpicklingError: invalid load key, 'v'.
During handling of the above exception, another exception occurred:
OSError Traceback (most recent call last)
<ipython-input-15-34a92ef6f41b> in <module>
2
3 # load model
----> 4 model = AutoModelForSequenceClassification.from_pretrained(configs.output_dir) # "textattack/albert-base-v2-MRPC"
5 #model = AlbertForSequenceClassification.from_pretrained(configs.output_dir)
6 model.to(configs.device)
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/models/auto/auto_factory.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
439 elif type(config) in cls._model_mapping.keys():
440 model_class = _get_model_class(config, cls._model_mapping)
--> 441 return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
442 raise ValueError(
443 f"Unrecognized configuration class {config.__class__} for this kind of AutoModel: {cls.__name__}.\n"
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
1368 if f.read().startswith("version"):
1369 raise OSError(
-> 1370 "You seem to have cloned a repository without having git-lfs installed. Please install "
1371 "git-lfs and run `git lfs install` followed by `git lfs pull` in the folder "
1372 "you cloned."
OSError: You seem to have cloned a repository without having git-lfs installed. Please install git-lfs and run `git lfs install` followed by `git lfs pull` in the folder you cloned.
albert-base-v2-MRPC/
config.json log.txt pytorch_model.bin README.md special_tokens_map.json spiece.model tokenizer_config.json train_args.json
如果我还有什么要补充的，请告诉我 post。
我现在在克隆的文件夹中安装并初始化了GIT LFS。
航站楼：
sh-4.2$ git lfs install
Git LFS initialized.
sh-4.2$ git clone https://huggingface.co/textattack/albert-base-v2-MRPC
Cloning into 'albert-base-v2-MRPC'...
remote: Enumerating objects: 27, done.
remote: Counting objects: 100% (27/27), done.
remote: Compressing objects: 100% (25/25), done.
remote: Total 27 (delta 7), reused 0 (delta 0)
Unpacking objects: 100% (27/27), done.
sh-4.2$ cd albert-base-v2-MRPC/
sh-4.2$ git lfs install
Updated git hooks.
Git LFS initialized.
sh-4.2$
对于 Windows 10 及更高版本，只需下载并安装它。然后 运行 命令将起作用。
我在 AWS SageMaker 上使用 Jupyter Labs。
内核：conda_pytorch_p36 并重新启动并 运行 全部。
我git cloned这个repo.
尝试安装 git-lfs:
!curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.rpm.sh | sudo bash
!sudo yum install git-lfs -y
!git lfs install
运行宁 fit lfs fetch 或 git lfs pull 后不更改回溯。
from transformers import AutoTokenizer, AutoModelForSequenceClassification
model = AutoModelForSequenceClassification.from_pretrained('albert-base-v2-MRPC')
回溯：
---------------------------------------------------------------------------
UnpicklingError Traceback (most recent call last)
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
1363 try:
-> 1364 state_dict = torch.load(resolved_archive_file, map_location="cpu")
1365 except Exception as e:
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/serialization.py in load(f, map_location, pickle_module, **pickle_load_args)
592 return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
--> 593 return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
594
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/serialization.py in _legacy_load(f, map_location, pickle_module, **pickle_load_args)
761
--> 762 magic_number = pickle_module.load(f, **pickle_load_args)
763 if magic_number != MAGIC_NUMBER:
UnpicklingError: invalid load key, 'v'.
During handling of the above exception, another exception occurred:
OSError Traceback (most recent call last)
<ipython-input-15-34a92ef6f41b> in <module>
2
3 # load model
----> 4 model = AutoModelForSequenceClassification.from_pretrained(configs.output_dir) # "textattack/albert-base-v2-MRPC"
5 #model = AlbertForSequenceClassification.from_pretrained(configs.output_dir)
6 model.to(configs.device)
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/models/auto/auto_factory.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
439 elif type(config) in cls._model_mapping.keys():
440 model_class = _get_model_class(config, cls._model_mapping)
--> 441 return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
442 raise ValueError(
443 f"Unrecognized configuration class {config.__class__} for this kind of AutoModel: {cls.__name__}.\n"
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
1368 if f.read().startswith("version"):
1369 raise OSError(
-> 1370 "You seem to have cloned a repository without having git-lfs installed. Please install "
1371 "git-lfs and run `git lfs install` followed by `git lfs pull` in the folder "
1372 "you cloned."
OSError: You seem to have cloned a repository without having git-lfs installed. Please install git-lfs and run `git lfs install` followed by `git lfs pull` in the folder you cloned.
---------------------------------------------------------------------------
UnpicklingError Traceback (most recent call last)
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
1363 try:
-> 1364 state_dict = torch.load(resolved_archive_file, map_location="cpu")
1365 except Exception as e:
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/serialization.py in load(f, map_location, pickle_module, **pickle_load_args)
592 return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
--> 593 return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
594
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/serialization.py in _legacy_load(f, map_location, pickle_module, **pickle_load_args)
761
--> 762 magic_number = pickle_module.load(f, **pickle_load_args)
763 if magic_number != MAGIC_NUMBER:
UnpicklingError: invalid load key, 'v'.
During handling of the above exception, another exception occurred:
OSError Traceback (most recent call last)
<ipython-input-15-34a92ef6f41b> in <module>
2
3 # load model
----> 4 model = AutoModelForSequenceClassification.from_pretrained(configs.output_dir) # "textattack/albert-base-v2-MRPC"
5 #model = AlbertForSequenceClassification.from_pretrained(configs.output_dir)
6 model.to(configs.device)
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/models/auto/auto_factory.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
439 elif type(config) in cls._model_mapping.keys():
440 model_class = _get_model_class(config, cls._model_mapping)
--> 441 return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
442 raise ValueError(
443 f"Unrecognized configuration class {config.__class__} for this kind of AutoModel: {cls.__name__}.\n"
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
1368 if f.read().startswith("version"):
1369 raise OSError(
-> 1370 "You seem to have cloned a repository without having git-lfs installed. Please install "
1371 "git-lfs and run `git lfs install` followed by `git lfs pull` in the folder "
1372 "you cloned."
OSError: You seem to have cloned a repository without having git-lfs installed. Please install git-lfs and run `git lfs install` followed by `git lfs pull` in the folder you cloned.
albert-base-v2-MRPC/
config.json log.txt pytorch_model.bin README.md special_tokens_map.json spiece.model tokenizer_config.json train_args.json
如果我还有什么要补充的，请告诉我 post。
我现在在克隆的文件夹中安装并初始化了GIT LFS。
航站楼：
sh-4.2$ git lfs install
Git LFS initialized.
sh-4.2$ git clone https://huggingface.co/textattack/albert-base-v2-MRPC
Cloning into 'albert-base-v2-MRPC'...
remote: Enumerating objects: 27, done.
remote: Counting objects: 100% (27/27), done.
remote: Compressing objects: 100% (25/25), done.
remote: Total 27 (delta 7), reused 0 (delta 0)
Unpacking objects: 100% (27/27), done.
sh-4.2$ cd albert-base-v2-MRPC/
sh-4.2$ git lfs install
Updated git hooks.
Git LFS initialized.
sh-4.2$
对于 Windows 10 及更高版本，只需下载并安装它。然后 运行 命令将起作用。